home | validator

JSON-P/JSONP Validator

JSON is a lightweight data-interchange format. It was formally standardized by Douglas Crockford, and since has been received almost universally as a simple and powerful representation of data for transmission between two entities, regardless of what computer language those entities run in natively.

Cross-domain Ajax -- brief introduction

The same-origin policy in browsers dictates that certain types of data transfer in the browser layer (via JavaScript) must be restricted to only occur if the target resource's domain is identical to the page making the request. This policy is in place in all modern browsers to help protect users from unwanted or unsafe malicious JavaScript behaviors.

Cross-domain Ajax refers to the idea of making requests across domains in opposition to the same-origin restriction. However, cross-domain Ajax is not inherently unsafe or evil -- it is in fact essential to many of the world's most popular and useful mashups. But cross-domain Ajax done properly seeks to make such communication using techniques which, for various reasons, are not subject to the same-origin policy.

JSON-P (JSONP)

One such mechanism which can request content cross-domain is the <script> tag. In December 2005, Bob Ippolito formally proposed JSONP (later dubbed JSON-P, or JSON-with-padding) as a way to leverage this property of <script> tags to be able to request data in the JSON format across domains. JSON-P works by making a <script> element (either in HTML markup or inserted into the DOM via JavaScript), which requests to a remote data service location. The response (the loaded "JavaScript" content) is the name of a function pre-defined on the requesting web page, with the parameter being passed to it being the JSON data being requested. When the script executes, the function is called and passed the JSON data, allowing the requesting page to receive and process the data.

Example:

function handle_data(data) {
   // `data` is now the object representation of the JSON data
}


---
http://some.tld/web/service?callback=handle_data:
---
handle_data({"data_1": "hello world", "data_2": ["the","sun","is","shining"]});

As you can see, the remote web service knew what function name to call based on being told the function name in the URL that made the request. As long as that function is in fact defined in the requesting page, it will be called and passed the data it was asking for.

The problem

Thus far, JSON-P has essentially just been a loose definition by convention, when in reality the browser accepts any abitrary JavaScript in the response. This means that authors who rely on JSON-P for cross-domain Ajax are in fact opening themselves up to potentially just as much mayhem as was attempted to be avoided by implementing the same-origin policy in the first place. For instance, a malicious web service could return a function call for the JSON-P portion, but slip in another set of JavaScript logic that hacks the page into sending back private user's data, etc.

JSON-P is, for that reason, seen by many as an unsafe and hacky approach to cross-domain Ajax, and for good reason. Authors must be diligent to only make such calls to remote web services that they either control or implicitly trust, so as not to subject their users to harm.

Alternatives

There are many alternatives to JSON-P, but each of them has their own drawbacks or challenges. These techniques will not be covered in detail here, except for one: CORS (aka, "Cross-Origin Resource Sharing") -- the most recent addition to browser JavaScript for making cross-domain Ajax calls. Put simply, CORS is an extension to the standard XMLHttpRequest (aka, "XHR") object, which allows the browser to make calls across domains (despite the same-origin restriction). But it does so by first "preflighting" a request to the target server to ask it for permission.

In other words, the remote server is able to opt-in or opt-out of such communication as it sees fit. For instance, a server may expose some content to Ajax requests from only a pre-defined list of approved site domains, and reject all other requests from any other pages. Or, a server may open up its content to be retrieved by any other domain, if it sees fit to do so.

At first glance, it may seem like CORS is an ideal solution for cross-domain Ajax, and makes "hacks" like JSON-P obsolete. Nicholas Zakas recently wrote about CORS for cross-domain Ajax, and gave it a glowing endorsement as the future of cross-domain Ajax in browsers.

Whether CORS as the "standard" will end up being widely implemented and relied upon for broad cross-domain Ajax usage on the web remains to be seen. There are some drawbacks, of course, as the devil is always in the details.

Firstly, CORS requires a server that implements a web-service to implement some non-trivial logic to intercept request-headers in the special "preflight" authorization requests, and to respond with properly formatted response headers depending on the server's intended policy response. If a JSON web-service needs to support cross-domain requests with JSON-P, it's a fairly straightforward change to wrap the JSON piece in a simple function call.

But asking web services across the internet to all implement lower-level logic at their web server layer to send and receive special custom headers is possibly a bit more tricky, depending on what web server software and permissions are in place. It may take several years for this technique to catch on and be to the point where the majority of web service providers is CORS-compliant. As of now, it's very new, and very few web services have done so, as opposed to tens of thousands of existing JSON-P enabled web service endpoints.

Moreover, CORS was implemented (albeit with some slight syntactic differences) only as of IE8, and not yet at all in Opera (as far as is known). So, there's a pretty decent chunk of web visitors whose browser does not support CORS, which means that a fallback for alternative cross-domain Ajax is still going to be necessary for the mid-term (1 to 3 years, perhaps).


The proposed solution

For now, JSON-P is a viable solution for cross-domain Ajax. While CORS may represent a more direct and less hacky way of such communication, it should probably be deployed in tandem with JSON-P techniques, so as to account for browsers and web services which do not support CORS. However, the safety concerns around JSON-P are valid and should be addressed.

So, a stricter subset definition for JSON-P is called for. The following is the proposed "standard" for only what should be considered valid, safe, allowable JSON-P.

functionName({JSON});

obj.functionName({JSON});

obj["function-name"]({JSON});

The intention is that only a single expression (function reference, or object property function reference) can be used for the function ("padding") reference of the JSON-P response, and must be immediately followed by a single ( ) enclosing pair, inside of which must be a strictly valid and parseable JSON object. The function call may optionally be followed by one ; semi-colon. No other content, other than whitespace or valid JavaScript comments, may appear in the JSON-P response, and whitespace and comments must be ignored by the browser JavaScript parser (as would normally be the case).

The most critical piece of this proposal is that browser vendors must begin to enforce this rule for script tags that are receiving JSON-P content, and throw errors (or at least stop processing) on any non-conforming JSON-P content.

In order for the browser to be able to know when it should apply such content-filtering to what might otherwise be seen as regular JavaScript content, the MIME-type "application/json-p" and/or "text/json-p" must be declared on the requesting <script> element. Furthermore, the browser can enforce that the response must be of the matching MIME-type, or fail with errors as well.

Drawbacks

It is fully known that existing browsers which don't support CORS also will not likely be updated to support this JSON-P strict enforcement, which means that users in those browsers will not be protected by their browser. However, all current browsers can add support for this content filtering, which would provide safer JSON-P for current browser users who are consuming data from web services which do not yet support CORS (or for which the author does not want to use CORS for whatever reason).

It's also recognized that this stricter definition may cause some "JSON-P" transmissions, which rely on the looser interpretation of just arbitrary JavaScript content, to fail. But this could easily be avoided by having the author (and the server) avoid referring to that content with the strictly defined JSON-P MIME-types as described above, which would then prevent the browser from selectively turning on such filtering.

Side note

One possible mitigation technique which could help users of older browsers still take advantage of JSON-P safety would be for the author to detect browsers without such support (up for discussion how this feature-test might work) and conditionally route such requests only for those unsafe browsers through a local server-proxy (or even through a client-side flash proxy like flXHR), which could act as a gateway to the content and do the filtering logic described above.

An important side-effect of codifying a standard for JSON-P is that a parser implementation can (and will) be written to check against the strict definition. Such a parser could easily be used in the local server-proxy or the client-side flash proxy to filter unsafe/invalid content that should have been strict JSON-P.

Alternate Use Case: Unit-testing

Another interesting use-case for having a standard for JSON-P that a parser can be written to is the notion of being able to unit-test APIs which produce JSON-P output. For instance, such testing is common for JSON APIs, using something like JSONLint, but for JSON-P APIs at the moment, there's no good way to pump the output through some parser to validate it.

Moving Forward

This is a first-pass at such a definition for safe JSON-P. I hereby open up the discussion to the community at large to weigh in on the pros and cons and to collaborate together to find a workable definition that can be advocated for to the W3C, WHATWG, and browser vendors.

The best way to get involved is to write blog posts in response to this proposal, and create links to and from such discussions. In addition, this site's markup/code is hosted on github, and you can fork and modify the page to continue the discussion, and then send a pull request to have the site updated. Lastly, you can make short comments in the comment form below, but please keep your comments brief and to the point, as they will be moderated as necessary to keep the discussion on track.


Comments



Comments are temporarily disabled
(name)*
(email -- not displayed)*
(url)


* required

This site is maintained and hosted by Kyle Simpson and Getify Solutions. The site and all its contents are hereby released as public-domain. Stylization choices are deliberately inspired/copied from Douglas Crockford's JSON.org site, given the obvious relationship of the content. Thanks to @hij1nx for the JSON-P logo, obviously also derivative of Douglas Crockford's JSON logo.

This entire site's code is on github, so please feel free to fork it and improve!