engin aydogan

It looks like my previous post about the browsers sending OPTIONS request instead of GET has nothing to do with Dojo, which got quite obvious as I saw Prototype is also behaving the same way. I’ve researched about the topic and here’s my insights.

It turned out that some new specifications were implemented in IE8, Safari 4, FF 3.5 and Chrome which allows you to do cross-domain XHR. Which means the pure JS implementation I have demonstrated wasn’t supposed to work at all unless this new specification was implemented. Here’s what the old XHR spec has to say about cross-domain (cross-origin) requests. Taken from http://www.w3.org/TR/XMLHttpRequest/#the-open-method

If the origin of url is not same origin with the XMLHttpRequest origin the user agent should raise a SECURITY_ERR exception and terminate these steps.

Not allowing cross-domain XHR was and is really a deal breaker and actually it pretty much stops you from implementing SOA (service oriented architectures) flexibly. But for some good reasons.

Here are a few theoretical scenarios:

1. Imagine you are visiting attacker.com which serves a script that requests bank.com/?action=money_transfer&to=attacker&amount=999999. Assuming you have an active session with the bank, if your browser sends this request to bank.com along with the session cookie, attacker would be able to transfer money to himself. This is called CSRF (Cross-site request forgery)
2. Imagine you are visiting attacker.com which serves a script that requests 10.0.0.50/confidential_intranet_document.html and sends it to himself via script. This means any client in the trusted LAN network might leak information from the LAN to internet.
3. Imagine you are visiting trusted.com which happens to have a security hole so that the attacker can inject malicious code in its web pages. For instance, imagine you could embed Javascript in the messages in Facebook. When other users see that message and the Javascript code you injected works on their browser, you could read their cookies, hence steal Facebook session. This is called XSS (cross-site scripting).

Though there are other transport mechanisms, such as <script> element which is not restricted by this Same Origin Policy. These mechanisms were used instead of the obvious XHR method to achieve cross-domain requests so far.

There is a new specification being drafted to address these issues,  http://www.w3.org/TR/access-control/ which is the reason why OPTIONS request was being sent instead of GET in my previous post. The new spec says that it is OK to send a simple request (which is defined as GET, HEAD and POST) cross-domain as long as there’s no custom header in it. If these conditions are not met, there should be a preflight request to ensure that the domain we’re requesting the document from allows us to fetch it — much like Flash’s policy file.

Not the custom headers clause above. That’s the exact reason why Prototype and Dojo was causing an OPTIONS request instead of GET, where regular JS was simply sending GET request. Dojo and Prototype adds custom headers to the requests.

So you might ask; cross-domain XHR was not allowed for a good reason, why is it being allowed now ?

Yes, cross-domain XHR is allowed now, but apparently no different than cross-domain requests you can send via img or script elements. Remember that you could always do cross-domain requests with img element too, but img element has two features that makes it not a security problem:

1. img only can send the cookies for the domain it is loaded from. i.e. it is hard to use a remote session since it won’t send the target site’s cookie.
   Consider the first scenario above. If the request does not include a cookie for the bank.com, there’ll be no session. It will be a anonymous request. (Of course unless the target site uses session ID as a part of the URL, and the attacker got that SID, which is very unlikely. And if he has the SID he’ll hijack your session all together anyway).
2. You cannot read the contents of an img element, hence you cannot steal sensitive information which you aren’t supposed to read.

Now, I have demonstrated myself in my previous post that cross-domain XHR worked out fine. My server received the GET request. BUT in the client xhr.responseText was empty and xhr.status was 0 (not 200). It is true that the request was actually made, but you cannot read the contents of the resource. Here’s what access-control spec says about this in http://www.w3.org/TR/access-control/#requirements

1. Should not allow loading and exposing of resources from 3^rd party servers without explicit consent of these servers as such resources can contain sensitive information.

One of the requirements of the spec is not to expose resources without explicit consent. From what I understand, here, explicit consent means Access-Control-Allow-Origin header. If the third party server allows other hosts to read its resources via this header, everything will be fine. So, this means that the new XHR is no security hole bigger than the IMG itself.

In fact, I’ve tested this. It turns out that when you add this header to your resource, cross-domain XHR starts to work to the fullest. i.e. you can read the content of the requested resource, as in, it is readable in xhr.responseText.

For your information, you can add any headers to your resources with mod_header module of Apache httpd. Just add this directive for whatever directory you want;

Header set Access-Control-Allow-Origin "*"

Keep in mind that, this will expose all of your resources in that directory for anyone to read. So, do this if your resources are public anyway. Or just allow the hosts you want. It could be better to do this in the programming layer, such as PHP or ASP.NET.

So in conclusion, with the new access-control spec, XHR is pretty similar to the Flash’s security design. Browser checks if the third party host allows you to read your resources, if so your script is allowed to read it. Note that you can make the request anyway, but reading the resource is not allowed.

This is a nice step forward actually, but since it will take some time that majority of the market is using browsers implement this new spec, web developers are bound to use iframe or script transports for cross-domain request.

I started implementing a daemon in Java. Essentially all our devices will connect to it and wait for commands over TCP/IP. Additionally it will offer a web service REST API over HTTP, so that administrators can send/receive data from the devices. This is basically a relaying architecture between devices and administrators. It overcomes any network topology problem (i.e. NAT traversals.) with performance penalty and bandwidth costs of relaying. Web service API is going to be totally self-sufficient, such that totally static HTML pages with Javascript can interact with it.

I’ve implemented the skeleton of the daemon, devices connect to it and it also offers web service API. I tested the web service API by simply requesting the web service URL from the browser as I would do any other URL and confirmed that the correct (JSON) response is given. It was time to see how it is to build a web UI for it. Given its reputation and apparent support, I chose Dojo to implement the web UI with totally static HTML pages. Here’s my experience.

Documentation

From the main Dojo site, it was stated that latest stable release was 1.4.0, and it was the default download link. So I grabbed it. By looking at an example in the demo section, I get an Ajax query working in seconds, only to find out that it is not working for me. Instead of a GET request it was sending OPTIONS request, more on this later. Obviously, I wanted to look at the documentation. Clicking on the Documentation link on the main Dojo site takes you to a place where documentation for 1.4.0 is not offered.

Luckily there were a handful of helpful folks in #dojo @ irc.freenode.net, whom told me that new documentation UI is on its way.

• http://api-staging.dojotoolkit.org/
• http://doc-staging.dojocampus.org/dojo/index.html

The first one was inaccurate by the time I’m writing this Dojo.xhrGet property list was quite short. I found doc-staging to be more accurate and since it is documentation rather than just reference it also offered much more detail.

Dojo.xhrGet results in OPTIONS request instead of GET

The firs thing I’ve tried with Dojo was obviously the Ajax API.

function getText() {
  dojo.xhrGet({
    url: "http://localhost:8182/hello",
    load: function(responseObject, ioArgs){
      return responseObject;
    },
    error: function(response, ioArgs){
      dojo.byId("toBeReplaced").innerHTML =
        "An error occurred, with response: " + response;
      return response;
    },
    handleAs: "json"
  });
}

This code snippet is taken from Dojo examples which can be found in the official web site. I removed the content of the first function though, it was supposed to do DOM manipulation obviously.

When I run this code, I noticed OPTIONS request in my daemon’s logs. When I was requesting the same URL by writing it to the address bar of the browser all I see was GET requests in my logs, as expected.

Then I’ve tried a pure JS implementation.

var client = new XMLHttpRequest();
client.onreadystatechange = handler;
client.open("GET", "http://192.168.1.94:8182/hello");
client.send();

With this simple implementation I started seeing GET requests in my server as expected. So Dojo should be doing something in a different way. I walked through the Dojo code, thanks to Firebug.  But it turned out that the code is indeed very similar to the regular JS code and there were no obvious bugs, as I expected. Then, I examined the HTTP requests via invaluable Wireshark.

Here’s what I got for Dojo request.

OPTIONS /hello HTTP/1.1
Host: 192.168.1.94:8182
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.49 Safari/532.5
Cache-Control: max-age=0
Access-Control-Request-Method: POST
Origin: file://
Access-Control-Request-Headers: X-Prototype-Version, X-Requested-With, Content-type, Accept
Accept: */*
Accept-Encoding: gzip,deflate
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-9,utf-8;q=0.7,*;q=0.3

And here’s what a regular JS XHR looks like.

GET /hello HTTP/1.1
Host: 192.168.1.94:8182
Connection: keep-alive
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-US) AppleWebKit/532.5 (KHTML, like Gecko) Chrome/4.0.249.49 Safari/532.5
Cache-Control: max-age=0
Origin: file://
Accept: */*
Accept-Encoding: gzip,deflate
Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.6,en;q=0.4
Accept-Charset: ISO-8859-9,utf-8;q=0.7,*;q=0.3

Obviously the difference is Access-Control-* properties. I tracked down the source of this was the lines from 10474 to 10477 of dojo.js

 // FIXME: is this appropriate for all content types?
10474 xhr.setRequestHeader("Content-Type", args.contentType || _defaultContentType);
10475 if(!args.headers || !("X-Requested-With" in args.headers)){
10476 xhr.setRequestHeader("X-Requested-With", "XMLHttpRequest");
10477 }

Funny thing, there’s a FIXME there :) Anyway, when I commented out these lines the request headers were the same for both pure JS Ajax implementation and the Dojo Ajax implementation. And both are now sending GET requests as expected.

I consulted the folks at #dojo about this problem and at first they couldn’t create the problem. Than I’ve stated that the web UI is hosted at X and the web service API of the deamon was at X:P. So it is a cross-domain request. To be precise the script was hosted at http://192.168.1.94/test.html and web service API of the daemon was available at http://192.168.1.94:8182/hello. sfoster from #dojo generously spent some time to test this situation and he also confirmed that OPTIONS requests were being sent. Apparently when Access-Control-* headers are set and it is a cross-domain request browsers decide to send OPTIONS request instead of GET. This is tested with Chrome and Firefox.

Though I believe Access-Control-* properties are there for a good reason. This same problem could also be demonstrated on prototype javascript framework, apparently they are taking the same approach on this.

I’m not sure what is the best practice on this yet, I’ll try to consult some core Dojo developers about this and figure it out.