engin aydogan

Chrome UTF-8 bug

engin — Wed, 28 Jul 2010 06:27:31 +0000

I cannot believe what I’m seeing. UTF-8 implementations failed miserably.

There’s a character in Turkish alphabet, it is capital “i”, just like “I” but with a dot on top of it. Why am I describing it instead of just showing it ? Well, WordPress also seems to be unable to handle it. From now on, [ci] refers to this special character in this post.

Chrome bug

toLowerCase() of Chrome 5.0.375.99 is buggy. If a string contains [ci], the resulting string has artifacts. For instance assume you have the original string “[ci]ello”, which is 5 characters long. When you feed it to toLowerCase() the resulting string will be 6 characters long with a “garbage” data after the converted [ci]. The results is something like i\xxxello. There’s a garbage after “i” which you can see via charCodeAt().

Firefox is plagued too

Then I tried in-case-sensitive RegExp matching. Trying to match “[ci]ello” with “/iello/i” failed both on Chrome 5.0.375.99 and Firefox 3.6.8.

Internet Explorer 8 kicked ass or what ?

Even though I, very objectively, hate IE series too I should note that IE8′s toLowerCase and RegExp matching works perfectly. In your face, all Microsoft haters :)

What about toUpperCase() ?

On all browsers toUpperCase converts “i” to “I” and not to “[ci]” and they should, since the context (language) is not known and English is naturally assumed to be the one in use, converting “i” to “I” is perfectly acceptable. Though it doesn’t make it right. In Turkish toUpperCase(“i”) should return “[ci]“. Which makes Javascript look inadequate at handling internationalization.

zero-configuration connectivity on embedded computers

engin — Tue, 20 Jul 2010 10:33:39 +0000

Our device is networked. It is supposed to:

Work when you plug it into any traditional Ethernet network (i.e. with router with embedded DHCP server).
It is also supposed to work when you directly connect it to another node (i.e. a computer which you’re want to configure the device with — where no DHCP server is available).

So our goal is:

Zero-configuration connectivity on above two scenarios.

Our requirement:

Severely scarce resources. Firmware should fit in a 64kb space.

Most such networked devices comes with a static IP already set and the node which is supposed to configure them (usually a PC) is supposed to be set to a specific IP address in the same subnet to configure them. This has several drawbacks:

Users need to have administration privilleges
Users need to have experience with computers
And this is OS dependent.

There are other zero-configuration technology stacks, even though I don’t have a scientific fact, from their specs I think they’d take up significant code space.

So here’s what I’ve did

During device boot I assign it a link-local IP address (i.e. AUTOIP or http://tools.ietf.org/html/rfc3927). I did not implement RFC3927 and just generated a fixed IP address according to MAC of the device — partly because I don’t have a quality random number generator, as there’s no clock on most of the devices and reading analog inputs often give same results.
Then I start DHCP negotiations

With the scenario the device is able to communicate with another node which also have been assigned to a link-local address without waiting for a DHCP timeout. So when you plug your device directly to your computer you can communicate with the device as soon as your computer also assigned itself a link-local address. This varies from OS to OS but we’re pleased with the results. Our customers are able to connect to their devices without any configuration.

But there’s a problem! Even though this configuration worked on our network at the office and many other places where we sold the devices to, it didn’t work on my home network. It turned out that my linksys router in my home network does not honor DHCP DISCOVER requests if their source IP addresses are not 0.0.0.0

Problem:

Some DHCP servers does not honor DHCP DISCOVER requests from 169.254.C.D. They strictly require source IP of the requests to be 0.0.0.0

To solve this problem I could either

First assign my device IP address 0.0.0.0 and then wait for DHCP timeout, then fallback to AUTO IP. This way DHCP DISCOVER request would work with all routers but other party would have to wait for my device to timeout DHCP and fallback to link-local IP before starting communicating with it — this is a serious tradeoff.
Or Add UDP IP Spoofing on top of our current implementation hence just make the DHCP DISCOVER source IP 0.0.0.0 and keep the rest the same.

Obviously I chose the second approach and patched uIP with the following code:

$ svn diff
Index: uip/uip.c
===================================================================
--- uip/uip.c   (revision 226)
+++ uip/uip.c   (working copy)
@@ -1181,7 +1181,17 @@
   BUF->srcport  = uip_udp_conn->lport;
   BUF->destport = uip_udp_conn->rport;

-  uip_ipaddr_copy(BUF->srcipaddr, uip_hostaddr);
+  extern int g_udp_spoof;
+  extern uip_ipaddr_t g_udp_spoof_ip;
+  if(!g_udp_spoof)
+  {
+    uip_ipaddr_copy(BUF->srcipaddr, uip_hostaddr);
+  }
+  else
+  {
+       g_udp_spoof = 0;
+       uip_ipaddr_copy(BUF->srcipaddr, g_udp_spoof_ip);
+  }
   uip_ipaddr_copy(BUF->destipaddr, uip_udp_conn->ripaddr);

   uip_appdata = &uip_buf[UIP_LLH_LEN + UIP_IPTCPH_LEN];

Now our device apparently has best of both worlds.

It can communicate with any node with link-local IP address right away at boot time — without waiting for any sort of (DHCP) timeout.
While link-local communication is working, it tries to acquire an IP address from DHCP server in parallel — thanks to source IP “spoofing”.

I’ll see if further testing will reveal any side effects.

See also:
DHCP RFC: http://tools.ietf.org/html/rfc2131
Link-local IP Address (AUTO IP): http://tools.ietf.org/html/rfc3927

ExternalInterface performance on different browsers

engin — Thu, 15 Apr 2010 18:13:06 +0000

ExternalInterface is the API provided by Flash to communicate with its hosting environment. For my case, it is the browsers hosting the Flash “movie”. I noticed different performance behavior on each browser. This is my attempt to measure performance penalty caused by each browser.

Here’s the software versions:

Flash plugin version: 10,0,32,18

Chrome 4.1.249
Firefox 3.6.3
IE 8.0

How did I measure

1. I call Been.FComm.send() function in Javascript and timestamp (T_js1) it. This JS function calls the send() in the Flash.

2. In the Flash application I timestamp (T_f1) send() function call.

3. In the Flash application I also timestamp (T_f2) handleData() and call been_fcomm_handle JS function.

4. Then I timestamp (T_js2) been_fcomm_handle() in Javascript.

Flash application is written in Flex. It sends a data over a TCP socket and reads the response back. As all functions, send() and handleData() functions in my application also has a debug() call — which includes a timestamp.

Here’s how I implemented measurement in Flash:

private function debug(str:String):void
{
	var date:Date = new Date();
	output.appendText(date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds() + "." + date.getMilliseconds() + ": " + str);
}

private function handleData(event:ProgressEvent):void
{
	var data:String = m_socket.readUTF();
	debug("handleData"+data+"\n");
	ExternalInterface.call("been_fcomm_handle", data);
}

private function send (value:String):void
{
	debug("sending "+value+"\n");
	m_socket.writeUTF(value);
	m_socket.flush();
}

So that I can measure how long Flash thinks it takes my server to give a response.

On the other hand, I also measure the timings in JS layer. Here’s the related code pieces — simplified to death to only contain relevant codes.

Here’s how I implemented measurement in Javascript:

Been.Debug = new function() {
    this.info = function(m) {
        log(date.toLocaleTimeString() + "." + date.getUTCMilliseconds() + ": " + m);
    };
};

Been.FComm = new function() {
    this.handle = function(msg) {
        Been.Debug.info("Been.FComm.Handle: " + msg);
    }

    this.send = function(data) {
        Been.Debug.info("Been.FComm.send: " + data);
        Been.Utils.getFlashMovieObject("FlashComm").send(data);
    }
    /* rest of the implementation */
};

Results

Here are the results samples.

Chrome

Avarage Flash request/response: 8.3ms
Avarage JS request/response: 161ms
Flash JS communication (marshaling) overhead: 161 - 8.3 = 152.7ms

Flash output
---------------------------------------------------------------------------------------
20:34:20.731: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:20.719: sending {asf}
20:34:19.519: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:19.513: sending {asf}
20:34:9.713: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:9.706: sending {asf}

Javascript output
---------------------------------------------------------------------------------------
20:34:20.990: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:20.720: Been.FComm.send: {asf}
20:34:19.710: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:19.512: Been.FComm.send: {asf}
20:34:09.718: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:34:09.702: Been.FComm.send: {asf}

Firefox

Avarage Flash request/response: 76ms
Avarage JS request/response: 77.6ms
Flash JS communication (marshaling) overhead: 77.6 - 76 = 1.6ms

Flash output
---------------------------------------------------------------------------------------
20:35:55.385: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:55.276: sending {asf}
20:35:54.582: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:54.506: sending {asf}
20:35:53.687: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:53.644: sending {asf}

Javascript output
---------------------------------------------------------------------------------------
20:35:55.414: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:55.303: Been.FComm.send: {asf}
20:35:54.610: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:54.533: Been.FComm.send: {asf}
20:35:53.715: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:35:53.670: Been.FComm.send: {asf}

IE

Avarage Flash request/response: 5.3ms
Avarage JS request/response: 7.6ms
Flash JS communication (marshaling) overhead:  7.6 - 5.3 = 1.3ms

Flash output
---------------------------------------------------------------------------------------
20:36:51.166: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:51.162: sending {asf}
20:36:48.211: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:48.207: sending {asf}
20:36:47.467: handleData{"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:47.459: sending {asf}

Javascript output
---------------------------------------------------------------------------------------
20:36:51.166: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:51.161: Been.FComm.send: {asf}
20:36:48.211: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:48.206: Been.FComm.send: {asf}
20:36:47.469: Been.FComm.Handle: {"Status":"FAILED","Error":"Internal server error.","ErrorNo":500}
20:36:47.456: Been.FComm.send: {asf}

Summary

Browser	Flash – JS communication overhead (ms)
Chrome	152.7
IE	1.3
FF	1.6

It looks like Chrome suffers from the most significant overhead, while IE and FF could be considered on a par.

Cross-domain data-push methods compared

engin — Wed, 31 Mar 2010 12:25:51 +0000

I’ve been playing with cross-domain (XD) data-pushing techniques lately and now I’ll try to compare each available technique. These techniques allows server to push messages to the client. A brief summary of comparison is available in the table below, following the table you can find more detailed explanation for each technique.

Technology	Reading Data	Writing Data (HTTP GET)	Writing Data (HTTP POST)	Advantages	Disadvantages	Conclusion
XHR	No	Yes	No	XHR is good at error handling	Only very recent browsers implementing access-control specs allow requests to be made. Cannot read responses at all Data writing is limited to 2kb	FAIL Because: a cross browser method with XHR for XD communication does NOT exist.
Script	Yes	Yes	No	Can read the responses Easy to implement	Hard to detect connection problems Data writing is limited to 2kb Long-polling is required. If remote site is not trusted, can run arbitrary code on your site.	Works
IFRAME + hack	Yes	Yes	No	Can read responses Remote sites response can be filtered by your proxy program (i.e. php) on the origin, hence a not trusted remote site can be used.	A relaying/proxying dynamic program (i.e. php) is necessary on the domain the request is originating from. Long polling is required.	Works
IFRAME + form	No	Yes	Yes	Can send large data	Cannot read responses “Start navigation” (aka click) sound of IE when you do form.submit() is really a sad deal breaker.	FAIL Because: see Disadvantages
IFRAME + form + hack	Yes	Yes	Yes	Can send large data	“Start navigation” (aka click) sound of IE when you do form.submit() (NOTE: This really is a deal breaker, funny fun though)	FAIL Because: see Disadvantages
Flash	Yes	Yes	Yes	A true persistent TCP connection ready to be used. Good error handling Can deal with binary protocols, hence can save bandwidth around 90%	Note that I’ve tried very very hard to find disadvantages of this method. Plug-in availability debate. Not available in mobile platforms Slim chance of network problems	Works
Yes: Able to send request and read the response. i.e. Being able to send a request to http://foo.com/write.php?data=bar, which writes data “bar”, and in return being able to read the response (such as “Write Successful” or “Write Fail” Yes: Able to send request but cannot read the response No: Cannot send request nor response.

Introduction

I’ve studied cross domain communication as it is two separate tasks; one is reading other one is writing.

Available Methods

1. XHR

XHR is the way to go with same-origin requests (i.e. non-XD), unfortunately it does not with with XD requests. So far browsers would simply throw an exception when you tried to do a XD request via XHR, which is defined by the standard.

If the origin of url is not same origin with the XMLHttpRequest origin the user agent should raise a SECURITY_ERR exception and terminate these steps.

A new standard is worked out called access-control which now enables you to do XD XHR requests. Essentially it is what Flash’s policy file is to the XHR. XHR is only allowed when the target domain explicitly allows your domain.

Should not allow loading and exposing of resources from 3rd party servers without explicit consent of these servers as such resources can contain sensitive information.

You can find more detailed information about the reasons of these decisions in my previous post.

So what this means is;

Only if a very new browser with the new specs implement is being used,
- and target domain explicitly allows the originating domain then XHR can be used for XD communication.
- and if target domain does NOT explicitly allows the originating domain, only simple requests such as GET will be send to the target domain, and response will not be readable by the script.
If a browser with new specs implement is NOT being used, then a security exception will be thrown right away.

Conclusion: FAIL.

Because a cross browser method with XHR for XD communication does NOT exist.

2. Script

Script is one of the elements which are not restricted by the same-origin policy (SOP). Script is abused in such a manner that it allows us to do XD communication. Below figure demonstrates how script method works.

Here’s what’s going on in the above figure:

origin.com tries to load a JS script from remote.com/service.php
as the script from remote.com loads, it immediately calls a message handler in origin.com
hence remote.com passed information to origin.com which is an actual XD communication.

Here’s what the mysterious $.getScript does roughly;

Create a

engin aydogan

Chrome UTF-8 bug

zero-configuration connectivity on embedded computers

ExternalInterface performance on different browsers

Here’s the software versions:

How did I measure

Results

Summary

Cross-domain data-push methods compared

FAIL

Works

Works

FAIL

FAIL

Works

Introduction

Available Methods

1. XHR

Conclusion: FAIL.

2. Script