Monday, October 30, 2006

Designing video applications for 3G users

Several operators in Asia have already launched some form of 3G service and many are going to do so in the near future. A common theme for all of them was the focus on video capabilities as the main driver for why users should adopt 3G and have a 3G mobile phone.

It seems to me that a lot of developers have simply forgotten that using a video application from a 3G terminal is still a visual activity that requires the content to be somewhat pleasant to the eye. Why would a user watch (for more than 1 minutes) a poorly designed video application offering an interactive menu to some streaming video content or a video blog or whatever similar video content that can be delivered to 3G mobile users?

3G mobile screens are still way too small compared to a laptop display or a TV! That requires in my opinion even more care and attention to produce clear and eye catching content for those small displays. But what I still don't get is why are most of the applications currently available still so poorly designed from a visual point of view: bitmapped fonts used to render text over the video screens which do not scale properly when the video size changes, poor color choice that does not make it easy for the user to read the text for the menus, poor use of the scarce real estate offered by the handset small display,...

I really hope that some of those developers will start soon thinking about some of the following methods to improve the visual quality of the video content served to mobile users and to make their experience more attaching:
  • Do not use static menus that are already integrated into the video content. Rather than doing that, use dynamic menu generation and rendering overlayed over the video content so that it can be adapted to the display size (font size, less options in one screen,...).
  • Consider different options when organizing the display: menu overlay with transparency vs. structured layout of the display.
  • Use a professional designer like we already do for web applications, TV broadcasting,...
  • Allow for personalization and customization (skins, theme,...) of the video screens to adjust to different user populations and different tastes.

Friday, October 27, 2006

On Internet Explorer and JavaScript memory leaks

I have recently been working on an AJAX application used to manage and monitor a system part of one of the solutions we are creating. Asynchronous events and near realtime updates to the management GUI are nice features to have in such application. I wasted a day or so tracking a memory leak (or apparently so) in the web application client GUI. I thought that it may be useful to share some of the experience and lessons with whom might be interested.

AJAX applications by design rely heavily on dynamic HTML and manipulation of the DOM using JavaScript, so it is not uncommon to see a large amount of the application being written as a single web page with a lot of scripts. This model is fundamentally different from the traditional web browsing experience where each piece of information or user action/result of that action is presented in a new web page obtained from the server. With the old web browsing experience, it is pretty common to open a browser and close it in less than 5 minutes. With AJAX applications however, as the whole application is presented within one single web page, this old model is no longer valid. Memory leaks in web pages, due to whatever broken browser or broken application script, do not really bother the user browsing the web the old way. They do however manifest quickly and in a noticeable way when using a moderately complex AJAX application. In my case, I could get Internet Explorer reaching 120 Mb in less than 5 minutes. That was pretty big and pretty unacceptable, so I decided to track the origin of that leak and fix it.

Understanding where memory leaks in JavaScript come from

It seems that the most usual source of memory leaks in JavaScript comes from the use of closures and event handlers attached to DOM nodes. Basically, most of the asynchronous processing in the application GUI is done by setting asynchronous event handlers on particular events that happen to particular nodes in the DOM. The event handlers are simply JavaScript functions, which could be declared as named JavaScript functions or anonymous functions within closures. The issue with closures is that a closure sees and can use all variables defined in the parent cope (the scope within which the closure is defined), which is a good thing as it simplifies a lot the coding, and a bad thing as those closures now keep a reference to all those variables defined within the parent scope. This situation may result in what is called circular references which cause memory leaks: A DOM node references a closure (which is a JavaScript function, i.e. a JavaScript object) which itself references the DOM node.
A good article explaining those circular reference issues with JavaScript closures has been written by Richard Cornford. You can also have a look at this page by James Mc Parlane which could be easier to understand than Mr Cornford's detailed study on JavaScript closures.
Closures are definitely the main source of memory leaks (in my experience) but are not the only source. Other sources of memory leaks include keeping references to temporary variables in global JavaScript variables (totally useless and very bad programming habit) as well as broken browser implementations like Internet Explorer .

How do I track that leak?

There is a serious lack of JavaScript development supporting tools that are ready for the explosion of AJAX applications and dynamic web pages. There is no memory profiling tools, no JavaScript heap walking tool, not even a simple dumper of the JavaScript heap which produces a readable format. Even mozilla and firefox lack such extensions... To be honest and fair with IE, the only decent tool I found that could help a little bit in tacking the leak source is an IE only tool called Drip.
Another useful tool that does not tell where your memory leak is, but at least can help you see if you have a memory leak or not is SysInternals free Process Explorer tool. Because it can accurately show the process memory usage, it can help in watching if the browser memory increases over time while using and reusing the application JavaScript code.
Of course, you need to have two or three running web browsers to compare and see (only guesswork) if the leak is really coming from your application code or from the broken browser. I usually run with mozilla or firefox or both and IE 6 or IE 7 or both.

Fixing the code won't always make the leak disappear at once

After reviewing all my JavaScript files, preparing memory stress tests and all, I basically managed to eliminate the leaks when using browsers like firefox or mozilla, but still see Internet Explorer memory growing as I use the application and run the test cases. It is absolutely frustrating and time consuming to find why this continues to happen with IE. At one point in time I almost commented out all the code of one of the JavaScript classes that was causing leaks in IE, and run the test case with the minimum required code to have at least some form of the object being used in the browser. The amazing thing is that the leak was still there. I just became crazy with IE, trying to figure out why would a DIV added to a parent DIV then removed cause a memory leak. There were no event handlers, no closures, nothing!
After some googling, I just came across some page recommending not to use removeChild DOM method on a DOM node with IE as it causes some "pseudo-leaks". Now this is a really dumb implementation of the most popular web browser in the world. Why would a method that is documented as removing a DOM node from the DOM tree not make that object garbage collectable and clean it up? Well, not in the IE world! All DOM nodes removed from the DOM tree with removeChild calls stay there until the page is unloaded. I do not know why, but I know for sure that if you reload the page or load another page, the memory is claimed back.

In conclusion, I could not manage to get IE memory not to grow constantly while using the application, but I decided that it is not worth to artificially reload the page from time to time to get rid of that growing memory with IE. It was better to keep a design of the AJAX application that works with most non-broken browsers and requires a page reload after a 100 uses of the application with IE.

Anyway, I was disappointed not to find an ultimate solution for this problem, but I understood quickly (thanks to the many other people who faced the same issue) that there is no such solution with IE .