JSLitmus – A Tool For Testing JavaScript Performance

November 7th, 2008 by Robert · comments · http://blog.zenbe.com/etfav

Zenbe’s webmail application makes heavy use of  JavaScript to provide a highly-interactive experience, so naturally JavaScript performance is an important consideration for us.  While we do use the handful of tools available for profiling JavaScript – Firebug’s “profile” feature being the most notable – often the best way to hone in on a particular issue or area is to write a small, dedicated test that allows a specific code pattern to be quickly evaluated and tuned on all of the platforms we have to support.

There are a number of bugaboos with ad-hoc tests however. Generating good timing data is tricky, as is formatting the results to get a useful report.  To take the pain out of this, we’ve been using a script we wrote in-house a while back, and that we’re now officially releasing as the “JSLitmus” tool.  JSLitmus is designed specifically to allow you to quickly and easily write a JavaScript test (or test suite), run it on any modern browser, and document and share the results.  You can find out more about it here, but to give you a feel for what it’s used for, let me walk you through an issue you may be familiar with: variable scope as it pertains to performance.

It’s fairly common knowledge that JavaScript performance is affected by the scope in which variables are defined. Variables defined locally within a function are faster than global variables or variables defined in closure scopes.  But what does that really mean?  When do we need to care about this?  How much faster are they?  Does it matter which browser?  How does the difference compare to, say, the overhead of a function call?

To answer these questions, I wrote up a simple test suite using JSLitmus, the results of which are shown in the graphs below.  Each graph shows the number of times per second a particular browser was able to perform a “++” operation on variables in different scopes.  The performance of an empty function call is also shown for reference.  Longer bars are better.

 JSLitmus   A Tool For Testing JavaScript Performance

 JSLitmus   A Tool For Testing JavaScript Performance

 JSLitmus   A Tool For Testing JavaScript Performance

 JSLitmus   A Tool For Testing JavaScript Performance

[All tests performed on a 2GHz dual-core MacBook in MacOSX or Vista. You can view the test code, or go to the JSLitmus home page to run the test for yourself - it's on the right-hand side.]

With these results, we can make some interesting observations that may prove useful as we optimize our code:

  • Local variables are faster, as expected, but the difference between local and global vars is surprisingly high: 5x-25x.
  • We can speculate that the top three browsers use similar variable resolution logic since the relative performance within each graph is similar.  Opera’s exceptional performance with closure variables makes it unique.  This has interesting ramifications for libraries that make heavy use of closures, such as Prototype and jQuery.
  • Roughly speaking, 2-4 references to a global or closure variable are as expensive as an empty function call – with IE and it’s markedly poor function call performance being the exception.

As you can see, JSLitmus lets us quickly gain a much better insight into these kinds of performance issues, with only a modicum of effort.  Enjoy, and happy testing!

9 Responses to “JSLitmus – A Tool For Testing JavaScript Performance”


  1. [...] Kieffer has announced JSLitmus a tool “designed specifically to allow you to quickly and easily write a JavaScript test [...]

  2. Cloudream says:

    Google Chrome:

    Test Operations Per Second
    global ∞
    local ∞
    closure ∞
    multi-closure ∞
    empty function call ∞

  3. Robert says:

    @clouddream: I hadn’t tested Google Chrome, but it looks like the “problem” (if you can call it that) is that Chrome has eliminated the overhead associated with variable resolution.

    I’ve posted a comment on the Ajaxian article that goes into more detail about what’s going on.

  4. Howard Katz says:

    Very nice, Robert. And very timely for me. I’ve been working on a jQuery plug-in and had been wondering about the difference in performance between using local variables vs. using instance variables. You just provided me the answer (which I found a bit surprising):

    http://www.fatdog.com/litmus_tests/InstanceVsLocalTest.html

    Apologies for the color scheme! :-)
    Howard

  5. Howard Katz says:

    I just had it pointed out to me elsewhere that my testing wasn’t testing what I thought it was. In particular, ‘this’ in the second test is referring to the window object and not the test function. Sigh, ‘this’ still manages to come along and hit me over the head now and then.
    Howard

  6. Also, it would be nice if the graph showed the OS or at least if you mentioned the OS used for the various graphs posted in this post. I got completely different profiles on XP and Vista.

  7. Robert says:

    Done. Just added browser detection (and display) for all supported browsers (FF, IE, Opera, Chrome, Safari, iPhone)

  8. [...] we find it valuable and devote time to measuring our Javascript performance using Robert’s JSLitmus Javascript benchmarking toolkit that he built while we were improving Zenbe render times [...]