2009-07-01

Firefox 3.5 Benchmarking in Linux

So I know there are tons of benchmarks already out there about the just released Firefox 3.5, but most of them are Windows focused. So if like me you run a Linux OS there's still a little more to know perhaps?

I know most benchmarkers prefer to do everything very cleanly with nothing but the browser running and maybe even a fresh reboot for each test...but I'm doing this more real world with other programs running in the background and no reboots. Just for reference though, none of the other programs running were changed during the tests to maintain some semblance of scientific objectivity.

I run Ubuntu 8.04 (LTS) still, and so the only official Firefox available to me thru Canonical's repositories is FF 3.0.11. In theory this one should be optimized specifically for use with Ubuntu, but as others have pointed out in the last few months, FF seems to run a bit slower on Linux than Windows of all things. Most believe this to be a matter of optimization, so I am trying out SwiftFox for the first time today too. The version I'm testing is 3.5rc3 and optimized to my Pentium D (prescott, 32bit) CPU, so supposedly it will be faster than the one Ubuntu has shipped and the vanilla binary from Mozilla's website, which is where I got the copy of 3.5 I'll be using for this test. And then on top of those three I'm also going to run the same tests on Google's new Linux alpha version of Chrome (ver:3.0.190.2).

So first off, just for curiosity's sake, let's see how they all do in the ACID3 test:

Firefox 3.0.11: 72%
Firefox 3.5.0: 93%
SwiftFox 3.5.rc3: 93%
Chrome 3.0.190.2: 99.9%

As expected Firefox 3.5 and SwiftFox get the same score as they're technically the same version of the rendering engine. Chrome actually says 100/100, but then it says "linktest failed" below that and there is a big X in the top-right corner, so I've marked it down to 99.9% as I don't know what that really constitutes a score of.

Next up, we'll be running Google's own V8 Benchmark, and the scores are rather surprising.

FF 3.0: 119
FF 3.5: 194
SF 3.5: 230
Chrome: 2492

So, first off the difference between Chrome's score and the other Mozilla based browsers is almost ridiculous. As Google created this benchmark themself, it almost makes you suspicious if they specifically put in tests they knew V8 would handle better than TraceMonkey? So I'll take that portion with a grain of salt. Also these "points" don't have any real intrinsic definition as to how they are calculated, more being better obviously. As for the other three, FF 3.5 doesn't even score twice as high as 3.0 did. Swiftfox however finally proves that it is indeed much better optimized that the other 2 by scoring almost 40 points more than the vanilla build.

Last we'll go with the tried and true SunSpider benchmark provided by the WebKit team.

FF 3.0.11: 5,583ms
FF 3.5: 2,421ms
SF 3.5: 2,111ms
Chrome: 986ms

Here FF 3.5 shows it is clearly much faster than 3.0 by completing the series of tests in less than half the time it took it's older brother. Once again SwiftFox shows us it is certainly faster than its not-so-well optimized cousins. And here Chrome really shines with completing the tests in less than a second! This is over twice as fast as Swiftfox, so maybe that V8Bench score wasn't as artificially bloated as I thought?

There are obviously many other browsers I could have included here, but decided not to bother with. When Konqueror finally switches to WebKit and Squirelfish it may be worth writing about, but for now is last decades' browser. Projects like Midori are also much too new to worry with at the moment. Safari nor IE have native ports for Linux, so they're obviously right out. And last there's Opera... This is a philosophical question for most, but I refuse to use a proprietary browser when there are open source options just as good if not better, so I personally could care less about Opera at all, that and their javascript performance is still quite far behind the likes of TraceMonkey, SquirellFish & V8 from what I understand.

To conclude, Chrome is still extremely alpha at the moment, with no plugin support so it's not really viable for daily use. With no Flash nor HTML5, half the web quickly becomes unusable, so we'll see what the situation is looking like when they finally make their first stable release for Linux based OSes. For now, I know I'll be using SwiftFox from now on ;)

No comments: