In my previous post I focused on how Firefox compares against itself with multiple content processes. In this post I’d like to take a look at how Firefox compares to other browsers.
For this task I automated as much as I could, the code is available as the atsy project on github. My goal here is to allow others to repeat my work, point out flaws, push fixes, etc. I’d love for this to be a standardized test for comparing browsers on a fixed set of pages.
As with my previous measurements, I’m going with:
total_memory = RSS(parent) + sum(USS(children))
An aside on the state of WebDriver and my hacky workarounds
When various WebDriver implementations get fixed we can make a cleaner test available. I had a dream of automating the tests across browsers using the WebDriver framework, alas, trying to do anything with tabs and WebDriver across browsers and platforms is a fruitless endeavor. Chrome’s actually the only one I could get somewhat working with WebDriver.
Luckily Chrome and Firefox are completely automated. I had to do some trickery to get Chrome working, filed a bug, doesn’t sound like they’re interested in fixing it. I also had to do some trickery to get Firefox to work (I ended up using our marionette framework directly instead), there are some bugs, not much traction there either.
IE and Safari are semi-automated, in that I launch a browser for you, you click a button, and then hit enter when it’s done. Safari’s WebDriver extension is completely broken, nobody seems to care. IE’s WebDriver completely failed at tabs (among other things), I’m not sure where to a file a bug for that.
Edge is mostly manual, its WebDriver implementation doesn’t support what I need (yet), but it’s new so I’ll give it a pass. Also you can’t just launch the browser with a file path, so there’s that. Also note I was stuck running it in a VM from modern.ie which was pretty old (they don’t have a newer one). I’d prefer not to do that, but I couldn’t upgrade my Windows 7 machine to 10 because Microsoft, Linux, bootloaders and sadness.
I didn’t test Opera, sorry. It uses blink so hopefully the Chrome coverage is good enough.
The big picture
The numbers
| OS | Browser | Version | RSS + USS |
|---|---|---|---|
| OSX 10.10.5 | Chrome Canary | 50.0.2627.0 | 1,354 MiB |
| OSX 10.10.5 | Firefox Nightly (e10s) | 46.0a1 20160122030244 | 1,065 MiB |
| OSX 10.10.5 | Safari | 9.0.3 (10601.4.4) | 451 MiB |
| Ubuntu 14.04 | Google Chrome Unstable | 49.0.2618.8 dev (64-bit) | 944 MiB |
| Ubuntu 14.04 | Firefox Nightly (e10s) | 46.0a1 20160122030244 (64-bit) | 525 MiB |
| Windows 7 | Chrome Canary | 50.0.2631.0 canary (64-bit) | 1,132 MiB |
| Windows 7 | Firefox Nightly (e10s) | 47.0a1 20160126030244 (64-bit) | 512 MiB |
| Windows 7 | IE | 11.0.9600.18163 | 523 MiB |
| Windows 10 | Edge | 20.10240.16384.0 | 795 MiB |
So yeah, Chrome’s using about 2X the memory of Firefox on Windows and Linux. Lets just read that again. That gives us a bit of breathing room.
It needs to be noted that Chrome is essentially doing 1 process per page in this test. In theory it’s configurable and I would have tried limiting its process count, but as far as I can tell they’ve let that feature decay and it no longer works. I should also note that Chrome has it’s own version of memshrink, Project TRIM, so memory usage is an area they’re actively working on.
Safari does creepily well. We could attribute this to close OS integration, but I would guess I’ve missed some processes. If you take it at face value, Safari is using 1/3 the memory of Chrome, 1/2 the memory of Firefox. Even if I’m miscounting, I’d guess they still outperform both browsers.
IE was actually on par with Firefox which I found impressive. Edge is using about 50% more memory than IE, but I wouldn’t read too much into that as I’m comparing running IE on Windows 7 to Edge on an outdated Windows 10 VM.