blog.pavlov.net/2008/03/11/firefox-3-memory-usage -> blog.pavlov.net/2008/03/11/firefox-3-memory-usage/
Firefox 3 Memory Usage As the web and web browsers have matured, people have started expecting different things out of them. When we first released Firefox, few people were browsing with tabs or add-ons.
This can occur as a result of mixing lots of various sized allocations and can leave a lot of small holes in memory that are hard to reuse. One of the things we did to help was to minimize the number of total allocations we did, to avoid unnecessarily churning memory. We've managed to reduce allocations in almost all areas of our code base. The graph below shows the number of allocations we do during startup. The graph below shows we were able to get rid of over 1/3 of them!
Jason Evans, to port and tune jemalloc for our platforms. It was a huge effort resulting in Jason doubling the number of lines of code in jemalloc over a 2 month period, but the results paid off. As of beta 4 we now use jemalloc on both Windows and Linux. Our automated tests on Windows Vista showed a 22% drop in memory usage when we turned jemalloc on. Fixed cycles with the Cycle collector Some leaks are harder to fix than others. One of the most difficult ones is where two objects have references to each other, holding each other alive. In previous versions, we've used very complex and annoying code to manually break cycles at the right times, but getting the code right and maintaining it always proved to be difficult. For Gecko 19, we've implemented an automated cycle collector that can recognize cycles in the in-memory object graph and break them automatically. This is great for our code as we can get rid of lots of complexity. It is especially significant for extensions, which can often inadvertently introduce cycles without knowing it because they have access to all of Firefox's internals. It isn't reasonable to expect all those authors to write code to manually break the cycles themselves. Basically, the cycle collector means there are whole classes of leak that we can easily avoid in both our code and in extensions, and that's good for everyone.
Tuned our caches Firefox uses various in-memory caches to keep performance up including a memory cache for images, a back/forward cache to speed up back and forward navigation, a font cache to improve text rendering speed, and others. We've taken a look at how much they cache and how long they cache it for. In many cases we've added expiration policies to our caches which give performance benefits in the most important cases, but don't eat up memory forever. We now expire cached documents in the back/forward cache after 30 minutes since you likely won't be going back to them anytime soon. We have timer based font caches as well as caches of computed text metrics that are very short lived. We also throw away our uncompressed image data as I describe below... Adjusted how we store image data Another big change we've made in Firefox 3 is improving how we keep image data around. Images on the web come in a compressed format (GIF, JPEG, PNG, etc). When we load images we uncompress them and keep the full uncompressed image in memory. In Firefox 2 we would keep these around even if the image is just sitting around on a tab that you haven't looked at in hours.
Federico Mena-Quintero (of GNOME fame), we now throw away the uncompressed data after it hasn't been used for a short while. Not only does this affect images that are on pages in background tabs but also ones that are in the memory cache that might not be attached to a document. This results in pretty dramatic memory reduction for images that aren't on the page you're actively looking at. If you have a 100KB JPEG image which uncompress to several megabytes, you won't be charged with the uncompressed size when you're not viewing it. Another fantastic change from Alfred Kayser changed the way we store animated GIFs so that they take up a lot less memory. We now store the animated frames as 8bit data along with a palette rather than storing them as 32 bits per pixel.
the bug showed us drop from using 368MB down to 108MB -- savings of 260MB! Hunted down leaks Most leaks are a pain in the ass to find and fix in any complex piece of software. If you leak a small piece of text once an hour you probably won't notice. If you leak a large image every time you move the cursor, you've got a big problem. Both are important to fix, because even the little ones add up. Some leaks are only leaks until you leave a page, so they don't show up with conventional leak-finding tools, but they make a difference if you have a page opened all day long like GMail. Leak Hunt Ben Turner has gotten pretty good at Leak Hunt. We've fixed many leaks, ranging from small DOM objects that get leaked on GMail until you leave the site to entire windows that were leaked holding on to everything inside of them when you closed them. Overall, we've been able to close over 400 leak bugs so far, most of which are very uncommon, but can still occur.
Carsten Book, in particular, has done an amazing job at finding and reporting leaks. Measuring Memory Use As I've learned the hard way, accurately measuring memory usage is hard. The short summary is Windows Vista (Commit Size) and Linux (RSS) provide pretty accurate memory measurement numbers while Windows XP and MacOS X do not. If you're running Windows Vista and take a look at Commit Size in task manager, you should get some pretty accurate memory numbers. If you're looking at Memory Usage under Windows XP, your numbers aren't going to be so great.
changed the meaning of "private bytes" between XP and Vista (for the better). On XP the number is the amount of virtual memory you're application has reserved for use. For performance reasons you often want to reserve more memory than you actually use. The application can tell the operating system that it isn't going to use parts of the reserved space and to not back the virtual space with physical space. On Vista, Private Bytes is the commit size, which only counts the memory the application has actually said it is actively using. Since virtual memory size has to be greater than or equal to your commit size, XP memory numbers will always appear bigger than Vista ones, even though the application is using the same amount of memory. On Mac, If you look at Activity Monitor it will look like we're using more memory than we actually are. Mac OS X has a similar, but different, problem to Windows XP. After extensive testing and confirmation from Apple employees we realized that there was no way for an allocator to give unused pages of memory back while keeping the address range reserved.. It does appear that pages mapped in that haven't been written to won't be accounted for in memory stats, but you've written to them they're going to show as taking up space until you unmap them. Since allocators will reuse space, you generally won't have that many pages mapped in that haven't been written to. Our application can and will reuse the free pages, so you should see Firefox hit a peak number and generally not grow a lot higher than that. Linux seems to do a pretty good job of reporting memory usage. It supports madvise(), allowing us to tell Linux about pages we don't need, and so its resident set size numbers are fairly accurate. Ways to test There are many ways to measure memory usage in a browser. Open up 10 tabs with your favorite websites in them and see how much memory the browser is using. Close all but the last tab and load about:blank or Google. Another simple test is simply loading Zimbra, Google Reader and Zoho each in their own tab and logging in. We've learned that users do so many things with the browser it is nearly impossible to construct a single test to measure memory usage. We wanted more of a stress test -- One that was more reproducible than loading random sites from the web.
modified it to cycle pages through a set of windows while opening and closing them to try and approximate people running for a long period of time. Talos makes it pretty straightforward to get this up and running, and is great for measuring things like memory usage and layout speed. This works great for Firefox and allows measuring perf...
|
http://csua.com/feed/