Memcached scalability

Behind the scenes of many of your favourite websites you will find an application, that speeds up their page creation times. It's called memcached. At the end it's not much more than a server that allows you to access remote memory to store and load objects like the output of a database query or frequently accessed web pages. Such an mechanism is able to reduce the load to your database by at least an order of a magnitude.

So it's a good idea to do some tuning on it to run on new types of hardware, like CPUs that look like 64 CPUs as such applications were initially developed on architecture with 4 procs at maximum at most deployments.

Zoran wrote about his and his teams efforts to optimize scaling of a single-instance memcached on an UltraSPARC T2 processor. They were able to yield 500.000 operations per second at 100 Byte and 255.000 operations per seconds at 4096 Byte. The second test almost saturated the 10 GBit/s interface of the system.

Shanti made the same test on a dual-cpu Nehalem server: She was able to yield 352,190 with a single instance and 470K ops/sec with two instances. I don't want to make a comment on the performance (I don't know if Shanti had the scalability patch Zoran used in his benchmarking), albeit it's interesting that an single-socket T2 system (announced more than one and half year ago) delivers more performance than a dual-socket Nehalem.

The more important takeaway is: The x86/x64 business is running into the same scalability problems as the massive multicore procs like UltraSPARC T2. So much power, so much work to do, but the software doesn't allow them to to it as it wasn't developed for such environments. Shantis article strongly hints towards such an situation. Sun just ran into this problem 3-4 years earlier than the rest. But: The more people have a problem, the more people working at the solution. I assume we will see more scalability patches in the future for a lot of software projects.

PS: It's nice to see that Sun was correct about a development ... again ...