The Joy of Load Testing
Those of you that know me will be aware of my general distaste of working for a living, but from time to time it can actually be quite interesting. Last night was one such occasion.
I don’t like to talk about my day job on my blog, but it should be OK to give you a bit of background. Part of what I do involves the testing and monitoring of web applications, helping to identify performance issues. This means I sometimes find myself sitting up through the night running performance tests against our clients’ web sites too, which is what I was doing last night.
A lot of the time people think they can just throw money at hardware to improve the performance of their sites. Of course it’s true that this can often help, but you can frequently get huge benefits just by tweaking your web and application server configuration files. Last night was a perfect example of this.
This was the second part of testing on a particular e-commerce web site. The first identified a memory leak which caused the whole site to eventually grind to a halt (this was a Java garbage collection issue), and there was an issue with the number of connections between the web servers and the application servers behind them too. This meant we could only get up to about 150 to 200 concurrent users on the web site before getting response timeouts.
So fixing these two issues meant that the site could now take a sustained load of about 300 users. Twice the number of users that they could handle before!
I won’t go into all the details of the testing, but CPU usage was still low and there was loads of free memory still floating around, so HTTP compression was now turned on. And what a difference! We now managed to get 600 users on the site with no problems at all (a four-fold improvement!), and the user experience improved as download times were now much faster.
I’ve long been an advocate of simple technologies like HTTP compression and caching to improve web application performance, and it’s always great to see it in action on a real commercial web site. There are lots of fairly cheap ways to implement this kind of thing: you can easily do caching at the application code level if you are still building the application, and both compression and caching can be performed within Apache and IIS using various modules.
Cost is often an important consideration, and adding more servers to your infrastructure doesn’t come cheap. Not only is there the capital outlay but it costs more to administer the extra boxes too. And with all that redundancy comes an increased chance of something going wrong… All in all a potential pain in the neck!
This post feels to be turning into a slightly more serious one than planned, but no discussion on improving web performance would be complete without mentioning web appliances. Increasingly popular but still frequently overlooked, these can be much cheaper to implement than building out your web farm. There are loads of players in that market, and I’ve seen some very impressive results from my clients with products like NetScaler from Citrix, ZXTM from Zeus, and also the JetNexus web appliance. (In fact I can put you in touch with people from all of these companies if you want 
)
Anyway, before you blow cash on any of these things I would recommend you take a look at some of the free (Open Source) technologies first. For compression on Apache 1.x take a look at mod_gzip, and Apache 2.x users should look at mod_deflate. Memcached is just amazing for caching at the application level (available for loads of programming languages), and PHP users may like to take a look at the PEAR Cache_Lite package. Apache 2.x users can also use mod_cache too.
This is a massive subject so I’m going to stop now before my beer gets too warm. I’d be interested to hear your experience with related technologies if you have any.
RSS Feed for posts