I’m thinking about web performance again…
Since taking a look at a little project, which is using the async event driven Tornado server, I thought I’d re-investigation performance data around python web servers. Often when I see something in action I question my previous choices and have to look into it. I’ve been working on Raindrop for a while, and one decision I made was to use Pylons with SQLAlchemy as the basis for the api server, which is essentially a wsgi application. I wanted something that gave a simple framework to work with, but didn’t need or want all the features of Django. Async servers like Tornado have a few benefits that make them attractive, but you have to write your apps differently, and my main question is, is it worth it?
While I found a few interesting things, I have come to the same conclusion once again, it doesn’t really matter what server you use, 99% of the time perf is an app issue. That in combo with the LOLapps setup and what benchmarking I’ve seen, I’m pretty resolved that Pylons+SQA is a good balance between not writing too low level and not having a large framework. Probably not the best performer, but not bad. I don’t think it’s worth reworking an existing app to fit a Tornado model, but it might be fun to try out a new app there sometime.
Here’s some of what I looked at:
Ian Bicking pointed out this blog post, and it’s probably one of the better benchmarking posts I’ve seen to date (specific to Python web servers). Ian’s post of course was calling for more in depth benchmarking, hopefully someone hears his call.
I also found this set of 3 posts (find the later 2 from the first) doing some varied simple tests with Apache Bench.
What I felt was really interesting is a talk from LOLapps at PyCON (see the video), probably due to the fuzzy feeling of validation about Pylons :).
I’ll summarize the video, they use pylons + paste + sqa, experimented with tornado and decided to stick with pylons for “most” systems, thought they’ll try to use tornado in some places.
They do some interesting cacheing with sqa that we can try at some point in the future,
A suggestion I don’t recall running into before was to disable the nagle algorithm (nice explanation). Of course, doing that right now may be premature optimization.
They also have some good input on profiling performance, which reminded me again that I need to dig out the wsgi middleware I used for profiling.
Anyway, I was writing this as an email, and thought, why not blog? It’s been a while.