From a Faction on the Web to a Slice of the Host
From a Faction on the Web
…
Until recently, 8bitb.us was a super-basic Django blog with no caching, running on Webfaction. ‘Twas slow, but got the job done. The only interesting bit was serving media from S3. Since the super-cheap Webfaction plan has pretty strict limits on resources, I needed to serve media outside of Apache so my two (yes, two) Apache clients could focus on serving dynamic content.
The basic setup looked like:
… to a Slice of the Host
A few nights ago I moved the whole mess over to Slicehost. As a sysadmin, it irked me that Webfaction limited the number of long-running processes I could have and didn’t give me root. I know why they do this, and it works for most people. But I had plans, big plans, and the plans involved daemons. Here’s the new setup:
So requests for both static and dynamic content come in to Varnish, which passes the requests to Nginx if the URL isn’t in Varnish‘s cache.
What the above diagram doesn’t show you is that every process is being controlled with Daemontools. I like Daemontools because I get process respawning, log rotating, and consistent signal-based control for free.
Files
in the Ointment
backend default { .host = "127.0.0.1"; .port = "8000";}#!/bin/shexec 2>&1exec /usr/local/sbin/varnishd -T 127.0.0.1:60001 -F -a 0.0.0.0:80 -f./8bitbus.vcl
/service/8bitbus-nginx/nginx.conf
#!/bin/shexec 2>&1exec /usr/local/sbin/nginx -c ./nginx.conf
#!/bin/shexport PYTHONPATH=/home/alex/pythonexec 2>&1cd /home/alex/djangoprojects/eightbitbusexec python manage.py runfcgi host=127.0.0.1 port=8080 maxchildren=5daemonize=False --settings=settings
Questions You Didn’t
Ask
Why no Memcached?
Since every vistor to the site (except for me) sees the same content, I don’t need fine-grained caching. All I need’s a coating of Varnish over the whole thing.
Why SQLite instead of
Postgresql or Mysql?
SQLite takes very little RAM (my slice only has 256MB). Postgresql and Mysql take a lot. SQLite has good read concurrency, and I minimize the number of database writes by using file-based sessions.I’m trying to avoid them for all but admin work. And when I do need a session, do I store it in the database? No! I’d like to minimize the number of writes to SQLite, partially because SQLite doesn’t handle concurrent writes well, and partially because I’m keeping the db in git.
How fast is it?
Hard to say - this is, after all, just a slice of a machine. And we all know benchmark numbers are dumb, but … fine. Via ab, I’ve gotten ~6000 homepage requests locally. Via Siege, I’ve gotten up to 1900 homepage requests per second locally, or 400 requests per second across two remote machines. And I still have some kinks to work out with Varnish, so I expect those numbers to go up. Other people on real hardware can get 4000 requests per second with Varnish.
