SanSay.Co.Uk

My day job involves managing a bunch of servers. I don't really want to be fiddling around making sure bits of hardware are working okay as I figure I could be doing something more interesting and worthwhile - I want to get around to actually getting some stuff done (that blasted email server, for one...). Hence I'm thinking of ways to make the whole hardware management thing easier (after I get the email server sorted, obviously). So, I'm sitting there thinking, and looking back over my thought-process I can see how many people must have thought of this before: instead of running the servers we need (email, web servers, database, MIS, Wiki, etc) on physical servers, virtualise those servers and run the services we need on top of a bunch of identical virtual machines. That way, if something conks out we can shift it somewhere else and continue running it.

Great! So I slap together a bunch of machines out of whatever bits I can find lying around the place, stuff them into a cupboard and painstakingly install FreeBuntu and compile Xen on each of them, then create a bunch of virtual machines on each server. This works. However, it's not that reliable and still somewhat fiddly (in fact, probably more fiddly as now I have another layer of abstraction to deal with). Hold on! Why don't I automate that layer of abstraction - create a CD/USB bootable Linux distribution that'll run Xen. Something minimal, no GUI needed, maybe the excellent Slax or similar. Then all I have to do is write a bit of script that looks at the hardware available to it on startup (number of disks, number of processors, etc), checks to see if there's any Xen VMs floating around for it to run, then reports to a central server. USB is probably best - have a config file in the root folder that you can edit on Windows to give it IP address of the server to talk to, maybe stuff like its own IP address and how to manage disks (does it assume the available volumes are already hardware-RAIDed, or does it run software RAID over them?).

Yay! Instant utility computing from random bits of commodity hardware. I've even got my eye on some bits of hardware to buy - I've got one of these turning up from eBay for my home server soon. I figure an internal version of that RAID box, a small 2-slot mini-itx case from the same company, a quad-core mini-itx motherboard and 2GB+ of RAM, couple of SATA-300 harddrives and a USB stick should make a nice little computing unit. Stuff a couple of dozen in a rack with UPS and a decent switch and air-conditioning and you've got a virtual supercomputer of sorts.

But hang on... this has all been done before! I've just come accross the Amazon Elastic Computing Cloud, which does this exact thing but lets me pay for it 10 US cents at a time. Only problem is that it's away On The Internet someplace - not really where we want it for fast local access. So how do I make my own Amazon-compatible elastic compute cloud - something that I can manage, and maybe swap machine images in and out of between Amazon's service?