One of the first things I learned when I was becoming a programmer was to be lazy. If someone had already written a piece of code, I was taught not to try and reinvent the wheel and to liberally use that other piece of code (This is a lesson a lot of developers forget but is a topic for another day). This same principle should be applied as you attempt to scale your website. Learn to copy yourself.
In my last post, I described how it was important to look carefully at your data as you attempted to scale. While I talked about how to increase performance of you database server, I did not talk about how to really achieve web scale. To do that, you need to scale horizontally. You can achieve this in a few different ways. The most straightforward of those is to clone all of your services and data and then put those objects behind a load balancer. This is a technique we use extensively at Oversee in order to handle the large loads on our systems.
So what does this actually mean? Let’s look at the data side of things. One of the things I said in the last post was to really examine your data to see how “fresh” it had to be and how often it is updated. If your system is like ours, you have a lot more reads of your data than writes and it is OK for the data not to be completely up to date. For Oversee, the ratio of read to writes is on the order of over 1000:1. What this allows you to do is to replicate the data over many servers. Almost all major database vendors support data replication right out of the box. Most relational databases will support multiple slaves to a single master. Writes are committed to the master database and eventually get propagated to the read-only slave nodes. You can fine tune how often the slave nodes get updated to reduce the data latency but for applications like ours, this is not a primary concern.
After you have successfully setup database replication, you should put your servers behind a load balancer. All requests for data should go through the load balancer which will correctly route your request to the server that is best equipped to handle it. You should design your application so the servers behind the load balancer act as a black box. It should not matter which of the servers, master or slave, handles the request. When done this way, handling more load to your system is straight forward; you setup replication on another server, place it in the pool of servers behind the load balancer, and requests will automatically be spread across the additional resources. So long as you can perfectly replicate the servers behind the load balancer, this will scale to hundreds if not thousands of servers. This technique has the additional benefit that if any server fails behind the load balancer, the service it is providing will still be available assuming there is another server to take the request. So not only do you achieve scalability, you get high availability for free!
You can, and should, apply this technique to servers other than your database. We do this for our entire web-serving stack. Almost every server we run sits behind a load balancer. This includes the pages responsible for keyword generation, page configuration, and rendering. This requires that you think carefully about how best break up the services your web application provides. That is the topic for the next blog post.
