Increase the Performance of Fragment Caching in Rails

Rails Caching OptimizationIf you use Rails on a high-traffic site, you know that as your number of users increases, you have three main options:

  1. Add servers to handle the load
  2. Optimize your queries
  3. Improve your caching scheme

You may be surprised to know that most people go for Door #1. It’s a lot easier. You don’t have to really do tons of work to rewrite your code and actually make it efficient. You don’t have to ditch “the Rails Way” and start actually thinking about what the database layer is doing to slow your site down to a crawl. And you don’t have to think about how Rails’ cache expiry functions actually work. Finally – and best of all – you can just pass the costs on to your customer, right??

I recently read an article that really slayed me. It was about a “niche site” that runs on Rails. They get 50 million hits a month, and they have SIX servers to handle the load, including multiple dedicated DB servers. I designed a Rails site that now gets 27 million hits a month, and it runs Rails on a single 1.86 GHz dual-core server with 3GB of RAM. By my calculations, the site could easily handle twice as many hits as it does now. Most of the time, the load is very low and the CPU and disk accesses hover at a few percent.

Of course, to achieve good performance, you can use things like Phusion Passenger. But that alone ain’t gonna cut it. You also have to optimize your queries, stop doing things The Rails Way and start thinking for yourself, and of course optimize your caching scheme.

In this episode, I’m going to tell you one very cool way to turbocharge your caching setup!

(more…)

Optimize Your Caching Scheme to Eliminate “Too many links” Errors on Linux

Say you have an application running on a Linux server using the ext2 or ext3 file systems. You set up a caching scheme in your application to store files like so:

/cache/stamps/1/main_content.cache
/cache/stamps/1/comments.cache

/cache/stamps/2/main_content.cache
/cache/stamps/2/comments.cache

/cache/stamps/3/main_content.cache
/cache/stamps/3/comments.cache
...

Eventually, you’re going to run into a problem: Your caching will stop working since your application won’t be able to write to the “stamps” directory any more. Instead, you’ll get an error message like this:

Couldn't create cache directory: /stamps/41134/main_content (Too many links - /var/www/your-app/tmp/cache/stamps/41134)

At that point, you’ll try to search for “too many links”, and probably you won’t find much information that is actually comprehensible to normal human beings. Fortunately, it really is quite simple to repair once you understand what’s going on.

(more…)