Expiring fragments from daemons

We have an application that gets its data from a series of daemons that go out and read in data. This works great, except, we are caching pages. And I'd like to expire those pages based on an update.
It turns out that an Observer doesn't have access to expire_action or fragment. And a Sweeper is not called from data-only (i.e non-controller) updates! Buggers!

But there is a solution. You can call the sweeper directly from your importer:

MySweeper.instance.clean_up(model_instance)

This works, except I couldn't get it reliably to expire the actions. So, I used direct calls to Rails.cache.delete to do this.

Thinking about it, I guess I could then have just written an observer! As those do get called from controller-less updates.

Toygaroo on Rails

Seeing as there has been a lot of press recently about my company Toygaroo I thought I might throw out some tech info for those of you who care.
For those who don't know, Toygaroo is America's biggest toy rental company (think netflix for kid's toys). Recently - March 25th, 2011 - we appeared on the season premiere of Shark Tank - a national television show on ABC. About 4.6 million people watched the show.

The Platform
Toygaroo is a Ruby on Rails 3 application. It is based heavily on the code we wrote for FilmAmora.com, Spain's leading DVD rental company. We are running on Ubuntu 10.04 LTS. We're using Passenger 3 and Nginx (doesn't everyone?!). We're using Dalli in front of MemCached (though we've had some issues with this that I really should blog about!).

The Host
Right now we are running on Amazon EC2 service - though with Mark Cuban coming on board that might change. I am a big fan of EC2, though I think the machines are a little underpowered for what you pay.

The numbers!
In the 2 hours after the show aired we received around 70,000 page views. The basic architecture is a load balancer sitting in front of a whack of app servers. We are not using a scaling solution right now - hey, we're a start up! - so we wind up more servers if we feel we need them. It is a pretty simple process - I have a script for us to follow to get an Ubuntu server up and running in no time.

Caching
I looked into other solutions, like Varnish, but decided that Rails could handle the job with a combination of page, action and fragment caching. And I haven't been wrong so far. Even under heavy load we are getting great response time. The key - as I found out with FilmAmora - is what level to cache on. We cache 'blocks'. i.e. if you look at an index page with lots of toys we cache each toy block. That block can appear on many different pages, so it is a nice solution I find.

As time goes on I'd like to post more about how Toygaroo is coded and running as I think it will provide a nice real world example for what you can do with Rails 3. If you have any questions drop me a line (comment on here).

expiring action_cache

I was having a bugger of a time expiring my cache. The problem was the cache_path.
I have index actions that take filters. So, I created a cache_path for my action caching:

caches_action :index, :layout => false, :cache_path => Proc.new { |c| "#{c.params[:league_id]}/index/#{c.params.values.sort.to_s}" }

Which means that all my caches end up in a nice tree of 'views/[league name]'. But, how to expire a whole part of the cache tree when there could be dozens of nodes as people could be searching on last names too.

What I found was that expire_action was NOT the way to go. expire_fragment lets you pass in a regular expression. Now in my sweeper I have this:

  def expire_cache_for(player)
    expire_fragment(%r{views/#{player.league.to_param}.*})
  end

Which will expire all the cached code for the player's league.

Rails 2.1 caching - nothing is ever easy!

Last night I watched the new Railscast episode that talked about the new caching features in Rails 2.1. I thought it looked cool and would add it to the new round of FilmAmora changes.

But... nothing is every easy!

I may look simple, but I encountered several problems

1. Conflicts with GetText

We use GetText for the translations on FilmAmora. We like it because there are free poEditor apps on every platform and we can easily send off the files to whomever to be translated.

The problem comes with this line:

require 'gettext/rails'

That you need to have to fire up some of the Rails-specific GetText stuff. This is all fine - it has been working for quite some time. But, when I tried to cache our Genres like this:

  def self.all_cached(language)
    key = "genres_#{language}"
    Rails.cache.fetch(key) {Genre.find(:all).sort_by {|genre| genre.get_description}.reject {|g| g.get_films_count == 0}}
  end

Here is the result:

undefined method `cache' for GetText::Rails:Module

What?!?! Yes, it seems that GetText::Rails will hide Rails. This is, quite frankly, SHIT. So, after a long time poking around I have discovered that you need to do this:

  def self.all_cached(language)
    key = "genres_#{language}"
    ::Rails.cache.fetch(key) {Genre.find(:all).sort_by {|genre| genre.get_description}.reject {|g| g.get_films_count == 0}}
  end

Yippee! It will all work now, right?

Wrong.

2. Class != Class

I hit refresh the first time and wow was I excited! I saw lines like this in the log:

Cache write (will save 0.59562): genres_es

Woo hoo! Look at all the time I will save!

So I hit refresh.

undefined method `get_description' for #<Genre id: 1, description: "Action and Adventure", order_by: 2>

What?!?!

I am using the 'default' memory store. Something is going funny with retrieving objects from it. I never solved this problem.

If I do this in the console:

>> @genres = Genre.all_cached("es")
>> @genres = Genre.all_cached("es")

I see this in the log:

Cache write (will save 0.51974): genres_es
Cache hit: genres_es ({})

And I can do this:

>> @genres[0].get_description("es")

So what is going on in my web app? I have no idea. It seems that the class retrieved from the memory cache is incomplete in some way. get_description is not an accessor, it does go off and get the translation, but... so what? it is still a method.
This had me stumped!

3. File Store no worky
So I added this to development.rb:

config.cache_store = :file_store, '/cache_store'

I ended up getting this:

undefined method `get_description' for #<String:0x5459f0c>

Ok, I am tired of this now.

4. MemCached
Everyone is talking about using MemCached for this kind of thing. Now, I know developers and we are a lazy bunch. My guess is that this whole caching stuff has been written with MemCache in mind and screw anything else (see points 2 and 3 above).
So I installed MemCache and changed the line in the development.rb to be this:

config.cache_store = :mem_cache_store

I restarted the server and hit refresh. Trying to contain my excitement I saw a properly rendered page.
I hit refresh again.

undefined class/module Subgenre

For CHRIST'S SAKE!
After another web-scouring exercise I discovered a solution.

My method now looks like this:

  def self.all_cached(language)
    key = "genres_#{language}"
    Subgenre
    ::Rails.cache.fetch(key) {Genre.find(:all).sort_by {|genre| genre.get_description}.reject {|g| g.get_films_count == 0}}
  end

And guess what? It works. Of course it now means running memcache on my local machine and installing it on the production box. But I've saved .52 seconds! It took 3 hours to get to the solution, so I figure in only 350 web page hits I will make it back.

Oh, of course I was already caching the html on the server for most things, but this is a little tidier.