Memcached and OSX

I was working on part of my soon to be launched rails project, snowballr, and found part of one of my processes to be particular expensive from a computational perspective. Being a bright sort of chap, my immediate reaction was “cache the results” and since I come from a proud lineage of Java developers I bravely set off to create my own caching scheme. This of course turned out to be a really bad case of the “Not Invented Here” syndrome which of course my enterprisy roots have tended to foster . I looked around for other solutions, and found the tried and true memcached solution that has been used by tons of people to accomplish the same thing. Memcached (pronounced mem-cache-dee) is a distributed caching framework that alleviates database load. It was originally developed for the website LiveJournal.com. My main reason for not considering that before, is that it can start to seem like you need a million moving pieces of code all working together in order to get a rails application properly running (you know the drill, mongrel, pound.. rmagick –YIKES). I just didn’t want to add to the complexity.

Furthermore I was hosting at a shared host, and things like memcached just aren’t going to happen. I had already got bitten once by assuming distributed ruby, drb would run at my shared host, and when it came to deploy time I was in for a rude awakening. Since then I have set up a virtual server at SliceHost, and so I control whatever is on that box. Memcached suddenly looked like a viable solution.

Here is what I did to set it up on my MacBook Pro. Since OSX by default has some issues with some of the binary releases available, Geoffrey Grosenbach has written a script which downloads, patches and compiles all the right stuff. Get the script here. This script will create binaries in /usr/local/bin, so make sure that is in your path.

after downloading the script, cd to a directory of your choice (I created a directory “memcached” in my development folder) and run the script. It should happily churn while you are getting your coffee.

At the conclusion of the script, you need to edit your .bashprofile to include EVENTNOKQUEUE=1. I am not sure what voodo this does, but its required . Go ahead and pull that configuration into your current environment by entering $ source ~/.bash_profile

Next you’ll need to install the Ruby memcached client. This allows the rails application to put and get things from the cache. Install it with:

$ sudo gem install memcache-client

Now that the client and server is installed, you can start doing some basic testing. To start the server type:

$ memcached -vv

( the -vv means verbose output)

Once the server is up and running, you have to tell your rails application how to find your distributed cache. On both my development and production environments, my cache will be running on localhost, so my configuration won’t change when I deploy. For that reason I will put the necessary rails configuration information in the generic config/environment.rb file rather than the more specific config/environments/development.rb (or production.rb). Configure the memcached client by adding the following lines to the end of environment.rb:

CACHE = MemCache.new :namespace => "myapplication", :cthreshold => 10000, :compression => true, :debug => false, :readonly => false, :urlencode => false

CACHE.servers = '127.0.0.1:11211' ActionController::Base.sessionoptions[:expires] = 1800 ActionController::Base.sessionoptions[:cache] = CACHE

(you could of course have run the memcached server on any arbitrary address, and included that address in the servers parameter shown above)

Now that this information is entered, you can test the basic operation of the cache from the rails console and observe inbound requests from the memcached process you started earlier. Enter the following from the rails console:

$ script/console Loading development environment

CACHE["test"] = "my cool cached info" =>"my cool cached info" CACHE["test"] =>"my cool cached info" CACHE.delete "test" =>"DELETED\r\n" CACHE["test"] =>nil

and thats pretty much it! To use it in your controllers or models simply put objects in the cache using CACHE[] = object or CACHE[] to retrieve.