Using backgroundrb to execute tasks asynchronously in Rails

Draft...

Planning on using BackgroundDRB? Take a long look at the alternatives first

Ask yourself, do you really need a complex solution like BackgroundDRB? Most likely you don't, so use a simple daemonized process instead, see this snippet about the daemons gem for more information.

Heck, even a simple Ruby script run by cron every 5 minutes will be more stable than BackgroundDRB and require less work.

Even if you really need to process a lot of data asynchronously in the background, I wouldn't recommend BackgroundDRB, it's riddled with bugs and unstable in production, so use the BJ plugin instead.

Anyway, continue reading if you want to use BackgroundDRB...

Installing the prerequisites:

$ sudo gem install chronic packet

Installing backgroundrb

$ cd rails_project
$ git clone git://gitorious.org/backgroundrb/mainline.git vendor/plugins/backgroundrb

You can also get the latest stable version from the Subversion repository:

svn co http://svn.devjavu.com/backgroundrb/trunk  vendor/plugins/backgroundrb

Setup backgroundrb

rake backgroundrb:setup

Create a worker

./script/generate worker feeds_worker
class FeedsWorker < BackgrounDRb::MetaWorker
  set_worker_name :feeds_worker
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
    logger.info "Created feeds worker"
  end
  
  def update(data)
    logger.info "Updating #{Feed.count} feeds."
    
    seconds = Benchmark.realtime do
      thread_pool.defer do
        Feed.update_all()
      end
    end

    logger.info "Update took #{'%.5f' % seconds}."
  end
end

Starting backgroundrb

First configure backgroundrb by opening config/backgroundrb.yml in your editor:

:backgroundrb:
  :ip: 0.0.0.0

:development:
  :backgroundrb:
    :port: 11111     # use port 11111
    :log: foreground # foreground mode,print log messages on console

:production:
  :backgroundrb:
    :port: 22222      # use port 22222

Next, start backgroundrb in development mode:

./script/backgroundrb -e development &

Call your worker

From the command line:

$ script/console
Loading development environment (Rails 2.0.2)
>> MiddleMan.worker(:feeds_worker).update()

When things go wrong

Asynchronous programming is complex, so expect bugs...

Rule #1 know who you're calling.

If you give your MiddleMan the wrong name of your worker, he'll just spit this crap at you:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.send_request
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in ask_worker'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:104:in process_work'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:35:in receive_data'
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in call'
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in extract'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:31:in receive_data'

So for example this command would generate the above mentioned error:

MiddleMan.worker(:illegal_worker).update()

It's always nice to see a cryptic error messages such as this, it really deserves an award.

Check for bugs and bug fixes

git mainline commits

Going to production

Starting the daemon:

./script/backgroundrb -e production start

Configuring your task to run periodically

The following example makes backgroundrb call the FeedsWorker's update method once every 15 minutes:

:production:
  :backgroundrb:
    :port: 22222      # use port 22222
    :lazy_load: true  # do not load models eagerly
    :debug_log: false # disable log workers and other logging
# Cron based scheduling
:schedules:
  :feeds_worker:
    :update:
      :trigger_args: * */15 * * * *
      :data: "Hello world"

At the time of writing, the cron scheduler seems to be broken, so I prefer hard-coding the interval in the worker's create method:

def create
           add_periodic_timer(15.minutes) { update }
         end

If using Vlad or Capistrano, it's also a good idea to fix script/backgroundrb by changing these lines:

pid_file = "#{RAILS_HOME}/../../shared/pids/backgroundrb_#{CONFIG_FILE[:backgroundrb][:port]}.pid"
SERVER_LOGGER = "#{RAILS_HOME}/../../shared/log/backgroundrb_server_#{CONFIG_FILE[:backgroundrb][:port]}.log"

Resources

Backgroundrb homepage

Backgroundrb best practices

Backgroundrb scheduling

Debugging backgroundrb

Backroundrb's README

topfunky's messaging article