messaging snippets

Using backgroundrb to execute tasks asynchronously in Rails

Tagged backgroundrb, rails, ruby, distributed, messaging  Languages ruby

Draft...

Planning on using BackgroundDRB? Take a long look at the alternatives first

Ask yourself, do you really need a complex solution like BackgroundDRB? Most likely you don't, so use a simple daemonized process instead, see this snippet about the daemons gem for more information.

Heck, even a simple Ruby script run by cron every 5 minutes will be more stable than BackgroundDRB and require less work.

Even if you really need to process a lot of data asynchronously in the background, I wouldn't recommend BackgroundDRB, it's riddled with bugs and unstable in production, so use the BJ plugin instead.

Anyway, continue reading if you want to use BackgroundDRB...

Installing the prerequisites:

$ sudo gem install chronic packet

Installing backgroundrb

$ cd rails_project
$ git clone git://gitorious.org/backgroundrb/mainline.git vendor/plugins/backgroundrb

You can also get the latest stable version from the Subversion repository:

svn co http://svn.devjavu.com/backgroundrb/trunk  vendor/plugins/backgroundrb

Setup backgroundrb

rake backgroundrb:setup

Create a worker

./script/generate worker feeds_worker
class FeedsWorker < BackgrounDRb::MetaWorker
  set_worker_name :feeds_worker
  
  def create(args = nil)
    # this method is called, when worker is loaded for the first time
    logger.info "Created feeds worker"
  end
  
  def update(data)
    logger.info "Updating #{Feed.count} feeds."
    
    seconds = Benchmark.realtime do
      thread_pool.defer do
        Feed.update_all()
      end
    end

    logger.info "Update took #{'%.5f' % seconds}."
  end
end

Starting backgroundrb

First configure backgroundrb by opening config/backgroundrb.yml in your editor:

:backgroundrb:
  :ip: 0.0.0.0

:development:
  :backgroundrb:
    :port: 11111     # use port 11111
    :log: foreground # foreground mode,print log messages on console

:production:
  :backgroundrb:
    :port: 22222      # use port 22222

Next, start backgroundrb in development mode:

./script/backgroundrb -e development &

Call your worker

From the command line:

$ script/console
Loading development environment (Rails 2.0.2)
>> MiddleMan.worker(:feeds_worker).update()

When things go wrong

Asynchronous programming is complex, so expect bugs...

Rule #1 know who you're calling.

If you give your MiddleMan the wrong name of your worker, he'll just spit this crap at you:

You have a nil object when you didn't expect it!
The error occurred while evaluating nil.send_request
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in ask_worker'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:104:in process_work'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:35:in receive_data'
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in call'
/usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in extract'
/Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:31:in receive_data'

So for example this command would generate the above mentioned error:

MiddleMan.worker(:illegal_worker).update()

It's always nice to see a cryptic error messages such as this, it really deserves an award.

Check for bugs and bug fixes

git mainline commits

Going to production

Starting the daemon:

./script/backgroundrb -e production start

Configuring your task to run periodically

The following example makes backgroundrb call the FeedsWorker's update method once every 15 minutes:

:production:
  :backgroundrb:
    :port: 22222      # use port 22222
    :lazy_load: true  # do not load models eagerly
    :debug_log: false # disable log workers and other logging
# Cron based scheduling
:schedules:
  :feeds_worker:
    :update:
      :trigger_args: * */15 * * * *
      :data: "Hello world"

At the time of writing, the cron scheduler seems to be broken, so I prefer hard-coding the interval in the worker's create method:

def create
           add_periodic_timer(15.minutes) { update }
         end

If using Vlad or Capistrano, it's also a good idea to fix script/backgroundrb by changing these lines:

pid_file = "#{RAILS_HOME}/../../shared/pids/backgroundrb_#{CONFIG_FILE[:backgroundrb][:port]}.pid"
SERVER_LOGGER = "#{RAILS_HOME}/../../shared/log/backgroundrb_server_#{CONFIG_FILE[:backgroundrb][:port]}.log"

Resources

Backgroundrb homepage

Backgroundrb best practices

Backgroundrb scheduling

Debugging backgroundrb

Backroundrb's README

topfunky's messaging article

Exactly-once delivery with RabbitMQ

Tagged amqp, exactly-once, messaging, rabbitmq, reconciliation  Languages 

Use late acknowledgment and idempotency to fake exactly-once delivery with RabbitMQ. Remember, there’s no exactly-once delivery even if your messaging software provider claims so.

Late acknowledgment = acknowledge the message after the message has been processed and the database transaction has been committed.

Idempotency = ensure the effect is the same when processing the same message multiple times. In other words, don’t process the same message twice.

Reconciliation step = if you need 100% reliability create a process (manual or automatic) that checks that all work is done, if not resend the message.

The customer is the last line of defense and they will let you know if one of the above steps have failed.