Using backgroundrb to execute tasks asynchronously in Rails

2 Draft... h2. Planning on using BackgroundDRB? Take a long look at the alternatives first Ask yourself, do you really need a complex solution like BackgroundDRB? Most likely you don't, so use a simple daemonized process instead, see "this snippet about the daemons gem":http://snippets.aktagon.com/snippets/212-How-to-create-a-daemon-process-using-Ruby-and-the-daemons-RubyGem for more information. Heck, even a simple Ruby script run by cron every 5 minutes will be more stable than BackgroundDRB and require less work. Even if you really need to process a lot of data asynchronously in the background, I wouldn't recommend BackgroundDRB, it's riddled with bugs and unstable in production, so use the "BJ plugin":http://agilewebdevelopment.com/plugins/bj instead. Anyway, continue reading if you want to use BackgroundDRB... h2. Installing the prerequisites: <code> $ sudo gem install chronic packet </code> h2. Installing backgroundrb <code> $ cd rails_project $ git clone git://gitorious.org/backgroundrb/mainline.git vendor/plugins/backgroundrb </code> You can also get the latest stable version from the Subversion repository: <code> svn co http://svn.devjavu.com/backgroundrb/trunk vendor/plugins/backgroundrb </code> h2. Setup backgroundrb <code> rake backgroundrb:setup </code> h2. Create a worker <code> ./script/generate worker feeds_worker </code> <code> class FeedsWorker < BackgrounDRb::MetaWorker set_worker_name :feeds_worker def create(args = nil) # this method is called, when worker is loaded for the first time logger.info "Created feeds worker" end def update(data) logger.info "Updating #{Feed.count} feeds." seconds = Benchmark.realtime do thread_pool.defer do Feed.update_all() end end logger.info "Update took #{'%.5f' % seconds}." end end </code> h2. Starting backgroundrb First configure backgroundrb by opening config/backgroundrb.yml in your editor: <code> :backgroundrb: :ip: 0.0.0.0 :development: :backgroundrb: :port: 11111 # use port 11111 :log: foreground # foreground mode,print log messages on console :production: :backgroundrb: :port: 22222 # use port 22222 </code> Next, start backgroundrb in development mode: <code> ./script/backgroundrb -e development & </code> h2. Call your worker From the command line: <code> $ script/console Loading development environment (Rails 2.0.2) >> MiddleMan.worker(:feeds_worker).update() </code> h2. When things go wrong Asynchronous programming is complex, so expect bugs... h3. Rule #1 know who you're calling. If you give your MiddleMan the wrong name of your worker, he'll just spit this crap at you: <code> You have a nil object when you didn't expect it! The error occurred while evaluating nil.send_request /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in `ask_worker' /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:104:in `process_work' /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:35:in `receive_data' /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in `call' /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in `extract' /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:31:in `receive_data' </code> So for example this command would generate the above mentioned error: <code> MiddleMan.worker(:illegal_worker).update() </code> It's always nice to see a cryptic error messages such as this, it really deserves an award. h3. Check for bugs and bug fixes "git mainline commits":http://gitorious.org/projects/backgroundrb/repos/mainline/logs/master h3. Going to production Starting the daemon: <code> ./script/backgroundrb -e production start </code> h2. Configuring your task to run periodically The following example makes backgroundrb call the FeedsWorker's update method once every 15 minutes: <code> :production: :backgroundrb: :port: 22222 # use port 22222 :lazy_load: true # do not load models eagerly :debug_log: false # disable log workers and other logging # Cron based scheduling :schedules: :feeds_worker: :update: :trigger_args: * */15 * * * * :data: "Hello world" </code> At the time of writing, the cron scheduler seems to be broken, so I prefer hard-coding the interval in the worker's create method: <code> def create add_periodic_timer(15.minutes) { update } end </code> If using Vlad or Capistrano, it's also a good idea to fix script/backgroundrb by changing these lines: <code> pid_file = "#{RAILS_HOME}/../../shared/pids/backgroundrb_#{CONFIG_FILE[:backgroundrb][:port]}.pid" SERVER_LOGGER = "#{RAILS_HOME}/../../shared/log/backgroundrb_server_#{CONFIG_FILE[:backgroundrb][:port]}.log" </code> h3. Resources "Backgroundrb homepage":http://backgroundrb.rubyforge.org/ "Backgroundrb best practices":http://gnufied.org/2008/02/12/backgroundrb-best-practises/ "Backgroundrb scheduling":http://backgroundrb.rubyforge.org/scheduling/index.html "Debugging backgroundrb":http://www.johnyerhot.com/2008/02/11/debugging-backgroundrb/ "Backroundrb's README":http://backgroundrb.rubyforge.org/files/README.html "topfunky's messaging article":http://nubyonrails.com/articles/about-this-blog-beanstalk-messaging-queue 0

2008-03-19T00:35:52+02:00

161

124

Draft… <h2>Planning on using BackgroundDRB? Take a long look at the alternatives first</h2> Ask yourself, do you really need a complex solution like BackgroundDRB? Most likely you don’t, so use a simple daemonized process instead, see <a href="http://snippets.aktagon.com/snippets/212-How-to-create-a-daemon-process-using-Ruby-and-the-daemons-RubyGem">this snippet about the daemons gem</a> for more information. Heck, even a simple Ruby script run by cron every 5 minutes will be more stable than BackgroundDRB and require less work. Even if you really need to process a lot of data asynchronously in the background, I wouldn’t recommend BackgroundDRB, it’s riddled with bugs and unstable in production, so use the <a href="http://agilewebdevelopment.com/plugins/bj">BJ plugin</a> instead. Anyway, continue reading if you want to use BackgroundDRB… <h2>Installing the prerequisites:</h2> <pre class="active4d"> 1 $ sudo gem install chronic packet </pre> <h2>Installing backgroundrb</h2> <pre class="active4d"> 1 $ cd rails_project 2 $ git clone git://gitorious.org/backgroundrb/mainline.git vendor/plugins/backgroundrb </pre> You can also get the latest stable version from the Subversion repository: <pre class="active4d"> 1 svn co http://svn.devjavu.com/backgroundrb/trunk vendor/plugins/backgroundrb </pre> <h2>Setup backgroundrb</h2> <pre class="active4d"> 1 rake backgroundrb:setup </pre> <h2>Create a worker</h2> <pre class="active4d"> 1 ./script/generate worker feeds_worker </pre> <pre class="active4d"> 1 class FeedsWorker < BackgrounDRb::MetaWorker 2 set_worker_name :feeds_worker 3 4 def create(args = nil) 5 # this method is called, when worker is loaded for the first time 6 logger.info "Created feeds worker" 7 end 8 9 def update(data) 10 logger.info "Updating #{Feed.count} feeds." 11 12 seconds = Benchmark.realtime do 13 thread_pool.defer do 14 Feed.update_all() 15 end 16 end 17 18 logger.info "Update took #{'%.5f' % seconds}." 19 end 20 end </pre> <h2>Starting backgroundrb</h2> First configure backgroundrb by opening config/backgroundrb.yml in your editor: <pre class="active4d"> 1 :backgroundrb: 2 :ip: 0.0.0.0 3 4 :development: 5 :backgroundrb: 6 :port: 11111 # use port 11111 7 :log: foreground # foreground mode,print log messages on console 8 9 :production: 10 :backgroundrb: 11 :port: 22222 # use port 22222 </pre> Next, start backgroundrb in development mode: <pre class="active4d"> 1 ./script/backgroundrb -e development & </pre> <h2>Call your worker</h2> From the command line: <pre class="active4d"> 1 $ script/console 2 Loading development environment (Rails 2.0.2) 3 >> MiddleMan.worker(:feeds_worker).update() </pre> <h2>When things go wrong</h2> Asynchronous programming is complex, so expect bugs… <h3>Rule #1 know who you’re calling.</h3> If you give your MiddleMan the wrong name of your worker, he’ll just spit this crap at you: <pre class="active4d"> 1 You have a nil object when you didn't expect it! 2 The error occurred while evaluating nil.send_request 3 /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_master.rb:44:in `ask_worker' 4 /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:104:in `process_work' 5 /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:35:in `receive_data' 6 /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in `call' 7 /usr/local/lib/ruby/gems/1.8/gems/packet-0.1.5/lib/packet/packet_parser.rb:29:in `extract' 8 /Users/christian/Documents/Projects/xxx/vendor/plugins/backgroundrb/server/lib/master_worker.rb:31:in `receive_data' </pre> So for example this command would generate the above mentioned error: <pre class="active4d"> 1 MiddleMan.worker(:illegal_worker).update() </pre> It’s always nice to see a cryptic error messages such as this, it really deserves an award. <h3>Check for bugs and bug fixes</h3> <a href="http://gitorious.org/projects/backgroundrb/repos/mainline/logs/master">git mainline commits</a> <h3>Going to production</h3> Starting the daemon: <pre class="active4d"> 1 ./script/backgroundrb -e production start </pre> <h2>Configuring your task to run periodically</h2> The following example makes backgroundrb call the FeedsWorker’s update method once every 15 minutes: <pre class="active4d"> 1 :production: 2 :backgroundrb: 3 :port: 22222 # use port 22222 4 :lazy_load: true # do not load models eagerly 5 :debug_log: false # disable log workers and other logging 6 # Cron based scheduling 7 :schedules: 8 :feeds_worker: 9 :update: 10 :trigger_args: * */15 * * * * 11 :data: "Hello world" </pre> At the time of writing, the cron scheduler seems to be broken, so I prefer hard-coding the interval in the worker’s create method: <pre class="active4d"> 1 def create 2 add_periodic_timer(15.minutes) { update } 3 end </pre> If using Vlad or Capistrano, it’s also a good idea to fix script/backgroundrb by changing these lines: <pre class="active4d"> 1 pid_file = "#{RAILS_HOME}/../../shared/pids/backgroundrb_#{CONFIG_FILE[:backgroundrb][:port]}.pid" 2 SERVER_LOGGER = "#{RAILS_HOME}/../../shared/log/backgroundrb_server_#{CONFIG_FILE[:backgroundrb][:port]}.log" </pre> <h3>Resources</h3> <a href="http://backgroundrb.rubyforge.org/">Backgroundrb homepage</a> <a href="http://gnufied.org/2008/02/12/backgroundrb-best-practises/">Backgroundrb best practices</a> <a href="http://backgroundrb.rubyforge.org/scheduling/index.html">Backgroundrb scheduling</a> <a href="http://www.johnyerhot.com/2008/02/11/debugging-backgroundrb/">Debugging backgroundrb</a> <a href="http://backgroundrb.rubyforge.org/files/README.html">Backroundrb’s README </a> <a href="http://nubyonrails.com/articles/about-this-blog-beanstalk-messaging-queue">topfunky’s messaging article</a>

Using backgroundrb to execute tasks asynchronously in Rails

2008-08-26T09:38:34+03:00