How to fetch URLs in parallell with EventMachine and Ruby

Ruby posted about 14 hours ago by christian

Save time by doing things in parallell:

   1  require 'rubygems'
   2  require 'eventmachine'
   3  require 'open-uri'
   4  require 'pp'
   5  require 'thread'
   6  require 'benchmark'
   7  
   8  class Worker
   9    include EM::Deferrable
  10  
  11    def run
  12      get_google
  13      set_deferred_status :succeeded
  14    end
  15  end
  16  
  17  def get_google
  18    # sorry for spamming you
  19    open('http://www.google.com/') do |f|
  20      #pp f.meta
  21    end
  22  end
  23  
  24  def asynchronous(i)
  25    worker = Worker.new
  26    # on success
  27    worker.callback do
  28      p "#{Thread.current} done #{i}!"
  29    end 
  30    worker.errback do 
  31      p "Unexpected error"    
  32      EM.stop  
  33    end
  34    #
  35    Thread.new do
  36      worker.run
  37      EM.stop
  38    end 
  39    #puts "scheduling done!"
  40  end
  41  
  42  def synchronous(i)
  43    get_google
  44  end
  45  
  46  # on error
  47  EM.error_handler do |e|  
  48    p "Unexpected error: #{e}" 
  49  end
  50  
  51  EM.run do
  52    seconds = Benchmark.realtime do
  53      50.times do |i|
  54        asynchronous i
  55      end
  56    end
  57    p "With EventMachine: #{seconds} elapsed..."
  58  
  59    seconds = Benchmark.realtime do
  60      50.times do |i|
  61        synchronous i
  62      end
  63    end
  64    p "Without EventMachine: #{seconds} elapsed..."
  65  end

Output:

   1  With EventMachine: 9.05974316596985 elapsed...
   2  Without EventMachine: 19.1381118297577 elapsed...

Conclusion

  • Speeds up blocking operations.
  • EventMachine is currently limited to one CPU core (native thread) per process.

References

Tagged eventmachine, ruby, asynchronous, job

How to use dual-purpose accessors in Ruby to create a DSL

Ruby posted about 1 month ago by christian

Instead of this:

   1  Sitemap('public/sitemap.xml') do
   2    self.stylesheet = 'public/sitemap.xls'  
   3    self.ping = ['http://www.google.com', 'http://www.google.com']  
   4  end

You could write this:

   1  Sitemap('public/sitemap.xml') do
   2    stylesheet 'public/sitemap.xls'  
   3    ping ['http://www.google.com', 'http://www.google.com']  
   4  end

Using dual-purpose accessors:

   1  class Sitemap
   2    def stylesheet(path = nil) 
   3      return @path unless path
   4     @path = path
   5    end
   6    alias_method :stylesheet=, :stylesheet 
   7    ...
   8  end

Tagged ruby, dsl

Fix: Library not loaded: /opt/local/lib/libreadline.6.dylib

Shell Script (Bash) posted 3 months ago by christian

If you get this error:

   1  % script/console
   2  Loading development environment (Rails 2.3.4)
   3  /Library/Ruby/Site/1.8/universal-darwin10.0/readline.bundle: dlopen(/Library/Ruby/Site/1.8/universal-darwin10.0/readline.bundle, 9): Library not loaded: /opt/local/lib/libreadline.6.dylib (LoadError)
   4    Referenced from: /Library/Ruby/Site/1.8/universal-darwin10.0/readline.bundle
   5    Reason: image not found - /Library/Ruby/Site/1.8/universal-darwin10.0/readline.bundle
   6          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/completion.rb:10
   7          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/init.rb:253:in `require'
   8          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/init.rb:253:in `load_modules'
   9          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/init.rb:251:in `each'
  10          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/init.rb:251:in `load_modules'
  11          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb/init.rb:21:in `setup'
  12          from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/irb.rb:54:in `start'
  13          from /usr/bin/irb:13

then this might help:

   1  mv /Library/Ruby/Site/1.8/universal-darwin10.0/readline.bundle /tmp

If not move it back. I got the error after having used and uninstalled MacPorts. Using Homebrew now…

Tagged readline, ruby, fuckup

How to pretty print JSON data with Ruby

Ruby Experimental posted 4 months ago by christian

This ugly one-liner will pretty print the JSON in countries.yml, so that it’s easier to read and edit:

   1  cat db/seeds/countries.yml | ruby -rubygems -e 'require "json";a=JSON.pretty_generate JSON.parse gets;File.open("db/seeds/countries.yml", "w"){|f| f << a}'

Tagged ruby, json, pretty print

Simple Webrick proxy.

Ruby posted 4 months ago by marko

A handy oneliner which I found here.

   1  ruby -r webrick/httpproxy -e 's = WEBrick::HTTPProxyServer.new(:Port => 9999, :RequestCallback => Proc.new{|req,res| puts req.request_line, req.raw_header}); trap("INT"){s.shutdown}; s.start'

Tagged ruby, webrick, proxy