benchmark snippets

How to benchmark your Ruby code

Tagged benchmark, performance, ruby  Languages ruby

You can easily benchmark your Ruby code like this:

require 'benchmark'
seconds = Benchmark.realtime do
    sleep 1
end
print "#{seconds} elapsed..."

The output should be close to 1 second.

How to profile your Rails and Ruby applications with ruby-prof

Tagged ruby-prof, rails, gem, profiling, benchmark, profile, kcachegrind  Languages ruby

Installing ruby-prof

First install ruby-prof:

git clone git://github.com/jeremy/ruby-prof.git
cd ruby-prof/
rake gem
sudo gem install pkg/ruby-prof-0.6.1.gem

Note that version 0.6.0 doesn't work, at least not with Rails 2.1.1. With 0.6.0 I got this message:

gem install ruby-prof to use the profiler

Setting up a new environment for profiling

Create config/environments/profiling.rb:

config.cache_classes = true
config.action_controller.consider_all_requests_local = false
config.action_controller.perform_caching             = true
config.action_view.cache_template_loading            = true

#config.log_level = :debug

Add the new environment to database.yml. You might want to reuse the development database.

Creating a profiling script

Next we'll create a script that simply fetches the homepage, save the following code in profiling/homepage.rb:

get '/'
say "GET / => #{path}"

Run the script

Now run the script 100 times:

RAILS_ENV=profiling ./script/performance/request -n 100 profiling/homepage.rb

Profiling plain Ruby applications

You can also profile a block of code by calling RubyProf from your code:

require 'ruby-prof'

# Profile the code
RubyProf.start
...
[code to profile]
...
results = RubyProf.stop

File.open "#{RAILS_ROOT}/tmp/profile-graph.html", 'w' do |file|
  RubyProf::GraphHtmlPrinter.new(results).print(file)
end

File.open "#{RAILS_ROOT}/tmp/profile-flat.txt", 'w' do |file|
  RubyProf::FlatPrinter.new(results).print(file)
end

File.open "#{RAILS_ROOT}/tmp/profile-tree.prof", 'w' do |file|
  RubyProf::CallTreePrinter.new(results).print(file)
end

Analyzing results

I prefer to use the RubyProf::CallTreePrinter to output data that kcachegrind can read. The HTML and text data is difficult to read so kcachegrind will definitely make your life easier.

On OSX you can install kcachegrind with Fink (or DarwinPorts):

sudo apt-get update ; sudo apt-get install fink
sudo apt-get install kcachegrind

There's also WinCacheGrind and MacCacheGrind, but I haven't tried those.

Timing and Benchmarking Minitest Tests

Tagged ruby, minitest, benchmark, timing  Languages ruby

Put this in test/support/timing.rb:

module MiniTestTiming
  class << self; attr_accessor :failure_threshold end

  def before_setup
    @start_time = Time.now
    super
  end

  def after_teardown
    super
    end_time = Time.now
    duration = (end_time - @start_time).seconds.round(1)
    time = duration.to_s + "s"
    name = "#{self.class.name}.#{__name__}"
    limit = MiniTestTiming.failure_threshold
    File.open 'log/test_duration.log', 'a' do |f|
      f << "#{time.ljust(5)}: #{name}\n"
    end
    assert duration < limit, "#{name} took too long (#{time} > #{limit}s)"
  end
end

# Set threshold for failing tests to 1 second
MiniTestTiming.failure_threshold = 1

class MiniTest::Test
  include MiniTestTiming
end

The time it takes your tests will show up in log/test_duration.log.

You can also output the time it took to run a test by using the verbose switch:

rake test TESTOPTS="--verbose"

Web Microframework Benchmark

Tagged benchmark, elixir, clojure, golang, ruby  Languages 

Results on an old MacBook Air:

Ruby 2.1.0 + Hobbit + Hat + puma (-t 8 -w 2) - ~100-150 req/s (Hat = Hobbit app template with i18n, asset pipeline, etc)
Ruby 2.1.0 + Hobbit + Hat custom + puma (-t 8 -w 2) - ~1500 req/s (Hat without asset pipeline)
Ruby 2.1.0 + Hobbit + puma (-t 8 -w 2) - ~1600 req/s
Ruby 2.1.0 + rack + puma (-t 8 -w 2) - ~1600 req/s
Golang 1.3.1 + net/http - ~2700 req/s
Elixir 1.0.0 + Phoenix 0.4.1 - ~1300 req/s
Clojure 1.6.0 + ring 1.3.1 - ~5000 req/s
Clojure 1.6.0 + ring 1.3.1 + slim - ~270 req/s

YMMV.