681 snippets – displaying 1–30

Finnish SSID/HETU generator in Ruby

module Ssid
  module Fake
    #
    # Generates fake Finnish SSIDs (HETU).
    #
    # Original Python version:
    # https://gist.github.com/puumuki/11172310
    #
    class Finnish
      CHECK_KEYS = "0123456789ABCDEFHJKLMNPRSTUVWXY"
      CENTURIES = { '18' => '+',
                    '19' => '-',
                    '20' => 'A' }
      def self.generate(start_year=1800, end_year=2014)
        year = start_year + rand(start_year - end_year)
        month = 1 + rand(12)
        day = 1 + rand(days_in_month(year, month))
        century_sep = CENTURIES[year.to_s[0..1]]
        order_num = 2 + rand(889)
        check_number = "%02d%02d%s%03d" % [ day, month, year.to_s[0..1], order_num ]
        check_number_index = check_number.to_i % 31
        key = CHECK_KEYS[check_number_index]
        "%02d%02d%s%s%03d%s" % [ day, month, year.to_s[0..1], century_sep, order_num, key ]
      end

      private

      def self.days_in_month(year, month)
        (Date.new(year, 12, 31) << (12-month)).day
      end
    end
  end
end

How to get notified when a Cron job fails

To get notified when a cron script fails you can tell cron to send any errors to you by email.

It already does this by default, but the emails are sent to the Unix account’s mailbox:

cat /var/spool/mail/<username> | less

To use a different email address use the MAILTO setting as described here:

MAILTO=email@example.com
0 */2 * * * /bin/backup.sh > /dev/null

Cron will now send an email whenever the script (/bin/backup.sh) prints something to stderr.

If the script is printing to stderr without it failing try Cronic:

0 */2 * * * cronic /bin/backup.sh

You can of course also try to fix the script by not printing to STDERR unless it’s really an error.

Check the mail log to see if it’s working:

sudo tail -f /var/log/mail.log

If you’re email server is rejecting emails sent by cron you might need to set the MAILFROM variable. However, only some cron version supports MAILFROM.

How to mark deprecated methods in Ruby

This code can be used to mark deprecated methods in Ruby:

module Kernel
  def mark_deprecated(new_method)
    warn "DEPRECATED: the #{__method__} called from #{caller.first} has been deprecated and replaced by #{new_method}!!!"
  end
end

Example:

class Horse
  def self.shit
    mark_deprecated("Bull.shit")
  end
end

Checklist for how to write background jobs/workers (Sidekiq/Resque/DelayedJob/<XXX>)

Checklist

  1. Idempotent
    Jobs should be idempotent. If a job is performed multiple times the work should only be performed once, or at least the end-result should be the same.
  1. Optimistic/pessimistic locking
    Use locking strategies to prevent code from being executed in parallel. This can happen if you have misconfigured Sidekiq or queued the same job twice. To solve this use, for example, database locks, optimistic/pessimistic locking, or database constraints.
  1. Retry
    Set retry to zero for jobs that don’t need to or shouldn’t be retried.
  1. Priority
    Specify a queue and a queue priority for jobs that need a higher or lower priority.
  1. Transactions
    Does the job need database transactions? Use transactions e.g. when a job does more than one insert, update, delete, or create, otherwise you will end up having data duplication/inconsistency issues.
  1. Messaging
    Sidekiq is not a messaging queue. At-least-once execution is 100% guaranteed by Sidekiq Pro. I don’t trust that guarantee 100%, but that is another story. The question is: what happens if the job fails (dead job queue, dev null)? What you usually need is “job successfully executed at-least-once” not “job executed at-least once”.

Ensuring jobs are performed at-least-once and at-most-once

You can’t rely on Sidekiq, or any other messaging or job queue, to execute your job (business logic) once and exactly once. All work needs to be checked for consistency.

For example, messages/jobs can be lost under many circumstances, and the same job (business logic) can be executed an unlimited amount of times unless your application takes the necessary steps to ensure that the job is performed correctly. A background processing/messaging server should be considered unreliable.

At-least-once

Use a reconsiliation step, e.g. a separate worker that runs once per day, to make sure jobs are executed at least once: http://www.mikeperham.com/2014/05/27/the-reconciliation-step/

  1. Databases: At-least-once is achieved when you have the data in your database and are polling to see if the job has been performed. Locking => performance issues.
  2. Sidekiq: At-least-once is achieved when you monitor the dead job queue 100% using available eyeballs.
  3. Messaging queues: At-least-once is achieved through message acknowledgment and/or through monitoring the dead letter queue 100% using available eyeballs.

At-most-once

Use a flag, nonce, or a guid to ensure a job is only run once, i.e. do nothing if a job is queued twice for some reason. Avoid locking database tables.

Conclusion

Don’t trust your messaging and background processing server to do its job without either manual supervision (eyeballs) or automated supervision (at-least-once and at-most-once).

Using Ruby 2.2.0 with Rails 3.x

rubyrailschristianVersion 4

Steps for upgrading a Rails 3.x app to Ruby 2.2.0:

  1. Install Ruby 2.2.0, e.g. brew update; brew upgrade ruby-build; rbenv install 2.2.0
  2. Set app’s ruby version to 2.2.0, e.g. rbenv local 2.2.0
  3. Update gems that are broken, e.g. bundle update pg eventmachine rubocop
  4. Add test-unit to Gemfile to fix “cannot load such file — test/unit/testcase (LoadError)” error:

gem 'test-unit'

  1. Rollback to 2.1 when you notice all warnings, related to e.g. Comparable, or continue with the warnings

JavaScript Promises With Q.js

This Q.js example calls two asynchronous methods in sequence:

class InitializeApp
  constructor: ->
    findDevice = ->
      df = Q.defer()
      onSuccess = (devices) ->
        console.log "Finding device"
        df.resolve("device 2")
      setTimeout(onSuccess, 2000)
      df.promise

    connectToDevice = (id) ->
      df = Q.defer()
      onSuccess = ->
        console.log "Connecting to #{id}"
        df.resolve("success")
      setTimeout(onSuccess, 2000)
      df.promise

    findDevice().then(connectToDevice)

Usage:

new InitializeApp()
# ...Sleep 2 seconds
# => Finding device
# ...Sleep 2 seconds
# => Connecting to device 2

How to use Font Awesome in iOS/Phonegap apps

First, install Font Awesome:

  1. Download the latest Font Awesome version
  2. Double-click fontawesome-webfont.ttf and install it on your laptop/desktop (optional)
  3. Copy fontawesome.css to www/css
  4. Remove the references to font files in fontawesome.css, i.e. remove @font-face declaration

Then in Xcode:

  1. Copy fontawesome-webfont.ttf into “Your Project/Resources/fonts/fontawesome-webfont.ttf”
  2. Add Font Awesome to the .plist file by selecting the “Info” tab
  3. Add a new property named “Fonts provided by application” and set the value to “fontawesome-webfont.ttf”

Clean and build your project. Font Awesome is now ready to be used.

Phonegap example:

<i class="fa fa-cog"/>

How to hide the status bar in a Phonegap iOS app

To hide the status bar in a phonegap project:

  1. Double click on your project, or .plist, in XCode
  2. Select the tab named “Info”
  3. Add a new property by selecting an item in the list and then clicking the plus sign
  4. Name the property “View controller-based status bar appearance”
  5. Set the property value to “NO”

Next:

  1. Select the tab named general
  2. Check the “Hide status bar” located under “Deployment Info”

Done.

How to Open Safari's Web Inspector With Applescript to Debug a Phonegap App

This applescript will make Safari’s Web Inspector connect to a Phonegap application running on my iPhone that was started through XCode:

#!/usr/bin/osascript

set device_name to "Christian Hellsten iPhone"
tell application "Safari"
	activate
	tell application "System Events"
		try
			click menu item "index.html" of menu device_name of menu item device_name of menu "Utvecklare" of menu bar item "Utvecklare" of menu bar 1 of application process "Safari"
		end try
	end tell
end tell

Save the file, as e.g. open-web-inspector.applescript, then execute it:

chmod +x open-web-inspector.applescript
./open-web-inspector.applescript

For English version, change “Utvecklare” to “Develop”.

Troubleshooting

  1. Error -1719 most likely means you need to enable assistive access for iTerm or whatever app you’re running the script from.
  1. Error -10810 means “bad luck” in Applescript. Try restarting the computer.

If all else fails, run the script using the Applescript Editor. Also see, this script.

Tested on OSX Yosemite.

CoffeeScript variable visibility

coffeescriptchristianVersion 3

This Coffeescript:

class Device
  x: 1 # Instance variable
  y = 2 # Private instance variable
  @z: 3 # Class variable

 # Address and name are instance variables
  constructor: (@address, @name) ->

gives us this JavaScript:

Device = (function() {
  var y;
  Device.prototype.x = 1;
  y = 2;
  function Device(address, name) {
    this.address = address;
    this.name = name;
  }
  return Device;
})();

and the following visibility:

dev = new Device('X', 'Y') # => Device {address: "X", name: "Y", x: 1}
dev.y # => undefined
Device.z # => 3

How to Debug Phonegap Applications With Safari's Remote Debugger

  1. On your iPhone/iPad: Enable Web Inspector . Open “Settings” → “Safari” → “Advanced”.
  2. On your desktop/laptop: Enable Developer tools in Safari. Open “Preferences” → “Advanced” check “Show Develop menu in menu bar”.
  3. Connect your iPhone via USB to your desktop/laptop.
  4. Start the application on your iPad/iPhone via XCode.
  5. On your desktop/laptop: Open the Safari Web Inspect debugger. Go to “Preferences” → “Your phone” → “index.html”.

You can automate the last step by using applescript to open Safari’s Web Inspector.

Note: Use CMD+R to reload index.html.

Phonegap Autorotate

In MainViewController#shouldAutorotateToInterfaceOrientation return YES:

- (BOOL)shouldAutorotateToInterfaceOrientation:(UIInterfaceOrientation)interfaceOrientation
{
    // Return YES for supported orientations
    return YES; //[super shouldAutorotateToInterfaceOrientation:interfaceOrientation];
}

Running a Phonegap App in a Specific Version of iPhone

Cordova:

cordova emulate ios --target="iPhone-4s"
cordova emulate ios --target="iPad-Air"
cordova emulate ios --target="iPhone-6"
cordova emulate ios --target="iPhone-6-Plus"

Phonegap:

phonegap run ios --target="iPhone-4s"
phonegap run ios --target="iPad-Air"
phonegap run ios --target="iPhone-6"
phonegap run ios --target="iPhone-6-Plus"

Gulpfile.js example with Slim, Coffeescript, Sass, Browser Sync, Lint

slimgulpchristianVersion 7

A Gulpfile.js example with Slim, Coffeescript, Sass, Browser Sync, Lint. The script is suitable for building e.g. static websites and prototyping mobile apps.

Gulpfile expects your project to be structured in the following way (configurable):
- src/css/.scss,.css
- src/js/.coffee,.js
- src/*.slim

Output is written to “www” folder.

#
# Gulpfile with:
#
# - Slim
# - Sass
# - Lint
# - Browsersync
# - CSS and HTML compression
#
# Install dependencies:
#
#   $ npm install gulp gulp-concat gulp-uglify event-stream gulp-coffee gulp-sass gulp-cssmin gulp-coffee gulp-coffeelint browser-sync gulp-util gulp-shell
#
# Then start developing:
#
#   $ gulp
#

gulp        = require 'gulp'
concat      = require 'gulp-concat'
es          = require('event-stream')
sass        = require 'gulp-sass'
uglify      = require 'gulp-uglify'
streamqueue = require 'streamqueue' # Preserves file order (vendor...)
coffee      = require 'gulp-coffee'
gutil       = require 'gulp-util'
shell       = require 'gulp-shell'
cssmin      = require 'gulp-cssmin'
coffeelint  = require 'gulp-coffeelint'
browserSync = require 'browser-sync'

isProd = gutil.env.type is 'prod'

sources =
  sass: 'src/css/**/*.scss'
  css: 'src/css/**/*.css'
  html: 'src/**/*.slim'
  js: 'src/js/**/*.js'
  coffee: 'src/js/**/*.coffee'

targets =
  css: 'www/css'
  html: 'www/'
  js: 'www/js'

# Check for errors
gulp.task 'lint', ->
  gulp.src(sources.js)
    .pipe(coffeelint())
    .pipe(coffeelint.reporter())

# Compile Coffeescript
gulp.task 'js', ->
  stream = streamqueue(objectMode: true)
  # Vendor files
  stream.queue(gulp.src(sources.js))
  # App files use Coffee
  stream.queue(gulp.src(sources.coffee).pipe(coffee(bare:true)))
  stream.done()
    .pipe(concat("all.js"))
    .pipe(if isProd then uglify() else gutil.noop())
    .pipe(gulp.dest(targets.js))

# Compile Slim
gulp.task 'slim', ->
  gulp.src(sources.html)
    .pipe(shell(["slimrb -r ./lib/helpers.rb -p <%= file.path %> > ./#{targets.html}/<%= file.relative.replace(\".slim\", \".html\") %>"]))

# Compile CSS
gulp.task 'css', ->
  stream = streamqueue(objectMode: true)
  # Vendor files
  stream.queue(gulp.src(sources.css))
  # App files
  stream.queue(gulp.src(sources.sass).pipe(sass(style: 'expanded', includePaths: ['src/css'], errLogToConsole: true)))
  stream.done()
    .pipe(concat("all.css"))
    .pipe(if isProd then uglify() else gutil.noop())
    .pipe(gulp.dest(targets.css))

# Reload browser
gulp.task 'server', ->
  browserSync.init null,
    open: true
    server:
      baseDir: targets.html
    reloadDelay: 2000 # Prevent white screen of death
    watchOptions:
      debounceDelay: 1000

# Watch files for changes
gulp.task 'watch', ->
  gulp.watch sources.js, ['js']
  gulp.watch sources.css, ['css']
  gulp.watch sources.html, ['slim']
  gulp.watch 'www/**/**', (file) ->
    browserSync.reload(file.path) if file.type is "changed"

# Build everything
gulp.task 'build', ['lint', 'js', 'css', 'slim']

# Start a server and watch for file changes
gulp.task 'default', ['watch', 'server']

Writing a DSL in Ruby

rubydslchristianVersion 5

When defining a simple DSL instance_eval is your friend:

class Worker
  attr_reader :where
  def call(where, &block)
    @where = where
    instance_eval &block
  end

  def cut_trees
    puts "Cutting trees"
  end

  def make_planks
    puts "Making planks"
  end

  def i(what, &block)
    instance_eval &block
  end
end

def work where, &block
  puts "Working #{where}"
  Worker.new.(where, &block)
  puts "Going home"
end

Use the DSL like this:

work "in the woods" do
  cut_trees
  make_planks
  i "take a coffee break" do
    puts "Cooking coffee #{where}"
  end
end

Output is:

Working in the woods
Cutting trees
Making planks
Cooking coffee in the woods
Going home

Ruby's call method

rubycallchristianVersion 4

What does this print:

class RubyCall
  def call(*args)
    puts "Call: #{args}"
  end
end

xxx = RubyCall.new
xxx.("hello", 1, 2, 3)

It calls the #call method and prints “Call: [”hello", 1, 2, 3]".

If you want you can do this:

xxx::call('adsf')
xxx.call('adsf')

How to record and replay TCP traffic

tcpreplaychristianVersion 2

First attempt

1) Record TCP traffic with tcpdump to pcap file

Record traffic on ethic from host 196.0.0.1 and port 2332, write to app-traffic.pcap:

tcpdump -vvv -i eth0 host 192.168.0.1 and port 2332 -w app-traffic.pcap &

2) View captured traffic

tcpdump -qns 0 -X -r app-traffic.pcap

Edit captured traffic if needed with Wireshark.

3) Edit source IP so that it’s on your own network

See the ”example in tcprewrite documentation”:http://tcpreplay.synfin.net/wiki/tcprewrite for details on how to rewrite source IP:

$ tcprewrite --pnat=10.0.0.0/8:172.16.0.0/12,192.168.0.0/16:172.16.0.0/12 --infile=input.pcap --outfile=output.pcap --skipbroadcast

4) Replay traffic with tcplivereplay from recorded pcap file (note step #3)

”See tcpliveplay documentation”:http://tcpreplay.synfin.net/wiki/tcpliveplay for details.

Note that tcpliveplay is only available on Linux not OSX.

Second attempt

Record:

sudo tcpdump -i en0 host 192.168.0.1 and port 2332 -w app-traffic.pcap


View:

tcpdump -s 0 -n -e -x -vvv -r app-traffic.pcap

Replay:

sudo tcpreplay -i en0 -t -K app-traffic.pcap

Or use tcplivereplay (Note: Linux only):
http://tcpreplay.synfin.net/wiki/tcpliveplay#tcpliveplay

Result

I couldn’t get it to work….

Big Search Field & Button = Beautiful?

<html>
<head>
<link rel="stylesheet" href="//cdnjs.cloudflare.com/ajax/libs/meyer-reset/2.0/reset.css"/>
<style>

.search-form {
  width: 40%;
  min-width: 400px;
  border: 6px solid rgba(0, 0, 0, 0.7);
  margin: 20px auto 0;
}

input.text {
  float: left;
  width: 80%;
  border: 5px solid #fff;
  margin: 0;
  outline: none;
  font-size: 17px;
  -webkit-border-radius: 0;
  -moz-border-radius: 0;
  -ms-border-radius: 0;
  -o-border-radius: 0;
  border-radius: 0;
  -webkit-appearance: none;
  color: #3c3c3c;
}


input.text, input.button {
  padding: 15px;
  font-size: 17px;
  font-weight: 800;
  display: inline-block;
  -webkit-transition: all 0.3s ease;
  -moz-transition: all 0.3s ease;
  -o-transition: all 0.3s ease;
  transition: all 0.3s ease;
}

input.button {
  float: right;
  width: 20%;
  margin: 0;
  border: 5px solid grey;
  color: #fff;
  background-color: grey;
  -webkit-appearance: none;
  padding-left: 0;
  padding-right: 0;
  -webkit-border-radius: 0;
  -moz-border-radius: 0;
  -ms-border-radius: 0;
  -o-border-radius: 0;
  border-radius: 0;
}

.clearfix {
  overflow: auto;
}

* {
  font-family: "Helvetica", sans-serif;
  -webkit-box-sizing: border-box;
  -moz-box-sizing: border-box;
  box-sizing: border-box;
}

</style>
</head>
<body>
  <div class="search-form clearfix">
    <input name="query" class="text" type="text" placeholder="Find your dog&hellip;">
    <input type="submit" value="Search" class="button">
  </div>
</body>
</html>

Eventmachine Echo Server Example

require 'rubygems'
require 'eventmachine'
require 'logger'

#
# Each connection creates a new EchoServer.
#
module EchoServer
  LOG = Logger.new('echo.log')

  #
  # Called by the event loop immediately after the network connection has been
  # established, and before resumption of the network loop.
  #
  def post_init
    LOG.info "-- Connection established #{remote_ip} --"
  end

  #
  # Called by the event loop whenever data has been received by the network
  # connection. It is never called by user code.
  #
  def receive_data data
    message = "#{remote_ip}: #{data}"
    LOG.info message
    send_data message
    close_connection if data =~ /quit/i
  end

  #
  # Called by the framework whenever a connection (either a server or client
  # connection) is closed.
  #
  def unbind
    LOG.info "-- Connection closed #{remote_ip} --"
  end

  #
  # Return the IP and port of the remote client.
  #
  def remote_ip
    @remote_ip ||= begin
                     port, ip = Socket.unpack_sockaddr_in(get_peername)
                     "#{ip}:#{port}"
                   end
  end
end

EventMachine::run do
  Signal.trap("INT")  { EventMachine.stop }
  Signal.trap("TERM") { EventMachine.stop }
  EventMachine::start_server "0.0.0.0", 8080, EchoServer
end

Use telnet to test it:

$ telnet localhost 8080

Daemonization Best-Practices

Don’t daemonize your daemons

<quote>

What’s the result? In development mode, your process will run in the foreground, as yourself and log to stdout: perfect for developers. In production mode, the init system will run your process as a configured user with logging sent to a specific location and log rotated automatically. Less system administration, easier debugging, simpler code, all because you leveraged the init system to do the work for you!</quote>

Running processes

<quote>

Note: In every case, it’s assumed that you have a program that wants to run that does not daemonize on its own. Self-daemonizing programs start you down the path to hell. You can’t use any sane keepalive techniques so you have to resort to polling process lists or checking the pid or something. Even managing that pidfile gets hard when you combine it with things that change their own uid for safety (because you should never run anything as root).</quote>

Ubuntu, upstart, and creating a pid for monitoring

<quote>

If start-stop-daemon is available on your machine, I would highly recommend using it to launch your process. start-stop-daemon will handle launching the process as an unprivileged user without forking from sudo or su (recommended in the upstart cookbook) AND it also has built in support for pid file management.</quote>

Foreman

<quote>

Foreman is a manager for Procfile-based applications. Its aim is to abstract away the details of the Procfile format, and allow you to either run your application directly or export it to some other process management format.</quote>

How to introduce Rubocop in your project

Installation

Edit Gemfile:

group :development do
  gem 'rubocop' # Style guide checker
  gem 'guard-rubocop' # Rubocop plugin for Guard
end

Install:

$ bundle

Usage

Run Rubocop:

$ rubocop --auto-gen-config

Tweaking

Edit .rubocop.yml:

inherit_from: .rubocop_todo.yml

Introduce new rules gradually in priority order by enabling and moving rules from .rubocop_todo.yml to .rubocop.yml.

You can also use the Ruby style-guide’s rubocop configuration:
https://github.com/bbatsov/rubocop/blob/master/config/default.yml

Workflow integration

Run:

$ guard init rubocop

Edit Guardfile:

guard :rubocop, all_on_start: false, cli: ['--format', 'clang', '--rails'] do
  watch(%r{.+\.rb$})
  watch(%r{(?:.+/)?\.rubocop\.yml$}) { |m| File.dirname(m[0]) }
end

Editor Integration

Optionally, integrate Rubocop with your editor:

CI integration

See How to introduce rubocop and integrate into Jenkins.

How to parse a cronline/crontab/cron with Ruby

You can use the rufus-scheduler gem to parse cronline/crontab/cron with Ruby.

Gemfile:

gem 'rufus-scheduler'

Example:

require 'rufus/scheduler/cronline'
cron = Rufus::Scheduler::CronLine.new("00 00 1 * *") # 00:00 every first day of the month
#
next_time = cron.next_time(Time.current)
previous_time = cron.previous_time(Time.current)

rufus-scheduler’s cronline implementation:
https://github.com/jmettraux/rufus-scheduler/blob/master/lib/rufus/scheduler/cronline.rb

See tests for details and syntax:
https://github.com/jmettraux/rufus-scheduler/blob/master/spec/cronline_spec.rb

Cron configuration examples.