configuration snippets

Change nginx configuration on the fly

Tagged nginx, reload, configuration  Languages bash

First make your changes to nginx.conf.

Then run the following command to test the new configuration:

# nginx -t -c /etc/nginx/nginx.conf 
2007/10/18 20:55:07 [info] 3125#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2007/10/18 20:55:07 [info] 3125#0: the configuration file /etc/nginx/nginx.conf was tested successfully

Next, look for the process id of the master nginx process:

# ps -ef|grep nginx
root      1911     1  0 18:00 ?        00:00:00 nginx: master process /usr/sbin/nginx
www-data  1912  1911  0 18:00 ?        00:00:00 nginx: worker process

Lastly, tell nginx to reload the configuration and restart the worker processes:

# kill -HUP 1911

Sphinx configuration file template

Tagged sphinx, template, configuration  Languages 
source feed_items
        type                    = mysql

        sql_host                =
        sql_user                = root
        sql_pass                =
        sql_db                  = xxx_production
        sql_port                = 3306  # optional, default is 3306
        sql_sock                = /var/run/mysqld/mysqld.sock

        sql_query_pre           = SET NAMES utf8
        #sql_query_pre          = SET SESSION query_cache_type=OFF

    # Unique ID should be first column
        sql_query               = \
                SELECT, i.title,,, f.title FROM feed_items i LEFT JOIN feeds f ON = i.feed_id

index feed_items
        source                  = feed_items
        path                    = /var/sphinx/xxx
        morphology              = libstemmer_sv
        charset_type            = utf-8

        mem_limit               = 32M

        address                 =
        port                    = 3312
        log                     = /var/log/sphinx/searchd.log
        query_log               = /var/log/sphinx/query.log
        pid_file                = /var/log/
        max_matches             = 1000

Awstats LogFormat configuration for nginx LogFormat

Tagged awstats, nginx, logformat, configuration  Languages 

This is my nginx LogFormat configuration:

log_format main '$remote_addr - $remote_user [$time_local] $status '
                    '"$request" $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "http_x_forwarded_for"';

And this is my Awstats LogFormat configuration:

LogFormat = "%host - %host_r %time1 %code %methodurl %bytesd %refererquot %uaquot %otherquot"

My .vimrc configuration

Tagged vimrc, vim, configuration  Languages 
set guifont=Monaco:h12

autocmd FileType ruby,eruby set omnifunc=rubycomplete#Complete
autocmd FileType ruby,eruby let g:rubycomplete_rails = 1

let leader = ","

set grepprg=ack
set grepformat=%f:%l:%m

set hidden
set history=1000
set tabstop=2
set smarttab
set shiftwidth=2
set autoindent
set expandtab
set backspace=start,indent
set ignorecase
set smartcase
set scrolloff=3
set backupdir=~/.vim-tmp,~/.tmp,~/tmp,/var/tmp,/tmp
set directory=~/.vim-tmp,~/.tmp,~/tmp,/var/tmp,/tmp
set ruler
set number
set hlsearch
set incsearch " ...dynamically as they are typed.
set visualbell

nnoremap <C-e> 3<C-e>
nnoremap <C-y> 3<C-y>

runtime macros/matchit.vim

autocmd FileType make     set noexpandtab
autocmd FileType python   set noexpandtab

syntax on
filetype on
filetype plugin on
filetype indent on

let g:fuzzy_ignore = "*.log"
let g:fuzzy_matching_limit = 70

map ,d :execute 'NERDTreeToggle ' . getcwd()<CR>
map ,t :FuzzyFinderTextMate<CR>
map ,b :FuzzyFinderBuffer<CR>
map ,f gg=G

set columns=160                                                     "set width of window off open
set lines=50                                                        "height of window off open

set gcr=a:blinkwait500-blinkon1300-blinkoff150                      "tune the blinking of the cursor, short on, long off.
set gcr=i:hor10                                                     "underline cursor. not too thick not too thin. goldielocks style
set gcr=v:block                                                     "selecting should cover the text

map ,c :ruby finder.rescan!

set fuopt+=maxhorz                      " grow to maximum horizontal width on entering fullscreen mode

A simple configuration framework for storing a Rails app's configuration in YAML

Tagged yaml, rails, configuration, ostruct  Languages ruby

Also see:

Create config/initializers/configuration.rb:

require 'ostruct'

# From
class NestedOstruct
  def self.to_ostruct(hash)
    mapping = {
      Hash => ->(s) { to_ostruct(s) },
      Array => ->(s) { s.collect { |i| i.is_a?(Hash) ? to_ostruct(i) : i } }
    } {}.tap { |h| hash.each { |k, v| h[k] = (mapping[v.class] || ->(s) { s })[v] } }

config = YAML.load_file(Rails.root.join('config', 'application.yml'))[Rails.env].symbolize_keys
AppConfig = NestedOstruct.to_ostruct(config)

Create config/application.yml:

    api_key: xxx
    api_secret: xxx

Use it in your application:


How to Secure an nginx Server with Fail2Ban

Tagged nginx, fail2ban, filters, configuration, security, throttling  Languages apacheconf

Our Security Requirements

  • Block anyone trying to run scripts (.pl, .cgi, .exe, etc)
  • Block anyone trying to use the server as a proxy
  • Block anyone failing to authenticate using nginx basic authentication
  • Block anyone failing to authenticate using our application's log in page
  • Block bad bots
  • Limit the number of connections per session

We could use ModSecurity to support these requirements, but it's not compatible with nginx. We want a lightweight and easy-to-use solution. We can fulfill all these requirements with fail2ban and nginx.

Configuring Fail2ban

All except the last requirement of connection throttling is supported by Fail2Ban. To start blocking unwanted guests, put this in Fail2Ban's jail.conf file:

enabled = true
filter = nginx-auth
action = iptables-multiport[name=NoAuthFailures, port="http,https"]
logpath = /var/log/nginx*/*error*.log
bantime = 600 # 10 minutes
maxretry = 6

enabled = true
filter = nginx-login
action = iptables-multiport[name=NoLoginFailures, port="http,https"]
logpath = /var/log/nginx*/*access*.log
bantime = 600 # 10 minutes
maxretry = 6
enabled  = true
filter = apache-badbots
action = iptables-multiport[name=BadBots, port="http,https"]
logpath = /var/log/nginx*/*access*.log
bantime = 86400 # 1 day
maxretry = 1
enabled = true
action = iptables-multiport[name=NoScript, port="http,https"]
filter = nginx-noscript
logpath = /var/log/nginx*/*access*.log
maxretry = 6
bantime  = 86400 # 1 day
enabled = true
action = iptables-multiport[name=NoProxy, port="http,https"]
filter = nginx-proxy
logpath = /var/log/nginx*/*access*.log
maxretry = 0
bantime  = 86400 # 1 day

Filter configuration

The following filter configuration files are stored in /etc/fail2ban/filter.d/:

# Proxy filter /etc/fail2ban/filter.d/nginx-proxy.conf:
# Block IPs trying to use server as proxy.
# Matches e.g.
# - - "GET
failregex = ^<HOST> -.*GET http.*
ignoreregex =
# Noscript filter /etc/fail2ban/filter.d/nginx-noscript.conf:
# Block IPs trying to execute scripts such as .php, .pl, .exe and other funny scripts.
# Matches e.g.
# - - "GET /something.php
failregex = ^<HOST> -.*GET.*(\.php|\.asp|\.exe|\.pl|\.cgi|\scgi)
ignoreregex =
# Auth filter /etc/fail2ban/filter.d/nginx-auth.conf:
# Blocks IPs that fail to authenticate using basic authentication
failregex = no user/password was provided for basic authentication.*client: <HOST>
            user .* was not found in.*client: <HOST>
            user .* password mismatch.*client: <HOST>
ignoreregex =
# Login filter /etc/fail2ban/filter.d/nginx-login.conf:
# Blocks IPs that fail to authenticate using web application's log in page
# Scan access log for HTTP 200 + POST /sessions => failed log in
failregex = ^<HOST> -.*POST /sessions HTTP/1\.." 200
ignoreregex =

Testing fail2ban Rules

You should test your fail2ban rules with the fail2ban-regex command:

# Test against a log file
# fail2ban-regex <log file> <filter configuration>
fail2ban-regex /var/log/nginx/access.log /etc/fail2ban/filter.d/nginx-login.conf 

# Test using strings
# fail2ban-regex <e.g. row from log file> <failure regex from filter configuration>
fail2ban-regex " - - [21/Feb/2012:10:31:08 +0200] \"POST /sessions HTTP/1.1\" 200 1532 \"\" \"Mozilla/5.0 Safari/535.11\" \"-\"" "^<HOST> -.*POST /sessions HTTP/1\..\" 200"

Limiting the number of connections

We limit the number of connections to 2 per second and allow a burst of 50 with the following nginx configuration:

http {
    limit_req_zone  $binary_remote_addr  zone=app:10m   rate=2r/s; 
    server {
        location / {
            limit_req   zone=app burst=50;

Note that you could also use Rack::Attack to implement many of these requirements.


Where should I store my application configuration in Sinatra?

Tagged configuration, sinatra, settings  Languages ruby

This is one option:

require 'yaml'
require 'ostruct'
require 'sinatra/base'

# Usage:
# config/application.yml:
#   development:
#     username: ...
#   production:
#     username: ...
#   url: http://...
# app.rb:
# class App < Sinatra::Base                    
#   register Sinatra::Configuration            
#   load_configuration 'config/application.yml'
#   get '/' do
#     App.configuration.username
#   end
module Sinatra
  module Configuration
    # From
    class NestedOstruct
      def{}){}){|r,p| r[p[0]] = p[1].kind_of?(Hash) ?[1]) : p[1]; r })

      def self.to_ostruct(object)
        return case object
        when Hash
          object = object.clone
          object.each do |key, value|
            object[key] = to_ostruct(value)

        when Array
          object = object.clone
! { |i| to_ostruct(i) }

    def self.registered(base)
      base.set :environments, %w[test development staging production]

    def load_configuration(file, namespace = :configuration)
      set namespace, parse_configuration(YAML.load_file(file)) || {}


    def parse_configuration(hash)
      base = hash.except(*environments)

  register Configuration
  Delegator.delegate :load_configuration

Git Configuration

Tagged git, configuration  Languages bash

Makes git output look better:

git config format.pretty oneline
git config color.ui true

Dot notation for Ruby configuration hash

Tagged ruby, configuration, dot-notation  Languages ruby

Convert a normal hash into a hash that allows e.g. this:

CONFIG['session.ttl'] => 60
CONFIG[''] => KeyError


def to_namespace_hash(object, prefix = nil)
  if object.is_a? Hash do |key, value|
      if prefix
        to_namespace_hash value, "#{prefix}.#{key}"
        to_namespace_hash value, "#{key}"
    { prefix => object }

CONFIG = ... # Load hash from YAML/JSON/database/etc
CONFIG = to_namespace_hash(CONFIG).freeze

def CONFIG.[](key)
rescue KeyError => e
  possible_keys { |x| x if x.match /.*?#{key}.*?/i }.delete_if(&:blank?).join("\n")
  fail KeyError, "Key '#{key}' not found. Did you mean one of:\n#{possible_keys}"