nginx snippets

How to improve your PageRank with 301 permanent redirects when using Nginx

Tagged seo, www, 301, permanent, nginx, redirect, rewrite, module  Languages 

Mathew Innman of seomoz.org fame wrote about how Digg could increase their revenue by using a so called canonical URL for their whole site. This can be implemented by redirecting users that type in, for example, www.digg.com to digg.com. The reasoning being that instead of having backlinks pointing to two different domains (www and no-www), all backlinks should point to just one, which increases your search engine ranking.

if ($host ~* "www") {
      rewrite ^(.*)$ http://aktagon.com$1 permanent;
      break;
    }

Permanent redirects are also a good idea, if you move your content to a new domain--digg.com to dugg.com, for example...

The syntax for the Nginx rewrite module is documented here.

Creating a local Debian mirror for your Xen servers

Tagged xen, debian, etch, local mirror, amd64, anonftpsync, nginx  Languages bash

Once you've bought a dual or quad Xeon and started to experiment with virtualization you will soon want to create your local mirror to make installs lightning fast. This is a step-by-step how i did it.

First create the Xen that will be our mirror server. The size requirements can be found here: Debian mirror sizes The combined size of amd64 architecture and architecture independent files was 39Gb on 1.9.2007. So I made the image 50Gb big. Remember to change this mirror to a location near you.

xen-create-image --hostname=mirrors.aktagon.com \
--size=50Gb --swap=256Mb --ip=10.0.0.44 \
--netmask=255.255.255.0 --gateway=10.0.0.2 \
--force --dir=/work/vserver --memory=256Mb \
--arch=amd64 \
--kernel=/boot/vmlinuz-2.6.18-5-xen-amd64 \
--debootstrap --dist=etch \
--mirror= http://ftp.fi.debian.org/debian/\
--passwd

Then ssh into your new Xen as root.

ssh -l root mirrors.aktagon.com

Make base configurations for a fresh Xen.

apt-get update && apt-get install locales console-data && dpkg-reconfigure locales

Then get the mirror synchronization script from Debian.

wget "http://www.debian.org/mirror/anonftpsync"
chmod a+x anonftpsync

Then install dependencies for anonftpsync script. Otherwise the script will fail with a -bash: lockfile: command not found error.

apt-get install procmail

Install nginx.

apt-get install nginx

Configure anonftpsync with your favorite editor and change the lines below. These settings will setup a mirror only for amd64 files. You could remove i386 from the excluded architectures, but then a 50Gb image won't fit all the files.

TO=/var/www/debian
RSYNC_HOST=ftp.fi.debian.org
RSYNC_DIR=debian
LOGDIR=/var/log/mirroring
ARCH_EXCLUDE="alpha arm hppa hurd-i386 i386 ia64 m68k mipsel mips powerpc s390 sh sparc source"

Make the log directory.

mkdir -p /var/log/mirroring

Configure nginx by modifying /etc/nginx/nginx.conf with your favorite editor. Just add the autoindex line into server { location / { context

# abbreviated start of file for clarity...
    server {
        listen       80;
        server_name  localhost;

        access_log  /var/log/nginx/localhost.access.log;

        location / {
            root   /var/www;
            # add the line below to allow directory listing
            autoindex  on;
            index  index.html index.htm;
        }
   # abbreviated end of file for clarity...

Do the synchronizing. And wait... for a long while. On a 8/1Mbit cable the first synchronize took roughly 20 hours.

./anonftpsync

Now modify your /etc/apt/sources.list on existing Xen images to use your local mirror. And remember to create new Xen images using your new mirror :) In the above case the URL is http://mirrors.aktagon.com/debian

NB: there is no public mirrors.aktagon.com available... sorry.

Change nginx configuration on the fly

Tagged nginx, reload, configuration  Languages bash

First make your changes to nginx.conf.

Then run the following command to test the new configuration:

# nginx -t -c /etc/nginx/nginx.conf 
2007/10/18 20:55:07 [info] 3125#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2007/10/18 20:55:07 [info] 3125#0: the configuration file /etc/nginx/nginx.conf was tested successfully

Next, look for the process id of the master nginx process:

# ps -ef|grep nginx
root      1911     1  0 18:00 ?        00:00:00 nginx: master process /usr/sbin/nginx
www-data  1912  1911  0 18:00 ?        00:00:00 nginx: worker process

Lastly, tell nginx to reload the configuration and restart the worker processes:

# kill -HUP 1911

Check if a file or directory exists with bash

Tagged nginx, daemon, bash, linux, debian  Languages bash

This script tests if nginx exists and is executable. The script prints a warning and exits, if nginx doesn't exists or isn't executable:

DAEMON=/usr/local/sbin/nbinx
if [ ! -x $DAEMON ]
then
   echo "Couldn't find $DAEMON. Please set path to DAEMON."
   exit 0
fi

See man test for more information on how to use the test command.

Generate a 56-bit DES encrypted (htpasswd) password with Ruby

Tagged apache, ruby, nginx, htpasswd, irb  Languages css

Run the following in an irb console to generate a 56-bit DES encrypted password:

"password".crypt("salt")

The password can be used in an Apache or Nginx htpasswd file to enable basic authentication.

The generated password can also be used in other Unix password files.

Password protecting a folder/resource with Nginx

Tagged nginx, perl, ruby, auth_basic, htpasswd  Languages bash

First add the following to your Nginx configuration file:

location / {
  auth_basic            "Restricted";
  auth_basic_user_file  /etc/nginx/htpasswd;
}

Then create the htpasswd file:

# this be passwords
thisbetheusername:thisbeencryptedpass:yercomment

To generate a htpasswd password without installing Apache you can use the following Perl or Ruby code:

Perl

perl -le 'print crypt("password", "salt")'

Ruby (run in irb)

"password".crypt("salt")

The crypt() method uses 56-bit DES encryption, which is used in /etc/passwd and htpasswd.

Installing nginx on Debian

Tagged nginx, install, debian  Languages bash

DRAFT...

Find latest version of nginx

http://sysoev.ru/en/ http://wiki.codemongers.com/NginxInstallOptions

Install a compiler otherwise: ./configure: error: C compiler gcc is not found

The command:

sudo apt-get install build-essential

Install pre-requisites otherwise you'll get:

Configuration summary + threads are not used + PCRE library is not found + OpenSSL library is not found + md5 library is not used + sha1 library is not used + zlib library is not found

The command:

sudo apt-get install libpcre3 libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev

Much better:

Configuration summary + threads are not used + using system PCRE library + using system OpenSSL library + md5 library is not used + sha1 library is not used + using system zlib library

Compile and install nginx

$ ./configure \
        --sbin-path=/usr/local/sbin \
        --conf-path=/etc/nginx/nginx.conf \
        --pid-path=/var/run/nginx.pid \
        --error-log-path=/var/log/nginx/error.log \
        --http-log-path=/var/log/nginx/access.log \
        --with-http_ssl_module \
        --http-client-body-temp-path=/tmp/nginx_client \
        --http-proxy-temp-path=/tmp/nginx_proxy \
        --http-fastcgi-temp-path=/tmp/nginx_fastcgi
$ make
$ sudo make install

Run the install script

cd /usr/local/src

wget http://sysoev.ru/nginx/nginx-0.5.35.tar.gz

tar zxvf nginx-0.5.35

cd nginx-0.5.35

Create an nginx user and group

$ useradd -g www-data -d /var/www nginx

Create the web server directory

mkdir /var/www
chown root.www-data /var/www
chmod ug=rwx,o= /var/www

Test configuration

nginx -t
2008/03/09 20:51:05 [info] 5034#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2008/03/09 20:51:05 [info] 5034#0: the configuration file /etc/nginx/nginx.conf was tested successfully

Start nginx

nginx

nginx startup script for Debian

Tagged debian, nginx, start, script, linux  Languages bash
sudo vim /etc/init.d/nginx

Paste in the following (remember to run 'set :paste' in VIM when pasting):

#! /bin/sh
##
# nginx start script
##

PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/usr/local/sbin/nginx
NAME=nginx
DESC=nginx

if [ ! -x $DAEMON ]
then
   echo "Couldn't find $DAEMON. Please set path to DAEMON."
   exit 0
fi


# Include nginx defaults if available
if [ -f /etc/default/nginx ] ; then
    . /etc/default/nginx
fi

set -e

case "$1" in
  start)
    echo -n "Starting $DESC: "
    start-stop-daemon --start --pidfile /var/run/$NAME.pid \
        --exec $DAEMON -- $DAEMON_OPTS
    echo "$NAME."
    ;;
  stop)
    echo -n "Stopping $DESC: "
    start-stop-daemon --stop --pidfile /var/run/$NAME.pid \
        --exec $DAEMON
    echo "$NAME."
    ;;
  restart|force-reload)
    echo -n "Restarting $DESC: "
    start-stop-daemon --stop --pidfile \
        /var/run/$NAME.pid --exec $DAEMON
    sleep 1
    start-stop-daemon --start --pidfile \
        /var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS
    echo "$NAME."
    ;;
  reload)
      echo -n "Reloading $DESC configuration: "
      start-stop-daemon --stop --signal HUP --pidfile /var/run/$NAME.pid \
          --exec $DAEMON
      echo "$NAME."
      ;;
  *)
    N=/etc/init.d/$NAME
    echo "Usage: $N {start|stop|restart|force-reload}" >&2
    exit 1
    ;;
esac

exit 0

Now make the script executable with this command:

sudo chmod 755 /etc/init.d/nginx

Lastly, run this command to make the script run when the server starts and stops:

sudo /usr/sbin/update-rc.d -f nginx defaults

How to use Vlad the Deployer with git, nginx, mongrel, mongrel_cluster and Rails

Tagged vlad, deployer, deploy, capistrano, nginx, mongrel, mongrel_cluster  Languages ruby

This is a draft...

Installing Vlad the Deployer

gem install vlad

Configuring Vlad the Deployer

Add this to the end of RakeFile:

begin
  require 'rubygems'
  require 'vlad'
  Vlad.load :scm => :git
rescue LoadError => e
  puts "Unable to load Vlad #{e}."
end

Note that we're telling Vlad to use git. This snippet gives you a quick introduction on how to use git with Rails.

Creating the deployment recipe

If you're uncertain what these variables mean, have a look at the docs. This folder is also worth a look, and don't forget to take a peek at the vlad source code.

#
# General configuration
#
set :ssh_flags,             '-p 666'
set :application,           'xxx.com'
set :domain,                '127.0.01'
set :deploy_to,             '/var/www/xxx.com'
set :repository,            '/var/lib/git/repositories/xxx.com/.git/'


#
# Mongrel configuration
#
set :mongrel_clean,         true
set :mongrel_command,       'sudo mongrel_rails'
set :mongrel_group,         'www-data'
set :mongrel_port,          9000
set :mongrel_servers,       3

#set :mongrel_address,       '127.0.0.1'
#set(:mongrel_conf)          { '#{shared_path}/mongrel_cluster.conf' }
#set :mongrel_config_script, nil
#set :mongrel_environment,   'production'
#set :mongrel_log_file,      nil
#set :mongrel_pid_file,      nil
#set :mongrel_prefix,        nil
#set :mongrel_user,          'mongrel'

#
# Customize Vlad to our needs
#
namespace :vlad do
  #
  # Add an after_update hook
  #
  remote_task :update do
    Rake::Task['vlad:after_update'].invoke
  end

  #
  # The after_update hook, which is run after vlad:update
  #
  remote_task :after_update do
  # Link to shared resources, if you have them in .gitignore
  #  run "ln -s #{deploy_to}/shared/system/database.yml #{deploy_to}/current/config/database.yml"
  end

  #
  # Deploys a new version of your application
  #
  remote_task :deploy => [:update, :migrate, :start_app]
end

Setup the server

$ rake vlad:setup

This will create the necessary folders and mongrel_cluster configuration file.

Deploy the application

Now deploy the application with vlad:deploy, which is a custom rake task that we added to the deployment recipe:

$ rake vlad:deploy

Copying your SSH public key to the remote server

Vlad uses ssh for executing commands on the remotely, and rsync for copying the build to your server, which means you'll quickly grow tired of typing your password each time a command is run.

This problem is solved by copying your public SSH keys to the remote server, this snippet explains how to do exactly that.