Register now and start sharing your code snippets.
-->

Logging nginx to remote loghost with syslog-ng.

Shell Script (Bash) posted 3 months ago by marko

Nginx does not support syslog by default, so you have to patch it with a third party module. This snippet relies on that you have configured xstow. See this snippet for instructions on xstow configuration.

   1  cd /tmp
   2  wget http://sysoev.ru/nginx/nginx-0.6.32.tar.gz
   3  tar zxvf nginx-0.6.32.tar.gz
   4  mv nginx-0.6.32 src-0.6.32-orig
   5  wget "http://wiki.codemongers.com/NginxModules?action=AttachFile&do=get&target=syslog.patch" -O syslog.patch
   6  patch -p0 < syslog.patch
   7  cd src-0.6.32-orig
   8  ./configure --prefix=/usr/local/stow/nginx --with-syslog
   9  make
  10  sudo make install
  11  cd /usr/local/stow
  12  sudo xstow nginx

I-am-a-noob-at-syslog-disclaimer: This might be a totally wrong way to configure the server and client(s), so it is subject for refinement. In my experience it works though.

Configuring the client. Add the following lines to the end of /etc/syslog-ng/syslog-ng.conf and restart syslog-ng with /etc/init.d/syslog-ng restart. Nginx logs in facility local5 and the hostname of the loghost is “loghost”. You could just as well use the IP of the loghost.

   1  filter f_local5 { facility(local5); };
   2  destination d_loghost {tcp("loghost" port(514));};
   3  log { source(s_all); filter(f_local5); destination(d_loghost); };

Configuring the server. Add the following lines to the end of /etc/syslog-ng/syslog-ng.conf and restart syslog-ng with /etc/init.d/syslog-ng restart. Also if you run a cluster of nginx servers it might be wise to put all the output in one file, instead of separate files per host.

   1  source s_remote { tcp(); };
   2  destination d_clients { file("/var/log/HOSTS/nginx.$HOST"); };
   3  log { source(s_remote); destination(d_clients); };

Test the logging by running this from the client.

   1  logger -p local5.info Hubbabubba

Tagged nginx, loghost, remote logging, syslog-ng

Fixing how nginx sends request URI to the backend server.

Shell Script (Bash) posted 4 months ago by marko

I needed to send an URI through Nginx to the backend servers (Mongrel) intact. However Nginx was constantly unescaping the URI , and removed slashes in the process. That resulted in an invalid URL at the backend server. The fix was simple, but extremely hard to find. Just remove the trailing slash from the proxy_pass directive, like below.

Invalid URI is sent to the backend server with this configuration of Nginx.

   1  proxy_pass         http://backend1:3000/;

Valid, untampered URI is sent with this configuration of Nginx.

   1  proxy_pass         http://backend1:3000;

Tagged nginx, uri unescape fix

Awstats LogFormat configuration for nginx LogFormat

Plain Text posted 5 months ago by christian

This is my nginx LogFormat configuration:

   1  log_format main '$remote_addr - $remote_user [$time_local] $status '
   2                      '"$request" $body_bytes_sent "$http_referer" '
   3                      '"$http_user_agent" "http_x_forwarded_for"';

And this is my Awstats LogFormat configuration:

   1  LogFormat = "%host - %host_r %time1 %code %methodurl %bytesd %refererquot %uaquot %otherquot"

Tagged awstats, nginx, logformat, configuration

How to use Vlad the Deployer with git, nginx, mongrel, mongrel_cluster and Rails

Ruby posted 9 months ago by christian

This is a draft…

Installing Vlad the Deployer

   1  gem install vlad

Configuring Vlad the Deployer

Add this to the end of RakeFile:

   1  begin
   2    require 'rubygems'
   3    require 'vlad'
   4    Vlad.load :scm => :git
   5  rescue LoadError => e
   6    puts "Unable to load Vlad #{e}."
   7  end

Note that we’re telling Vlad to use git. This snippet- gives you a quick introduction on how to use git with Rails.

Creating the deployment recipe

If you’re uncertain what these variables mean, have a look at the docs. This folder is also worth a look, and don’t forget to take a peek at the vlad source code.

   1  #
   2  # General configuration
   3  #
   4  set :ssh_flags,             '-p 666'
   5  set :application,           'xxx.com'
   6  set :domain,                '127.0.01'
   7  set :deploy_to,             '/var/www/xxx.com'
   8  set :repository,            '/var/lib/git/repositories/xxx.com/.git/'
   9  
  10  
  11  #
  12  # Mongrel configuration
  13  #
  14  set :mongrel_clean,         true
  15  set :mongrel_command,       'sudo mongrel_rails'
  16  set :mongrel_group,         'www-data'
  17  set :mongrel_port,          9000
  18  set :mongrel_servers,       3
  19  
  20  #set :mongrel_address,       '127.0.0.1'
  21  #set(:mongrel_conf)          { '#{shared_path}/mongrel_cluster.conf' }
  22  #set :mongrel_config_script, nil
  23  #set :mongrel_environment,   'production'
  24  #set :mongrel_log_file,      nil
  25  #set :mongrel_pid_file,      nil
  26  #set :mongrel_prefix,        nil
  27  #set :mongrel_user,          'mongrel'
  28  
  29  #
  30  # Customize Vlad to our needs
  31  #
  32  namespace :vlad do
  33    #
  34    # Add an after_update hook
  35    #
  36    remote_task :update do
  37      Rake::Task['vlad:after_update'].invoke
  38    end
  39  
  40    #
  41    # The after_update hook, which is run after vlad:update
  42    #
  43    remote_task :after_update do
  44    # Link to shared resources, if you have them in .gitignore
  45    #  run "ln -s #{deploy_to}/shared/system/database.yml #{deploy_to}/current/config/database.yml"
  46    end
  47  
  48    #
  49    # Deploys a new version of your application
  50    #
  51    remote_task :deploy => [:update, :migrate, :start_app]
  52  end

Setup the server

   1  $ rake vlad:setup

This will create the necessary folders and mongrel_cluster configuration file.

Deploy the application

Now deploy the application with vlad:deploy, which is a custom rake task that we added to the deployment recipe:

   1  $ rake vlad:deploy

Copying your SSH public key to the remote server

Vlad uses ssh for executing commands on the remotely, and rsync for copying the build to your server, which means you’ll quickly grow tired of typing your password each time a command is run.

This problem is solved by copying your public SSH keys to the remote server, this snippet- explains how to do exactly that.

Tagged vlad, deployer, deploy, capistrano, nginx, mongrel, mongrel_cluster

nginx startup script for Debian

Shell Script (Bash) posted 10 months ago by christian

   1  sudo vim /etc/init.d/nginx

Paste in the following (remember to run ‘set :paste’ in VIM when pasting):

   1  #! /bin/sh
   2  ##
   3  # nginx start script
   4  ##
   5  
   6  PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
   7  DAEMON=/usr/local/sbin/nginx
   8  NAME=nginx
   9  DESC=nginx
  10  
  11  if [ ! -x $DAEMON ]
  12  then
  13     echo "Couldn't find $DAEMON. Please set path to DAEMON."
  14     exit 0
  15  fi
  16  
  17  
  18  # Include nginx defaults if available
  19  if [ -f /etc/default/nginx ] ; then
  20  	. /etc/default/nginx
  21  fi
  22  
  23  set -e
  24  
  25  case "$1" in
  26    start)
  27  	echo -n "Starting $DESC: "
  28  	start-stop-daemon --start --pidfile /var/run/$NAME.pid \
  29  		--exec $DAEMON -- $DAEMON_OPTS
  30  	echo "$NAME."
  31  	;;
  32    stop)
  33  	echo -n "Stopping $DESC: "
  34  	start-stop-daemon --stop --pidfile /var/run/$NAME.pid \
  35  		--exec $DAEMON
  36  	echo "$NAME."
  37  	;;
  38    restart|force-reload)
  39  	echo -n "Restarting $DESC: "
  40  	start-stop-daemon --stop --pidfile \
  41  		/var/run/$NAME.pid --exec $DAEMON
  42  	sleep 1
  43  	start-stop-daemon --start --pidfile \
  44  		/var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS
  45  	echo "$NAME."
  46  	;;
  47    reload)
  48        echo -n "Reloading $DESC configuration: "
  49        start-stop-daemon --stop --signal HUP --pidfile /var/run/$NAME.pid \
  50            --exec $DAEMON
  51        echo "$NAME."
  52        ;;
  53    *)
  54  	N=/etc/init.d/$NAME
  55  	echo "Usage: $N {start|stop|restart|force-reload}" >&2
  56  	exit 1
  57  	;;
  58  esac
  59  
  60  exit 0

Now make the script executable with this command:

   1  sudo chmod 755 /etc/init.d/nginx

Lastly, run this command to make the script run when the server starts and stops:

   1  sudo /usr/sbin/update-rc.d -f nginx defaults

Tagged nginx, start, script, linux, debian