Logging nginx to remote loghost with syslog-ng.
Nginx does not support syslog by default, so you have to patch it with a third party module. This snippet relies on that you have configured xstow. See this snippet for instructions on xstow configuration.
1 cd /tmp 2 wget http://sysoev.ru/nginx/nginx-0.6.32.tar.gz 3 tar zxvf nginx-0.6.32.tar.gz 4 mv nginx-0.6.32 src-0.6.32-orig 5 wget "http://wiki.codemongers.com/NginxModules?action=AttachFile&do=get&target=syslog.patch" -O syslog.patch 6 patch -p0 < syslog.patch 7 cd src-0.6.32-orig 8 ./configure --prefix=/usr/local/stow/nginx --with-syslog 9 make 10 sudo make install 11 cd /usr/local/stow 12 sudo xstow nginx
I-am-a-noob-at-syslog-disclaimer: This might be a totally wrong way to configure the server and client(s), so it is subject for refinement. In my experience it works though.
Configuring the client. Add the following lines to the end of /etc/syslog-ng/syslog-ng.conf and restart syslog-ng with /etc/init.d/syslog-ng restart. Nginx logs in facility local5 and the hostname of the loghost is “loghost”. You could just as well use the IP of the loghost.
1 filter f_local5 { facility(local5); }; 2 destination d_loghost {tcp("loghost" port(514));}; 3 log { source(s_all); filter(f_local5); destination(d_loghost); };
Configuring the server. Add the following lines to the end of /etc/syslog-ng/syslog-ng.conf and restart syslog-ng with /etc/init.d/syslog-ng restart. Also if you run a cluster of nginx servers it might be wise to put all the output in one file, instead of separate files per host.
1 source s_remote { tcp(); }; 2 destination d_clients { file("/var/log/HOSTS/nginx.$HOST"); }; 3 log { source(s_remote); destination(d_clients); };
Test the logging by running this from the client.
1 logger -p local5.info Hubbabubba
Fixing how nginx sends request URI to the backend server.
I needed to send an URI through Nginx to the backend servers (Mongrel) intact. However Nginx was constantly unescaping the URI , and removed slashes in the process. That resulted in an invalid URL at the backend server. The fix was simple, but extremely hard to find. Just remove the trailing slash from the proxy_pass directive, like below.
Invalid URI is sent to the backend server with this configuration of Nginx.
1 proxy_pass http://backend1:3000/;
Valid, untampered URI is sent with this configuration of Nginx.
1 proxy_pass http://backend1:3000;
Awstats LogFormat configuration for nginx LogFormat
This is my nginx LogFormat configuration:
1 log_format main '$remote_addr - $remote_user [$time_local] $status ' 2 '"$request" $body_bytes_sent "$http_referer" ' 3 '"$http_user_agent" "http_x_forwarded_for"';
And this is my Awstats LogFormat configuration:
1 LogFormat = "%host - %host_r %time1 %code %methodurl %bytesd %refererquot %uaquot %otherquot"
How to use Vlad the Deployer with git, nginx, mongrel, mongrel_cluster and Rails
This is a draft…
Installing Vlad the Deployer
1 gem install vlad
Configuring Vlad the Deployer
Add this to the end of RakeFile:
1 begin 2 require 'rubygems' 3 require 'vlad' 4 Vlad.load :scm => :git 5 rescue LoadError => e 6 puts "Unable to load Vlad #{e}." 7 end
Note that we’re telling Vlad to use git. This snippet- gives you a quick introduction on how to use git with Rails.
Creating the deployment recipe
If you’re uncertain what these variables mean, have a look at the docs. This folder is also worth a look, and don’t forget to take a peek at the vlad source code.
1 # 2 # General configuration 3 # 4 set :ssh_flags, '-p 666' 5 set :application, 'xxx.com' 6 set :domain, '127.0.01' 7 set :deploy_to, '/var/www/xxx.com' 8 set :repository, '/var/lib/git/repositories/xxx.com/.git/' 9 10 11 # 12 # Mongrel configuration 13 # 14 set :mongrel_clean, true 15 set :mongrel_command, 'sudo mongrel_rails' 16 set :mongrel_group, 'www-data' 17 set :mongrel_port, 9000 18 set :mongrel_servers, 3 19 20 #set :mongrel_address, '127.0.0.1' 21 #set(:mongrel_conf) { '#{shared_path}/mongrel_cluster.conf' } 22 #set :mongrel_config_script, nil 23 #set :mongrel_environment, 'production' 24 #set :mongrel_log_file, nil 25 #set :mongrel_pid_file, nil 26 #set :mongrel_prefix, nil 27 #set :mongrel_user, 'mongrel' 28 29 # 30 # Customize Vlad to our needs 31 # 32 namespace :vlad do 33 # 34 # Add an after_update hook 35 # 36 remote_task :update do 37 Rake::Task['vlad:after_update'].invoke 38 end 39 40 # 41 # The after_update hook, which is run after vlad:update 42 # 43 remote_task :after_update do 44 # Link to shared resources, if you have them in .gitignore 45 # run "ln -s #{deploy_to}/shared/system/database.yml #{deploy_to}/current/config/database.yml" 46 end 47 48 # 49 # Deploys a new version of your application 50 # 51 remote_task :deploy => [:update, :migrate, :start_app] 52 end
Setup the server
1 $ rake vlad:setup
This will create the necessary folders and mongrel_cluster configuration file.
Deploy the application
Now deploy the application with vlad:deploy, which is a custom rake task that we added to the deployment recipe:
1 $ rake vlad:deploy
Copying your SSH public key to the remote server
Vlad uses ssh for executing commands on the remotely, and rsync for copying the build to your server, which means you’ll quickly grow tired of typing your password each time a command is run.
This problem is solved by copying your public SSH keys to the remote server, this snippet- explains how to do exactly that.
nginx startup script for Debian
1 sudo vim /etc/init.d/nginx
Paste in the following (remember to run ‘set :paste’ in VIM when pasting):
1 #! /bin/sh 2 ## 3 # nginx start script 4 ## 5 6 PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 7 DAEMON=/usr/local/sbin/nginx 8 NAME=nginx 9 DESC=nginx 10 11 if [ ! -x $DAEMON ] 12 then 13 echo "Couldn't find $DAEMON. Please set path to DAEMON." 14 exit 0 15 fi 16 17 18 # Include nginx defaults if available 19 if [ -f /etc/default/nginx ] ; then 20 . /etc/default/nginx 21 fi 22 23 set -e 24 25 case "$1" in 26 start) 27 echo -n "Starting $DESC: " 28 start-stop-daemon --start --pidfile /var/run/$NAME.pid \ 29 --exec $DAEMON -- $DAEMON_OPTS 30 echo "$NAME." 31 ;; 32 stop) 33 echo -n "Stopping $DESC: " 34 start-stop-daemon --stop --pidfile /var/run/$NAME.pid \ 35 --exec $DAEMON 36 echo "$NAME." 37 ;; 38 restart|force-reload) 39 echo -n "Restarting $DESC: " 40 start-stop-daemon --stop --pidfile \ 41 /var/run/$NAME.pid --exec $DAEMON 42 sleep 1 43 start-stop-daemon --start --pidfile \ 44 /var/run/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS 45 echo "$NAME." 46 ;; 47 reload) 48 echo -n "Reloading $DESC configuration: " 49 start-stop-daemon --stop --signal HUP --pidfile /var/run/$NAME.pid \ 50 --exec $DAEMON 51 echo "$NAME." 52 ;; 53 *) 54 N=/etc/init.d/$NAME 55 echo "Usage: $N {start|stop|restart|force-reload}" >&2 56 exit 1 57 ;; 58 esac 59 60 exit 0
Now make the script executable with this command:
1 sudo chmod 755 /etc/init.d/nginx
Lastly, run this command to make the script run when the server starts and stops:
1 sudo /usr/sbin/update-rc.d -f nginx defaults