Your typical server log directory looks something like this:

   1  access.log
   2  access.log.1
   3  access.log.1.gz
   4  access.log.2.gz

Now what if you want to extract data from day x to day y? You could use gzip and grep to uncompress the files, but there’s a better way: the z commands.

All you have to do is:

   1  zgrep "2010" /var/log/apache2/access.log*

More info on the subject can be found here: http://www.thegeekstuff.com/2009/05/zcat-zless-zgrep-zdiff-zcmp-zmore-gzip-file-operations-on-the-compressed-files/

First you need to enable the mod_expires module:

   1  a2enmod expires

Next add this to your configuration:

   1  ExpiresActive On
   2  # Set Expires header to current time by default
   3  ExpiresDefault A0
   4  
   5  <FilesMatch "\.(flv|ico|pdf|avi|mov|ppt|doc|mp3|wmv|wav)$">
   6    ExpiresDefault "access plus 30 days"
   7  </FilesMatch>
   8  
   9  <FilesMatch "\.(jpg|jpeg|png|gif|swf|bmp|)$">
  10    ExpiresDefault "access plus 7 days"
  11  </FilesMatch>
  12  
  13  <FilesMatch "\.(txt|xml|js|css)$">
  14    ExpiresDefault "access plus 1 day"
  15  </FilesMatch>

Now restart Apache:

   1  $ sudo /etc/init.d/apache2 force-reload

Check that the proper headers are set with Firebug, Yahoo YSlow or Google Page speed.

In config/initializers/paperclip.rb put:

   1  if RAILS_ENV == "development"
   2    Paperclip.options[:command_path] = '/opt/local/bin/'
   3  else
   4    Paperclip.options[:command_path] = '/usr/bin/'
   5  end

Note image_magick_path is deprecated alias of command_path.

Only supports the combined format at the moment…

   1  class ApacheLog
   2    FORMATS = {
   3      :combined => %r{^(\S+) - - \[(\S+ \+\d{4})\] "(\S+ \S+ [^"]+)" (\d{3}) (\d+|-) "(.*?)" "([^"]+)"$}
   4    }
   5    
   6    class << self
   7      def each_line(log_file, log_format = FORMATS[:combined])
   8  
   9        f = File.open(log_file, "r")
  10  
  11        f.each_line do|line|
  12          data = line.scan(log_format).flatten
  13  
  14          if data.empty?
  15            p "Line didn't match pattern: #{line}"
  16  
  17            next
  18          end
  19  
  20          yield data
  21        end
  22      end
  23    end
  24  end
  25  
  26  
  27  log_file   = ARGV[0]
  28  
  29  ApacheLog.each_line(log_file) do |data|
  30    host, date, url_with_method, status, size, referrer, agent = data
  31  end

First enable the mod_headers module:

   1  sudo a2enmod headers

Then add this to your apache2.conf:

   1  # Hide X-Powered-By and Server headers
   2  Header always unset "X-Powered-By"
   3  ServerTokens Prod
   4  ServerSignature Off

Now restart Apache:

   1  /etc/init.d/apache2 force-reload

This is security through obscurity at it’s finest…