Once you’ve bought a dual or quad Xeon and started to experiment with virtualization you will soon want to create your local mirror to make installs lightning fast. This is a step-by-step how i did it.

First create the Xen that will be our mirror server. The size requirements can be found here: Debian mirror sizes The combined size of amd64 architecture and architecture independent files was 39Gb on 1.9.2007. So I made the image 50Gb big. Remember to change this mirror to a location near you.

   1  xen-create-image --hostname=mirrors.aktagon.com \
   2  --size=50Gb --swap=256Mb --ip=10.0.0.44 \
   3  --netmask=255.255.255.0 --gateway=10.0.0.2 \
   4  --force --dir=/work/vserver --memory=256Mb \
   5  --arch=amd64 \
   6  --kernel=/boot/vmlinuz-2.6.18-5-xen-amd64 \
   7  --debootstrap --dist=etch \
   8  --mirror= http://ftp.fi.debian.org/debian/\
   9  --passwd

Then ssh into your new Xen as root.

   1  ssh -l root mirrors.aktagon.com

Make base configurations for a fresh Xen.

   1  apt-get update && apt-get install locales console-data && dpkg-reconfigure locales

Then get the mirror synchronization script from Debian.

   1  wget "http://www.debian.org/mirror/anonftpsync"
   2  chmod a+x anonftpsync 

Then install dependencies for anonftpsync script. Otherwise the script will fail with a -bash: lockfile: command not found error.

   1  apt-get install procmail

Install nginx.

   1  apt-get install nginx

Configure anonftpsync with your favorite editor and change the lines below. These settings will setup a mirror only for amd64 files. You could remove i386 from the excluded architectures, but then a 50Gb image won’t fit all the files.

   1  TO=/var/www/debian
   2  RSYNC_HOST=ftp.fi.debian.org
   3  RSYNC_DIR=debian
   4  LOGDIR=/var/log/mirroring
   5  ARCH_EXCLUDE="alpha arm hppa hurd-i386 i386 ia64 m68k mipsel mips powerpc s390 sh sparc source"

Make the log directory.

   1  mkdir -p /var/log/mirroring

Configure nginx by modifying /etc/nginx/nginx.conf with your favorite editor. Just add the autoindex line into server { location / { context

   1  # abbreviated start of file for clarity...
   2      server {
   3          listen       80;
   4          server_name  localhost;
   5  
   6          access_log  /var/log/nginx/localhost.access.log;
   7  
   8          location / {
   9              root   /var/www;
  10              # add the line below to allow directory listing
  11              autoindex  on;
  12              index  index.html index.htm;
  13          }
  14     # abbreviated end of file for clarity...

Do the synchronizing. And wait… for a long while. On a 8/1Mbit cable the first synchronize took roughly 20 hours.

   1  ./anonftpsync

Now modify your /etc/apt/sources.list on existing Xen images to use your local mirror. And remember to create new Xen images using your new mirror :) In the above case the URL is http://mirrors.aktagon.com/debian

NB: there is no public mirrors.aktagon.com available… sorry.

Mathew Innman of seomoz.org fame wrote about how Digg could increase their revenue by using a so called canonical URL for their whole site. This can be implemented by redirecting users that type in, for example, www.digg.com to digg.com. The reasoning being that instead of having backlinks pointing to two different domains (www and no-www), all backlinks should point to just one, which increases your search engine ranking.

   1  if ($host ~* "www") {
   2        rewrite ^(.*)$ http://aktagon.com$1 permanent;
   3        break;
   4      }

Permanent redirects are also a good idea, if you move your content to a new domain—digg.com to dugg.com, for example…

The syntax for the Nginx rewrite module is documented here.