Creating a local Debian mirror for your Xen servers

3 Once you've bought a dual or quad Xeon and started to experiment with virtualization you will soon want to create your local mirror to make installs lightning fast. This is a step-by-step how i did it. First create the Xen that will be our mirror server. The size requirements can be found here: "Debian mirror sizes":http://www.debian.org/mirror/size The combined size of amd64 architecture and architecture independent files was 39Gb on 1.9.2007. So I made the image 50Gb big. Remember to change this mirror to a location near you. <code> xen-create-image --hostname=mirrors.aktagon.com \ --size=50Gb --swap=256Mb --ip=10.0.0.44 \ --netmask=255.255.255.0 --gateway=10.0.0.2 \ --force --dir=/work/vserver --memory=256Mb \ --arch=amd64 \ --kernel=/boot/vmlinuz-2.6.18-5-xen-amd64 \ --debootstrap --dist=etch \ --mirror= http://ftp.fi.debian.org/debian/\ --passwd </code> Then ssh into your new Xen as root. <code> ssh -l root mirrors.aktagon.com </code> Make base configurations for a fresh Xen. <code> apt-get update && apt-get install locales console-data && dpkg-reconfigure locales </code> Then get the mirror synchronization script from Debian. <code> wget "http://www.debian.org/mirror/anonftpsync" chmod a+x anonftpsync </code> Then install dependencies for anonftpsync script. Otherwise the script will fail with a -bash: lockfile: command not found error. <code> apt-get install procmail </code> Install nginx. <code> apt-get install nginx </code> Configure anonftpsync with your favorite editor and change the lines below. These settings will setup a mirror _only_ for amd64 files. You could remove i386 from the excluded architectures, but then a 50Gb image won't fit all the files. <code> TO=/var/www/debian RSYNC_HOST=ftp.fi.debian.org RSYNC_DIR=debian LOGDIR=/var/log/mirroring ARCH_EXCLUDE="alpha arm hppa hurd-i386 i386 ia64 m68k mipsel mips powerpc s390 sh sparc source" </code> Make the log directory. <code> mkdir -p /var/log/mirroring </code> Configure nginx by modifying /etc/nginx/nginx.conf with your favorite editor. Just add the autoindex line into server { location / { context <code> # abbreviated start of file for clarity... server { listen 80; server_name localhost; access_log /var/log/nginx/localhost.access.log; location / { root /var/www; # add the line below to allow directory listing autoindex on; index index.html index.htm; } # abbreviated end of file for clarity... </code> Do the synchronizing. And wait... for a long while. On a 8/1Mbit cable the first synchronize took roughly 20 hours. <code> ./anonftpsync </code> Now modify your /etc/apt/sources.list on existing Xen images to use your local mirror. And remember to create new Xen images using your new mirror :) In the above case the URL is http://mirrors.aktagon.com/debian NB: there is no public mirrors.aktagon.com available... sorry. 0

2007-09-01T15:03:15+03:00

131

Once you’ve bought a dual or quad Xeon and started to experiment with virtualization you will soon want to create your local mirror to make installs lightning fast. This is a step-by-step how i did it. First create the Xen that will be our mirror server. The size requirements can be found here: <a href="http://www.debian.org/mirror/size">Debian mirror sizes</a> The combined size of amd64 architecture and architecture independent files was 39Gb on 1.9.2007. So I made the image 50Gb big. Remember to change this mirror to a location near you. <pre class="active4d"> 1 xen-create-image --hostname=mirrors.aktagon.com \ 2 --size=50Gb --swap=256Mb --ip=10.0.0.44 \ 3 --netmask=255.255.255.0 --gateway=10.0.0.2 \ 4 --force --dir=/work/vserver --memory=256Mb \ 5 --arch=amd64 \ 6 --kernel=/boot/vmlinuz-2.6.18-5-xen-amd64 \ 7 --debootstrap --dist=etch \ 8 --mirror= http://ftp.fi.debian.org/debian/\ 9 --passwd </pre> Then ssh into your new Xen as root. <pre class="active4d"> 1 ssh -l root mirrors.aktagon.com </pre> Make base configurations for a fresh Xen. <pre class="active4d"> 1 apt-get update && apt-get install locales console-data && dpkg-reconfigure locales </pre> Then get the mirror synchronization script from Debian. <pre class="active4d"> 1 wget "http://www.debian.org/mirror/anonftpsync" 2 chmod a+x anonftpsync </pre> Then install dependencies for anonftpsync script. Otherwise the script will fail with a -bash: lockfile: command not found error. <pre class="active4d"> 1 apt-get install procmail </pre> Install nginx. <pre class="active4d"> 1 apt-get install nginx </pre> Configure anonftpsync with your favorite editor and change the lines below. These settings will setup a mirror only for amd64 files. You could remove i386 from the excluded architectures, but then a 50Gb image won’t fit all the files. <pre class="active4d"> 1 TO=/var/www/debian 2 RSYNC_HOST=ftp.fi.debian.org 3 RSYNC_DIR=debian 4 LOGDIR=/var/log/mirroring 5 ARCH_EXCLUDE="alpha arm hppa hurd-i386 i386 ia64 m68k mipsel mips powerpc s390 sh sparc source" </pre> Make the log directory. <pre class="active4d"> 1 mkdir -p /var/log/mirroring </pre> Configure nginx by modifying /etc/nginx/nginx.conf with your favorite editor. Just add the autoindex line into server { location / { context <pre class="active4d"> 1 # abbreviated start of file for clarity... 2 server { 3 listen 80; 4 server_name localhost; 5 6 access_log /var/log/nginx/localhost.access.log; 7 8 location / { 9 root /var/www; 10 # add the line below to allow directory listing 11 autoindex on; 12 index index.html index.htm; 13 } 14 # abbreviated end of file for clarity... </pre> Do the synchronizing. And wait… for a long while. On a 8/1Mbit cable the first synchronize took roughly 20 hours. <pre class="active4d"> 1 ./anonftpsync </pre> Now modify your /etc/apt/sources.list on existing Xen images to use your local mirror. And remember to create new Xen images using your new mirror :) In the above case the URL is http://mirrors.aktagon.com/debian NB: there is no public mirrors.aktagon.com available… sorry.

2008-07-08T22:02:19+03:00