Code tagged with 'etch'

How to install and use the Sphinx search engine and acts_as_sphinx plugin on Debian Etch

Shell Script (Bash) posted 4 months ago by christian

Inspiration for this snippet was taken from this post on the Sphinx forum, plus this blog post.

Compiling Sphinx

First install the prerequisites:

   1  sudo aptitude install libmysql++-dev libmysqlclient15-dev checkinstall

Next download sphinx, libstemmer and install everything and the fish:

   1  cd /usr/local/src
   2  
   3  wget http://sphinxsearch.com/downloads/sphinx-0.9.8-rc2.tar.gz
   4  tar zxvf sphinx-0.9.8-rc2.tar.gz 
   5  
   6  cd sphinx-0.9.8-rc2/
   7  
   8  # Add stemming support for Swedish, Finnish and other fun languages.
   9  wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
  10  tar zxvf libstemmer_c.tgz
  11  
  12  ./configure --with-libstemmer
  13  make
  14  
  15  make install

Configure Sphinx

Create a sphinx.conf file in your Rails config directory, as described here, or use this template.

Install acts_as_sphinx plugin

   1  ./script/plugin install http://svn.datanoise.com/acts_as_sphinx

Add acts_as_sphinx to your model:

   1  class Documents
   2     acts_as_sphinx
   3  end

Indexing content

   1  rake sphinx:index
   2  
   3  (in /var/www/xxx.com/releases/20080429144230)
   4  Sphinx 0.9.8-rc2 (r1234)
   5  Copyright (c) 2001-2008, Andrew Aksyonoff
   6  
   7  using config file './sphinx.conf'...
   8  indexing index 'xxx.com'...
   9  collected 5077 docs, 0.6 MB
  10  sorted 0.1 Mhits, 100.0% done
  11  total 5077 docs, 632096 bytes
  12  total 0.160 sec, 3950427.25 bytes/sec, 31729.86 docs/sec

Reindexing content

sphinx:index shouldn’t be run while the searchd process is running, so use rake sphinx:rotate instead, which restarts the searchd process after indexing.

Starting the daemon

   1  mkdir -m 664 /var/log/sphinx
   2  rake sphinx:start
   3  
   4  (in /var/www/xxx.com/releases/20080429144230)
   5  Sphinx 0.9.8-rc2 (r1234)
   6  Copyright (c) 2001-2008, Andrew Aksyonoff
   7  
   8  using config file './sphinx.conf'...
   9  Sphinx searchd server started.

Searching

   1  Documents.find_with_sphinx 'why did I write this'

Tagged sphinx, search, acts_as_sphinx, debian, etch, rails, install, libstemmer

Creating a local Debian mirror for your Xen servers

Shell Script (Bash) posted about 1 year ago by marko

Once you’ve bought a dual or quad Xeon and started to experiment with virtualization you will soon want to create your local mirror to make installs lightning fast. This is a step-by-step how i did it.

First create the Xen that will be our mirror server. The size requirements can be found here: Debian mirror sizes The combined size of amd64 architecture and architecture independent files was 39Gb on 1.9.2007. So I made the image 50Gb big. Remember to change this mirror to a location near you.

   1  xen-create-image --hostname=mirrors.aktagon.com \
   2  --size=50Gb --swap=256Mb --ip=10.0.0.44 \
   3  --netmask=255.255.255.0 --gateway=10.0.0.2 \
   4  --force --dir=/work/vserver --memory=256Mb \
   5  --arch=amd64 \
   6  --kernel=/boot/vmlinuz-2.6.18-5-xen-amd64 \
   7  --debootstrap --dist=etch \
   8  --mirror= http://ftp.fi.debian.org/debian/\
   9  --passwd

Then ssh into your new Xen as root.

   1  ssh -l root mirrors.aktagon.com

Make base configurations for a fresh Xen.

   1  apt-get update && apt-get install locales console-data && dpkg-reconfigure locales

Then get the mirror synchronization script from Debian.

   1  wget "http://www.debian.org/mirror/anonftpsync"
   2  chmod a+x anonftpsync

Then install dependencies for anonftpsync script. Otherwise the script will fail with a -bash: lockfile: command not found error.

   1  apt-get install procmail

Install nginx.

   1  apt-get install nginx

Configure anonftpsync with your favorite editor and change the lines below. These settings will setup a mirror only for amd64 files. You could remove i386 from the excluded architectures, but then a 50Gb image won’t fit all the files.

   1  TO=/var/www/debian
   2  RSYNC_HOST=ftp.fi.debian.org
   3  RSYNC_DIR=debian
   4  LOGDIR=/var/log/mirroring
   5  ARCH_EXCLUDE="alpha arm hppa hurd-i386 i386 ia64 m68k mipsel mips powerpc s390 sh sparc source"

Make the log directory.

   1  mkdir -p /var/log/mirroring

Configure nginx by modifying /etc/nginx/nginx.conf with your favorite editor. Just add the autoindex line into server { location / { context

   1  # abbreviated start of file for clarity...
   2      server {
   3          listen       80;
   4          server_name  localhost;
   5  
   6          access_log  /var/log/nginx/localhost.access.log;
   7  
   8          location / {
   9              root   /var/www;
  10              # add the line below to allow directory listing
  11              autoindex  on;
  12              index  index.html index.htm;
  13          }
  14     # abbreviated end of file for clarity...

Do the synchronizing. And wait… for a long while. On a 8/1Mbit cable the first synchronize took roughly 20 hours.

   1  ./anonftpsync

Now modify your /etc/apt/sources.list on existing Xen images to use your local mirror. And remember to create new Xen images using your new mirror :) In the above case the URL is http://mirrors.aktagon.com/debian

NB: there is no public mirrors.aktagon.com available… sorry.

Tagged xen, debian, etch, local mirror, amd64, anonftpsync, nginx

Quick step by step on creating a xen environment in Debian Etch