At Aktagon we use MephistoBlog as CMS , and I couldn’t find any information on how to SEO optimize MephistoBlog on Google, so I’m sharing my notes here.

This tip shows you how to make your pages more search engine friendly.

First, add the title tag, plus the meta description and keywords tags to your layout’s Liquid template , as shown here:

   1  <meta name="description" content="{% if article %} {{ article.excerpt }}  {% else %} YOUR DEFAULT SITE DESCRIPTION {% endif %}" />
   2  	<meta name="keywords" content="{% if article %} {% for tag in article.tags %}{{ tag }}, {% endfor %} {% endif %} YOUR DEFAULT KEYWORDS" />
   3  	<title>{% if article %} {{ article.title }} &raquo; {{ site.title }} {% else %} {{ site.title }} &raquo; {{ site.subtitle }} {% endif %}</title>

Remember to update the default description and keywords in the meta tags’ body.

Now, whenever you publish an article, simply add an excerpt and some tags to it. The excerpt is used as the meta description and the article’s tags as the meta keywords, both make Google a bit happier, but the description is by far the more important.

• Google – Googlebot/2.1 ( http://www.googlebot.com/bot.html)
• Google Image – Googlebot-Image/1.0 ( http://www.googlebot.com/bot.html)
• MSN Live – msnbot-Products/1.0 (+http://search.msn.com/msnbot.htm)
• Yahoo – Mozilla/5.0 (compatible; Yahoo! Slurp;)

The code (via):

   1  user_agent = request.user_agent.downcase
   2  @bot = [ 'msnbot', 'yahoo! slurp','googlebot' ].detect { |bot| user_agent.include? bot }

When the Google bot visists your site the @bot string will contain ‘googlebot’.

If you need to detect more bots than these, then the user-agents.org site contains a list of various user agents for both bots and browsers.

First read this... then this

   1  draft

Inspiration for this snippet was taken from this post on the Sphinx forum, plus this blog post.

Compiling Sphinx

First install the prerequisites:

   1  sudo aptitude install libmysql++-dev libmysqlclient15-dev checkinstall

Next download sphinx, libstemmer and install everything and the fish:

   1  cd /usr/local/src
   2  
   3  wget http://sphinxsearch.com/downloads/sphinx-0.9.8-rc2.tar.gz
   4  tar zxvf sphinx-0.9.8-rc2.tar.gz 
   5  
   6  cd sphinx-0.9.8-rc2/
   7  
   8  # Add stemming support for Swedish, Finnish and other fun languages.
   9  wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
  10  tar zxvf libstemmer_c.tgz
  11  
  12  ./configure --with-libstemmer
  13  make
  14  
  15  make install

Configure Sphinx

Create a sphinx.conf file in your Rails config directory, as described here, or use this template.

Install acts_as_sphinx plugin

   1  ./script/plugin install http://svn.datanoise.com/acts_as_sphinx

Add acts_as_sphinx to your model:

   1  class Documents
   2     acts_as_sphinx
   3  end

Indexing content

   1  rake sphinx:index
   2  
   3  (in /var/www/xxx.com/releases/20080429144230)
   4  Sphinx 0.9.8-rc2 (r1234)
   5  Copyright (c) 2001-2008, Andrew Aksyonoff
   6  
   7  using config file './sphinx.conf'...
   8  indexing index 'xxx.com'...
   9  collected 5077 docs, 0.6 MB
  10  sorted 0.1 Mhits, 100.0% done
  11  total 5077 docs, 632096 bytes
  12  total 0.160 sec, 3950427.25 bytes/sec, 31729.86 docs/sec

Reindexing content

sphinx:index shouldn’t be run while the searchd process is running, so use rake sphinx:rotate instead, which restarts the searchd process after indexing.

Starting the daemon

   1  mkdir -m 664 /var/log/sphinx
   2  rake sphinx:start
   3  
   4  (in /var/www/xxx.com/releases/20080429144230)
   5  Sphinx 0.9.8-rc2 (r1234)
   6  Copyright (c) 2001-2008, Andrew Aksyonoff
   7  
   8  using config file './sphinx.conf'...
   9  Sphinx searchd server started.

Searching

   1  Documents.find_with_sphinx 'why did I write this'

This is a slightly modified version of some Japanese fellow’s documention on how to install Hyper Estraier on Mac OS X

First we need libiconv:

   1  $ cd /usr/local/src
   2  $ wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz
   3  $ tar zxvf libiconv-1.11.tar.gz
   4  $ cd libiconv-1.11
   5  $ ./configure
   6  $ make
   7  $ sudo make install

zlib:

   1  $ cd /usr/local/src
   2  $ wget http://www.zlib.net/zlib-1.2.3.tar.gz
   3  $ tar zxvf zlib-1.2.3.tar.gz
   4  $ ./configure
   5  $ make
   6  $ sudo make install

QDBM :

   1  $ cd /usr/local/src
   2  $ wget http://qdbm.sourceforge.net/qdbm-1.8.74.tar.gz
   3  $ tar zxvf qdbm-1.8.74.tar.gz
   4  $ cd qdbm-1.8.74
   5  $ ./configure --enable-zlib
   6  $ make mac
   7  $ make check-mac
   8  $ sudo make install-mac

Hyper Estraier

   1  $ cd /usr/local/src
   2  $ wget http://hyperestraier.sourceforge.net/hyperestraier-1.4.9.tar.gz
   3  $ tar zxvf hyperestraier-1.4.9.tar.gz
   4  $ cd hyperestraier-1.4.9
   5  $ ./configure
   6  $ make mac
   7  $ make check-mac
   8  $ sudo make install-mac

Finally we’ll install the pure ruby bindings:

   1  $ cd rubypure
   2  $ ./configure
   3  $ make
   4  $ sudo make install

To verify that Hyper Estraier is installed and working, try one of the examples in the examples folder, or follow these instructions:

First create and start a P2P node:

   1  estmaster init casket
   2  estmaster start casket

Open http://localhost:1978/master_ui in your browser and create a node called dictionary.

Then run this code which adds a record to the index:

   1  require "estraierpure"
   2  include EstraierPure
   3  
   4  node = Node::new
   5  node.set_url("http://localhost:1978/node/dictionary")
   6  node.set_auth("admin", "admin")
   7  
   8  doc = Document::new
   9  # @uri : the location of a document which any document should have.
  10  doc.add_attr("@uri", "This is the URL, required?")
  11  # @title : the title used as a headline in the search result.
  12  doc.add_attr("@title", "This is the title, required?")
  13  doc.add_text("Text goes here")
  14  
  15  result = node.put_doc(doc)
  16  unless result
  17    printf("error: %s\n", node.status)
  18  end

Next we’ll perform a query which returns the object we just added:

   1  require "estraierpure"
   2  include EstraierPure
   3  
   4  # create and configure the node connecton object
   5  node = Node::new
   6  node.set_url("http://localhost:1978/node/dictionary")
   7  
   8  # create a search condition object
   9  cond = Condition::new
  10  
  11  # set the search phrase to the search condition object
  12  cond.set_phrase("Text goes here")
  13  
  14  # get the result of search
  15  nres = node.search(cond, 0);
  16  if nres
  17    # for each document in the result
  18    for i in 0...nres.doc_num
  19      # get a result document object
  20      rdoc = nres.get_doc(i)
  21      # display attributes
  22      value = rdoc.attr("@uri")
  23      printf("URI: %s\n", value) if value
  24      value = rdoc.attr("@title")
  25      printf("Title: %s\n", value) if value
  26      # display the snippet text */
  27      printf("%s", rdoc.snippet)
  28    end
  29  else
  30    STDERR.printf("error: %d\n", node.status)
  31  end

The query language is documented here.

If you’re indexing ActiveRecord objects use acts_as_searchable:

   1  gem install acts_as_searchable