install snippets

Installing Ultraviolet and Oniguruma

Tagged onigurama, ultraviolet, ruby, ubuntu, install, oniguruma  Languages bash

First install Oniguruma

Oniguruma is a regular expression engine that Ultraviolet uses to parse text; Ruby also uses Oniguruma by the way. If you don't have Oniguruma on your system you'll get this error while installing Ultraviolet (at least on Ubuntu Linux):

oregexp.c:2:23: error: oniguruma.h: No such file or directory

This tells you that you should download and install Oniguruma. For me version 5.8.0 was the only version that worked, so execute this command to get the right version of Oniguruma:

$ wget http://www.geocities.jp/kosako3/oniguruma/archive/onig-5.8.0.tar.gz

You now have the source package on your computer, so decompress it with the following command:

$ tar zxvf onig-5.8.0.tar.gz

If everything went fine, change current directory:

$ cd onig-5.8.0/

Next, run configure:

$ ./configure

Watch the output closely and fix any errors reported, then run make:

$ make

To build and install Onigurama run:

$ sudo make install

I managed to get the following errors from Ultraviolet with other versions of Oniguruma, but these went away after installing 5.8.0 and re-installing Oniguruma:

Parsing error in // ==UserScript==: wrong number of arguments (2 for 0) #<Textpow::SyntaxNode:0xb7c91780>

Installing Ultraviolet and dependencies

Next install Ultraviolet with RubyGems:

$ sudo gem install -r ultraviolet --include-dependencies

Select which gem to install for your platform (i486-linux)
 1. oniguruma 1.1.0 (mswin32)
 2. oniguruma 1.1.0 (ruby)
 3. Skip this gem
 4. Cancel installation
> 2
Building native extensions.  This could take a while...
Successfully installed ultraviolet-0.10.0
Successfully installed textpow-0.9.0
Successfully installed oniguruma-1.1.0
Successfully installed plist-3.0.0

Test that Ultraviolet works by running the following code with irb:

$ irb

require 'rubygems'
require 'uv'
puts Uv.syntaxes.join( "\n" )
puts Uv.themes.join( "\n" )
input = <<HTML
<html>
  <body>
  </body>
</html>
HTML

puts Uv.parse( input, "xhtml", "html", true, "slush_poppies")

Problems

You might get this error:

require 'uv'
LoadError: libonig.so.2: cannot open shared object file: No such file or directory - /usr/lib/ruby/gems/1.8/gems/oniguruma-1.1.0/lib/oregexp.so

This message is a bit confusing. It means Ruby can't find libonig.so.2, not oregexp.so as you could believe.

To fix this, check if the library has been linked:

$ ldconfig -p|grep libonig

If the library is not linked, add the path to the directory where the file is located to /etc/ld.so.conf:

/usr/local/lib

include /etc/ld.so.conf.d/*.conf

Then run:

$ ldconfig

Another way of fixing this problem would be to tell the build script to install it to /usr/lib.

How to install the exception_logger Rails plugin and protect the logs with basic authentication

Tagged ruby, exception_logger, install, routes, rails, basic, authentication  Languages ruby

This snippet explains how to install and use the Rails exception_logger plugin. I'll also show you how to protect your logs by extending the plugin with basic authentication.

script/plugin source http://svn.techno-weenie.net/projects/plugins
script/plugin install exception_logger

I'm using Rails Edge on this project, so I had to install classic pagination also:

script/plugin install svn://errtheblog.com/svn/plugins/classic_pagination

Next create and execute the migration file:

./script/generate exception_migration
rake db:migrate

Before starting the server we need to setup the routes:

map.exceptions '/logged_exceptions/:action/:id', :controller => 'logged_exceptions', :action => 'index', :id => nil

You also need to include the ExceptionLoggable in your ApplicationController:

class ApplicationController < ActionController::Base
  include ExceptionLoggable
...

Start your server and access the exception log at /logged_exceptions.

Exceptions can contain email addresses, passwords, credit card numbers, so you'll want to protect /logged_exceptions from the public. This can be done by adding the following code to the end of environment.rb:

config.after_initialize do
  require 'application' unless Object.const_defined?(:ApplicationController)
  LoggedExceptionsController.class_eval do
    before_filter :authenticate

    protected

    def authenticate
      authenticate_or_request_with_http_basic do |username, password|
        username == "foo" && password == "bar"
      end
    end
  end
end

With this code we add a before filter that shows a login dialog to anyone trying to access /logged_exception/. Note that this requires Rails 2.0 basic authentication to work, so make sure you have the proper version installed.

How to install Hyper Estraier and the Ruby bindings on Mac OS X, including a mini example on how to use the P2P capabilities

Tagged hyper estraier, search, ruby, install  Languages bash

This is a slightly modified version of some Japanese fellow's documention on how to install Hyper Estraier on Mac OS X

First we need libiconv:

$ cd /usr/local/src
$ wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.11.tar.gz
$ tar zxvf libiconv-1.11.tar.gz
$ cd libiconv-1.11
$ ./configure
$ make
$ sudo make install

zlib:

$ cd /usr/local/src
$ wget http://www.zlib.net/zlib-1.2.3.tar.gz
$ tar zxvf zlib-1.2.3.tar.gz
$ ./configure
$ make
$ sudo make install

QDBM:

$ cd /usr/local/src
$ wget http://qdbm.sourceforge.net/qdbm-1.8.74.tar.gz
$ tar zxvf qdbm-1.8.74.tar.gz
$ cd qdbm-1.8.74
$ ./configure --enable-zlib
$ make mac
$ make check-mac
$ sudo make install-mac

Hyper Estraier

$ cd /usr/local/src
$ wget http://hyperestraier.sourceforge.net/hyperestraier-1.4.9.tar.gz
$ tar zxvf hyperestraier-1.4.9.tar.gz
$ cd hyperestraier-1.4.9
$ ./configure
$ make mac
$ make check-mac
$ sudo make install-mac

Finally we'll install the pure ruby bindings:

$ cd rubypure
$ ./configure
$ make
$ sudo make install

To verify that Hyper Estraier is installed and working, try one of the examples in the examples folder, or follow these instructions:

First create and start a P2P node:

estmaster init casket
estmaster start casket

Open http://localhost:1978/master\_ui in your browser and create a node called dictionary.

Then run this code which adds a record to the index:

require "estraierpure"
include EstraierPure

node = Node::new
node.set_url("http://localhost:1978/node/dictionary")
node.set_auth("admin", "admin")

doc = Document::new
# @uri : the location of a document which any document should have.
doc.add_attr("@uri", "This is the URL, required?")
# @title : the title used as a headline in the search result.
doc.add_attr("@title", "This is the title, required?")
doc.add_text("Text goes here")

result = node.put_doc(doc)
unless result
  printf("error: %s\n", node.status)
end

Next we'll perform a query which returns the object we just added:

require "estraierpure"
include EstraierPure

# create and configure the node connecton object
node = Node::new
node.set_url("http://localhost:1978/node/dictionary")

# create a search condition object
cond = Condition::new

# set the search phrase to the search condition object
cond.set_phrase("Text goes here")

# get the result of search
nres = node.search(cond, 0);
if nres
  # for each document in the result
  for i in 0...nres.doc_num
    # get a result document object
    rdoc = nres.get_doc(i)
    # display attributes
    value = rdoc.attr("@uri")
    printf("URI: %s\n", value) if value
    value = rdoc.attr("@title")
    printf("Title: %s\n", value) if value
    # display the snippet text */
    printf("%s", rdoc.snippet)
  end
else
  STDERR.printf("error: %d\n", node.status)
end

The query language is documented here.

If you're indexing ActiveRecord objects use acts_as_searchable:

gem install acts_as_searchable

Installing nginx on Debian

Tagged nginx, install, debian  Languages bash

DRAFT...

Find latest version of nginx

http://sysoev.ru/en/ http://wiki.codemongers.com/NginxInstallOptions

Install a compiler otherwise: ./configure: error: C compiler gcc is not found

The command:

sudo apt-get install build-essential

Install pre-requisites otherwise you'll get:

Configuration summary + threads are not used + PCRE library is not found + OpenSSL library is not found + md5 library is not used + sha1 library is not used + zlib library is not found

The command:

sudo apt-get install libpcre3 libpcre3-dev libpcrecpp0 libssl-dev zlib1g-dev

Much better:

Configuration summary + threads are not used + using system PCRE library + using system OpenSSL library + md5 library is not used + sha1 library is not used + using system zlib library

Compile and install nginx

$ ./configure \
        --sbin-path=/usr/local/sbin \
        --conf-path=/etc/nginx/nginx.conf \
        --pid-path=/var/run/nginx.pid \
        --error-log-path=/var/log/nginx/error.log \
        --http-log-path=/var/log/nginx/access.log \
        --with-http_ssl_module \
        --http-client-body-temp-path=/tmp/nginx_client \
        --http-proxy-temp-path=/tmp/nginx_proxy \
        --http-fastcgi-temp-path=/tmp/nginx_fastcgi
$ make
$ sudo make install

Run the install script

cd /usr/local/src

wget http://sysoev.ru/nginx/nginx-0.5.35.tar.gz

tar zxvf nginx-0.5.35

cd nginx-0.5.35

Create an nginx user and group

$ useradd -g www-data -d /var/www nginx

Create the web server directory

mkdir /var/www
chown root.www-data /var/www
chmod ug=rwx,o= /var/www

Test configuration

nginx -t
2008/03/09 20:51:05 [info] 5034#0: the configuration file /etc/nginx/nginx.conf syntax is ok
2008/03/09 20:51:05 [info] 5034#0: the configuration file /etc/nginx/nginx.conf was tested successfully

Start nginx

nginx

Installing Rails, mongrel and mongrel_cluster on Debian

Tagged rails, ruby, debian, install, sqlite3, mongrel, mongrel_cluster  Languages bash

DRAFT...

Install RubyGems

http://rubyforge.org/frs/download.php/29548/rubygems-1.0.1.tgz

tar zxvf rubygems-1.0.1.tgz

cd rubygems-1.0.1

ruby setup.rb

Install Rails

gem install rails

Install sqlite3 (optional)

apt-get install sqlite3 libsqlite3-dev
gem install sqlite3-ruby

Install mongrel and mongrel_cluster

$ gem install mongrel mongrel_cluster

$ mongrel_rails cluster::configure -e production \
  -p 8000 \
  -a 127.0.0.1 \
  -N 3 \
  -c /var/www/xyz/current


$ mongrel_rails cluster::start


$ useradd -g www-data -d /var/www mongrel

Surviving reboots

sudo mkdir /etc/mongrel_cluster

sudo ln -s /var/www/xyz/config/mongrel_cluster.yml /etc/mongrel_cluster/xyz.yml

sudo cp /usr/local/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/

sudo chmod +x /etc/init.d/mongrel_cluster

sudo /usr/sbin/update-rc.d -f mongrel_cluster defaults

mongrel_cluster_ctl status

Stale pids

If your mongrels crash or if you kill them, mongrel_cluster won't start your mongrels because mongrel_cluster believes the processes are still running, instead mongrel_cluster complains and does nothing:

** !!! PID file tmp/pids/mongrel.8000.pid already exists.  Mongrel could be running already.  Check your log/mongrel.8000.log for errors.
** !!! Exiting with error.  You must stop mongrel and clear the .pid before I'll attempt a start.

To fix this simply add the --clean switch to the /usr/local/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster startup script:

mongrel_cluster_ctl start -c $CONF_DIR --clean

How to install the stemmer4r gem on Mac OS X and Linux

Tagged stemming, stemmer4r, install, osx, linux, gem  Languages ruby

The stemmer4r gem is fubar. Warning draft snippet...

# gem install stemmer4r
Bulk updating Gem source index for: http://gems.rubyforge.org
Building native extensions.  This could take a while...
ERROR:  While executing gem ... (Gem::Installer::ExtensionBuildError)
    ERROR: Failed to build gem native extension.

ruby extconf.rb install stemmer4r

Gem files will remain installed in /usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6 for inspection.
Results logged to /usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6/ext/stemmer4r/gem_make.out


1. Change path of Ruby executable

cd /usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6/ext/stemmer4r/
vim extconf.rb

#!/usr/bin/ruby -w

to

#ruby -w

2. Compile libstemmer_c

cd /usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6/ext/stemmer4r/libstemmer/
make

3. Compile stemmer4r

cd /usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6/ext/stemmer4r/

Change path:
/usr/local/ruby/lib/ruby/1.8/i686-linux/
To:
/usr/lib/ruby/1.8/x86_64-linux/

Or wherever you have it installed

ruby extconf.rb


4. Build stemmer4r gem


gem build stemmer4r.gemspec

gem install stemmer4r-0.6.gem


Problems

gcc -shared -rdynamic -Wl,-export-dynamic   -L"/usr/lib" -o stemmer4r.so stemmer4r.o libstemmer_c/libstemmer.o  -lruby1.8  -lpthread -ldl -lcrypt -lm   -lc
/usr/bin/ld: libstemmer_c/libstemmer.o(libstemmer.o): relocation R_X86_64_32 against a local symbol' can not be used when making a shared object; recompile with -fPIC
libstemmer_c/libstemmer.o: could not read symbols: Bad value
collect2: ld returned 1 exit status
make: *** [stemmer4r.so] Error 1


Add CFLAGS:

root@aktagon:/usr/lib/ruby/gems/1.8/gems/stemmer4r-0.6/ext/stemmer4r/libstemmer_c# make
include mkinc.mak
CFLAGS   =  -fPIC
libstemmer.o: $(snowball_sources:.c=.o)
        $(AR) -cru $@ $^

How to install and use the Sphinx search engine and acts_as_sphinx plugin on Debian Etch

Tagged sphinx, search, acts_as_sphinx, debian, etch, rails, install, libstemmer  Languages bash

Inspiration for this snippet was taken from this post on the Sphinx forum, plus this blog post.

Compiling Sphinx

First install the prerequisites:

sudo aptitude install libmysql++-dev libmysqlclient15-dev checkinstall

Next download sphinx, libstemmer and install everything and the fish:

cd /usr/local/src

wget http://sphinxsearch.com/downloads/sphinx-0.9.9.tar.gz
tar zxvf sphinx-0.9.9.tar.gz 

cd sphinx-0.9.9/

# Add stemming support for Swedish, Finnish and other fun languages.
wget http://snowball.tartarus.org/dist/libstemmer_c.tgz
tar zxvf libstemmer_c.tgz

./configure --with-libstemmer
make

make install

Configure Sphinx

Create a sphinx.conf file in your Rails config directory, as described here, or use this template.

Install acts_as_sphinx plugin

./script/plugin install http://svn.datanoise.com/acts_as_sphinx

Add acts_as_sphinx to your model:

class Documents
   acts_as_sphinx
end

Indexing content

rake sphinx:index

(in /var/www/xxx.com/releases/20080429144230)
Sphinx 0.9.8-rc2 (r1234)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file './sphinx.conf'...
indexing index 'xxx.com'...
collected 5077 docs, 0.6 MB
sorted 0.1 Mhits, 100.0% done
total 5077 docs, 632096 bytes
total 0.160 sec, 3950427.25 bytes/sec, 31729.86 docs/sec

Reindexing content

sphinx:index shouldn't be run while the searchd process is running, so use rake sphinx:rotate instead, which restarts the searchd process after indexing.

Starting the daemon

mkdir -m 664 /var/log/sphinx
rake sphinx:start

(in /var/www/xxx.com/releases/20080429144230)
Sphinx 0.9.8-rc2 (r1234)
Copyright (c) 2001-2008, Andrew Aksyonoff

using config file './sphinx.conf'...
Sphinx searchd server started.

Searching

Documents.find_with_sphinx 'why did I write this'

How to install and use the mysql-python library

Tagged python, mysql, mysql-python, install  Languages python

First download mysql-python from http://sourceforge.net/projects/mysql-python.

Extract it and run:

python setup.py build
sudo python setup.py install

If you get this error you need to install python-dev package:

In file included from _mysql.c:29:
pymemcompat.h:10:20: error: Python.h: No such file or directory
_mysql.c:30:26: error: structmember.h: No such file or directory
In file included from /usr/include/mysql/mysql.h:44,
                 from _mysql.c:40:
.
.
.
_mysql.c:2808: warning: return type defaults to 'int'
_mysql.c: In function 'DL_EXPORT':
_mysql.c:2808: error: expected declaration specifiers before 'init_mysql'
_mysql.c:2886: error: expected '{' at end of input
error: command 'gcc' failed with exit status 1

Installing the python-dev package on Debian is done with apt-get or synaptic:

apt-get install python-dev

Installing the library should now work:

python setup.py build
python setup.py install

Next test the library in the python console:

import MySQLdb

# Note that this example uses UTF-8 encoding
conn = MySQLdb.connect(host='localhost', user='...', passwd='...', db='...', charset = "utf8", use_unicode = True)
cursor = conn.cursor()


cursor.execute ("SELECT * FROM cities")
rows = cursor.fetchall ()

for row in rows:
  print "%s, %s" % (row[0], row[1].encode('utf-8'))

print "Number of rows returned: %d" % cursor.rowcount

Don't forget to close the cursor and connection, and if you're inserting data commit before closing, because autocommit is disabled by default:

cursor.close ()
conn.commit ()
conn.close ()

For more information about MySQLdb see this article.

How to fix "fatal: 'gitosis-admin.git': unable to chdir or not a git archive"

Tagged git, fatal, gitosis, install, setup  Languages bash

Are the SSH keys valid?

When, not if, you get this error you most probably have the wrong keys listed in /home/git/.ssh/authorized_keys:

$ git  clone git@xxx.com:gitosis-admin.git 
Initialized empty Git repository in /Users/christian/Documents/Projects/gitosis-admin/.git/
fatal: 'gitosis-admin.git': unable to chdir or not a git archive
fatal: The remote end hung up unexpectedly

Try removing the keys in authorized_keys one by one, and verify that the names of the users match the ones in .gitosis.conf

http://scie.nti.st/2007/11/14/hosting-git-repositories-the-easy-and-secure-way

Try using the full path to the repository

$ git clone git@xxx.com::/home/git/repositories/xxx.git

Check if the repository exists

$ cd /home/git/repositories/xxx.git
$ git clone /home/git/repositories/xxx.git

What does SSH say?

ssh -v git@xxx.com

Use Google

http://stackoverflow.com/questions/1264262/connecting-to-gitosis-server-through-an-ssh-tunnel http://stackoverflow.com/questions/733057/can-git-work-via-ssh-port-forwarding

How to install memcached on OSX Snow Leopard

Tagged memcached, install, snowleopard  Languages bash

These instructions are from wincent.com:

cd /usr/local/src
curl -O http://www.monkey.org/~provos/libevent-1.4.12-stable.tar.gz
tar xzvf libevent-1.4.12-stable.tar.gz 
cd libevent-1.4.12-stable
./configure
make
make verify
sudo make install


cd /usr/local/src
curl -O http://memcached.googlecode.com/files/memcached-1.4.1.tar.gz
tar xzvf memcached-1.4.1.tar.gz 
 cd memcached-1.4.1
./configure
make
make test
sudo make install

memcached -d -P pidfile -l 127.0.0.1