bash snippets

Bulk renaming of files

Tagged rename, regexp, bulk, filename, bash, perl, linux  Languages bash

Rename the files in a directory by replacing a space with an underscore. The rename program comes with most modern Linux distros.

rename 's/\ /_/g' *.*

Colorize grep

Tagged grep, bash, color, linux  Languages bash

Put this export in .bashrc to make grep colorize the text it found.

export GREP_OPTIONS='--color=auto'

Automatically giving predetermined answers to a program.

Tagged script automation, bash, linux  Languages bash

Sometimes you want to automate the running of a script or program, but the darn thing keeps asking questions and expects interactive answers. Shell scripting makes it easy to give the answers automatically, using a stream redirection from a file.

./qurious_program < predetermined_answers.txt

Appending after a pattern from a file in sed

Tagged sed, linux, bash  Languages bash

Useful when you want to append the contents of a whole file into something you are sed'ing. Be careful if you use -i, it will replace the working file.

sed -i "/attr_accessor :config/r ../patch_for_application.rb.tmpl" app/controllers/application.rb

Check if a file or directory exists with bash

Tagged nginx, daemon, bash, linux, debian  Languages bash

This script tests if nginx exists and is executable. The script prints a warning and exits, if nginx doesn't exists or isn't executable:

DAEMON=/usr/local/sbin/nbinx
if [ ! -x $DAEMON ]
then
   echo "Couldn't find $DAEMON. Please set path to DAEMON."
   exit 0
fi

See man test for more information on how to use the test command.

Converting WMA files to MP3 on Linux

Tagged wma, mp3, convert script, bash  Languages bash

Use this to convert WMA files to MP3 in Linux.

#!/bin/bash
# By Marko Haapala
# converts wma to mp3 recursively. does not delete any static files, so 
# cleanup and renaming is needed afterwards. 
#
# requirements:
# lame    - http://lame.sourceforge.net/download.php
# mplayer - apt-get install mplayer or http://www.mplayerhq.hu/design7/dload.html


current_directory=$(pwd)
wma_files=$(find "${current_directory}" -type f -iname "*.wma")
# Need to change IFS or files with filenames containing spaces will not
# be handled correctly by for loop
IFS=$'\n' 
for wma_file in ${wma_files}; do 
    mplayer -vo null -vc dummy -af resample=44100 \
    -ao pcm -ao pcm:waveheader "${wma_file}" && lame -m s \
    audiodump.wav -o "${wma_file}".mp3
    rm audiodump.wav
done

Remove ID3 tags from MP3 files recursively.

Tagged id3ed, id3 tag, bash  Languages bash

Remove id3 tags from MP3 files recursively.

#!/bin/bash
# By Marko Haapala
# requirements:
# id3ed - apt-get install id3ed
IFS=$'\n'
for mp3_file in $(find ./ -type f -iname '*.mp3'); do
    id3ed -r "${mp3_file}"
done

Using expect for automation of bulk scp copying.

Tagged expect, bash, automation, scp, bulk  Languages bash

Expect can come in handy when you can't configure ssh public key authentication on the servers :) (and the system "architect" hasn't yet realized the wonderfulness of a log host).

#!/bin/bash
#
# Usage: script <username> <password> <build>
#
# Example ./copy_logs_from_production.sh marko hubbabubba current
#

username=$1
password=$2
build=$3
mkdir $build

instance_1_server=10.0.0.1
instance_2_server=10.0.0.1
instance_3_server=10.0.0.2
instance_4_server=10.0.0.2
instance_5_server=10.0.0.3
instance_6_server=10.0.0.3
instance_7_server=10.0.0.4
instance_8_server=10.0.0.4
instance_9_server=10.0.0.5

servers=("$instance_1_server" "$instance_2_server" "$instance_3_server" "$instance_4_server" "$instance_5_server" "$instance_6_server" "$instance_7_server" "$instance_8_server" "$instance_9_server" )
i=1
for server in ${servers[@]}; do
  expect -c "
            # exp_internal 1 # uncomment for debugging
            spawn /usr/bin/scp $username@$server:/var/logs/application/$build/server${i}/error.log $build/error-${i}.log
            expect { 
              "*password:*" { send $password\r\n; interact } 
              eof { exit }
            }
            exit
            "
  let "i=i+1"
done