Shell

BASH Shell: For Loop File Names With Spaces

Posted on November 20, 2009. Filed under: Linux, Mac, Shell | Tags: , , , , , , |

BASH for loop works nicely under UNIX / Linux / Windows and OS X while working on set of files. However, if you try to process a for loop on file name with spaces in them you are going to have some problem. for loop uses $IFS variable to determine what the field separators are. By default $IFS is set to the space character. There are multiple solutions to this problem.

Set $IFS variable

Try it as follows:

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
for f in *
do
  echo "$f"
done
IFS=$SAVEIFS

OR

#!/bin/bash
SAVEIFS=$IFS
IFS=$(echo -en "\n\b")
# set me
FILES=/data/*
for f in $FILES
do
  echo "$f"
done
# restore $IFS
IFS=$SAVEIFS

More examples using $IFS and while loop

Now you know that if the field delimiters are not whitespace, you can set IFS. For example, while loop can be used to get all fields from /etc/passwd file:

....
while IFS=: read userName passWord userID groupID geCos homeDir userShell
do
      echo "$userName -> $homeDir"
done < /etc/passwd

Using old good find command to process file names

To process the output of find with a command, try as follows:

find . -print0 | while read -d $'' file
do
  echo -v "$file"
done

Try to copy files to /tmp with spaces in a filename using find command and shell pipes:

find . -print0 | while read -d $'' file; do cp -v "$file" /tmp; done

Processing filenames using an array

Sometimes you need read a file into an array as one array element per line. Following script will read file names into an array and you can process each file using for loop. This is useful for complex tasks:

#!/bin/bash
DIR="$1"

# failsafe - fall back to current directory
[ "$DIR" == "" ] && DIR="." 

# save and change IFS
OLDIFS=$IFS
IFS=$'\n' 

# read all file name into an array
fileArray=($(find $DIR -type f))

# restore it
IFS=$OLDIFS

# get length of an array
tLen=${#fileArray[@]}

# use for loop read all filenames
for (( i=0; i<${tLen}; i++ ));
do
  echo "${fileArray[$i]}"
done

Playing mp3s with spaces in file names

Place following code in your ~/.bashrc file:

mp3(){
	local o=$IFS
	IFS=$(echo -en "\n\b")
	/usr/bin/beep-media-player "$(cat  $@)" &
	IFS=o
}

Keep list of all mp3s in a text file such as follows (~/eng.mp3.txt):

/nas/english/Adriano Celentano - Susanna.mp3
/nas/english/Nick Cave & Kylie Minogue - Where The Wild Roses Grow.mp3
/nas/english/Roberta Flack - Kiling Me Softly With This Song.mp3
/nas/english/The Beatles - Girl.mp3
/nas/english/John Lennon - Stand By Me.mp3
/nas/english/The Seatbelts, Cowboy Bebop - 01-Tank.mp3

To play just type:
$ mp3 eng.mp3.txt

Another example about IFS:

#!/bin/bash

IFS=$’\n’
set $(cat my.file)

# Now the lines are stored in $1, $2, $3, …

echo $1
echo $2
echo $3
echo $4

Reference:

http://www.cyberciti.biz/tips/handling-filenames-with-spaces-in-bash.html

Read Full Post | Make a Comment ( None so far )

Bash small tricks

Posted on November 2, 2009. Filed under: Linux, Shell |

1. How to get the output of the command? (Not the return/exit status of the command)

ret=$(echo 123)

or

ret=`echo 123`

Read Full Post | Make a Comment ( None so far )

Useful Linux commands: Screen, ttyload, mytop, mtop

Posted on October 23, 2009. Filed under: Linux, MySQL, Shell | Tags: , , , , , |

1. Screen

Screen is a full-screen window manager that multiplexes a physical terminal between several processes (typically interactive shells). The same way tabbed browsing revolutionized the web experience, GNU Screen can do the same for your experience in the command line. Instead of opening up several terminal instances on your desktop or using those ugly GNOME/KDE-based tabs, Screen can do it better and simpler. Not only that, with GNU Screen, you can share sessions with others and detach/attach terminal sessions. It is a great tool for people who have to share working environments between work and home

References:

http://www.cyberciti.biz/tips/how-to-use-screen-command-under-linux.html

http://linux.die.net/man/1/screen

http://en.wikipedia.org/wiki/GNU_Screen

2. ttyload

ttyload is a little *NIX utility I wrote which is meant to give a color-coded graph of load averages over time.

Reference:

http://www.daveltd.com/src/util/ttyload/

3. mtop/mkill

mtop (MySQL top) monitors a MySQL server showing the queries which are taking the most amount of time to complete. Features include ‘zooming’ in on a process to show the complete query, ‘explaining’ the query optimizer information for a query and ‘killing’ queries. In addition, server performance statistics, configuration information, and tuning tips are provided.

mkill (MySQL kill) monitors a MySQL server for long running queries and kills them after a specified time interval. Queries can be selected based on regexes on the user, host, command, database, state and query.

Reference:

http://mtop.sourceforge.net/

http://astguiclient.sourceforge.net/vicidial.html

http://www.voip-info.org/wiki/view/VICIDIAL+Dialer

4. mytop

mytop is a console-based (non-gui) tool for monitoring the threads and overall performance of a MySQL 3.22.x, 3.23.x, and 4.x server. It runs on most Unix systems (including Mac OS X) which have Perl, DBI, and Term::ReadKey installed. And with Term::ANSIColor installed you even get color.

Reference:

http://jeremy.zawodny.com/mysql/mytop/

Read Full Post | Make a Comment ( None so far )

Useful Linux tools: sipsak,Balance,eAccelerator,ploticus

Posted on October 20, 2009. Filed under: Linux, Shell | Tags: , , , , , |

1. sipsak

sipsak is a small command line tool for developers and administrators of Session Initiation Protocol (SIP) applications. It can be used for some simple tests on SIP applications and devices

Reference:

http://sipsak.org/

2.Balance

Balance is our surprisingly successful load balancing solution being a simple but powerful generic tcp proxy with round robin load balancing and failover mechanisms. Its behaviour can be controlled at runtime using a simple command line syntax.

Reference:

http://www.inlab.de/balance.html

3.eAccelerator

eAccelerator is a free open-source PHP accelerator & optimizer. It increases the performance of PHP scripts by caching them in their compiled state, so that the overhead of compiling is almost completely eliminated. It also optimizes scripts to speed up their execution. eAccelerator typically reduces server load and increases the speed of your PHP code by 1-10 times.

Reference:

http://eaccelerator.net/

4.ploticus

A free, GPL, non-interactive software package for producing plots, charts, and graphics from data. It was developed in a Unix/C environment and runs onvarious Unix, Linux, and win32 systemsploticus is good for automated or just-in-time graph generation, handles date and time data nicely, and has basic statistical capabilities. It allows significant user control over colors, styles, options and details.

Reference:

http://ploticus.sourceforge.net/doc/welcome.html

Read Full Post | Make a Comment ( None so far )

Liked it here?
Why not try sites on the blogroll...