Bash Tips and Pitfalls
Tips
Parsing Command-Line Option Parameters
- To ease parsing, pre-parse with executable getopt (see here for more information and examples).
#!/bin/bash
# (old version)
args=`getopt abc: $*`
if test $? != 0
then
echo 'Usage: -a -b -c file'
exit 1
fi
set -- $args
for i
do
case "$i" in
-c) shift;echo "flag c set to $1";shift;;
-a) shift;echo "flag a set";;
-b) shift;echo "flag b set";;
esac
done
$ ./g -abc "foo"
flag a set
flag b set
flag c set to foo
- Better yet, parse using Bash/sh built-in getopts (see here for more information and examples).
#!/bin/bash
while getopts "abc:" flag
do
echo "$flag" $OPTIND $OPTARG
done
$ ./g -abc "foo"
a 1
b 1
c 3 foo
- To parse option like --value=name ([1])
until [[ ! "$*" ]]; do
if [[ ${1:0:2} = '--' ]]; then
PAIR=${1:2}
PARAMETER=$(echo ${PAIR%=*} | tr [:lower:]- [:upper:]_)
eval P_$PARAMETER=${PAIR##*=}
fi
shift
done
Empty a file keeping permissions
Empty a file named filename, keeping the same permission and user/group:
>filename
Print multi-lines with echo
Print multi-lines text with echo:
$ echo -e "Some text\n...on 2 lines..." # Enable interpretation of backslash escapes (must be quoted!)
Some text
...on 2 lines...
Print multi-line variables with echo
One can save in a variable the multi-line output of a command. Later this variable can echoed while preserving the linefeeds if the variable is enclosed in quotes "...":
$ mymultilinevar=$(<myfile.txt sed -e'/first line/,/last line/')
$ echo "$mymultilinevar"
first line
second line
...
last line
Echo with colors
The command echo can display colors thanks to escape sequence commands [2]:
echo -e "\033[35;1m Shocking \033[0m" #Display "shocking" in bright purple
The first character is the escape character 27 (033 in octal). One can also type directly ^[ (i.e. Ctrl-AltGr-[). The syntax is (where spaces were added for clarity)
\033 [ <command> m \033 [ <command> ; <command> m
Note that commands can be chained. The set of commands is given in the color table below:
| code | style | code | foreground | code | foreground | code | background | code | background |
|---|---|---|---|---|---|---|---|---|---|
| 0 | default colour | 90 | dark grey | 40 | black | 100 | dark grey | ||
| 1 | bold | 31 | red | 91 | light red | 41 | red | 101 | light red |
| 4 | underlined | 32 | green | 92 | light green | 42 | green | 102 | light green |
| 5 | flashing text | 33 | orange | 93 | yellow | 43 | orange | 103 | yellow |
| 7 | reverse field | 34 | blue | 94 | light blue | 44 | blue | 104 | light blue |
| 35 | purple | 95 | light purple | 45 | purple | 105 | light purple | ||
| 36 | cyan | 96 | turquoise | 46 | cyan | 106 | turquoise | ||
| 37 | grey | 47 | grey |
Get file size
The different ways to extract file size in a Bash script:
SIZE=$(stat -c%s "$FILENAME") # Using stat
SIZE=$(ls -l $FILENAME | awk -F" "'{ print $5 }') # Using ls / awk
SIZE=$(du -b $FILENAME | sed 's/\([0-9]*\)\(.*\)/\1/') # Using du
SIZE=$(cat $FILENAME | wc -c) # Using cat / wc
SIZE=$(ls -l $FILENAME | cut -d " " -f 6) # Using ls / cut
Read file content into env variable
Read the content of a file into an environment variable:
PID=`cat $PIDFILE`
read PID < $PIDFILE
Get the PID of a new process
Getting the pid of a new process (when other processes with same name are already running)
oldPID=`pidofproc /usr/bin/ssh`
/usr/bin/ssh -f -N -n -q -D 1080 noekeon
RETVAL=$?
newPID=`pidofproc /usr/bin/ssh`
uniqPID=`echo $oldPID $newPID|sed -e 's/ /\n/g'|sort|uniq -u`
echo $uniqPID
Get the PID of a running process
Getting the pid of a running process
pid=$(pidof -o $$ -o $PPID - o %PPID -x /bin/ssh)
Detect if a given process PID is running
This is actually a tricky one. Some good solutions, all giving answer in $?:
[ -e /proc/$pid ] # nice, but is it portable?
ps -p $pid >/dev/null # need redirect, otherwise ps will print the process found
/bin/kill -O $pid 2>/dev/null # need redirect, otherwise kill will complain if no process found
# ... also works with bash built-in kill
Some wrong / bad solutions:
ps -aef | grep $pid # --== FAIL ==-- Will match grep process itself + $pid as ppid
ps -aef | grep $name # --== FAIL ==-- Will match grep process itself
ps -aef | grep -v grep | grep $pid # --== UGLY ==-- ... and slow. Better use ps -fp $(pgrep $pid)
ps -p $pid | grep $pid # --== SLOW ==-- better test $? immediately
Don't use this method for locking in startup scripts. Be careful with race condition. The best solution is to use a mutex, or use an atomic command (like mkdir). See for example:
- http://flabdablet.nfshost.com/linux-scripts/test-locking.sh
- http://www.davidpashley.com/articles/writing-robust-shell-scripts.html#id2326620
Launch a process in the background
Different ways to launch process in the background (unordered - might be useful one day...). The double ampersand trick comes from here.
myprocess.exe &
exec myprocess.exe
exec myprocess.exe &
( ( exec myprocess.exe & ) & )
nohup myprocess.exe &
( ( nohup myprocess.exe & ) & )
Display the name / body of functions
To list the functions declared in the current environment, or to list the body of a function:
declare -f # List all defined functions and their bodies
declare -f name # List the body of function "name"
declare -F # List name of all defined functions
Return the subnet address
Solution from [3].
/sbin/ifconfig eth0 |
grep 'inet addr' | tr .: ' ' |
(read inet addr a b c d Bcast e f g h Mask i j k l;
echo $(( $a & $i )).$(( $b & $j )).$(( $c & $k )).$(( $d & $l )) )
Remove file name extensions
FILENAME="myfile.pdf"
echo ${FILENAME%%.pdf} # only matches '.pdf', not '.PDF'
echo ${FILENAME%%.???} # only matches 3-letter extension
Formatted output / printing using printf
printf is a Bash built-in function that allows printing formatted output much like the standard C printf instructions.
printf "%02d" 1 # outputs '01'
Delete files with special characters
find . -inum [inode] -exec rm -i {} \; # Use inode
rm -- -foo # Special case for name with a heading dash
rm ./-foo
Pits
A list of frequent gotcha's !
| Description | Example |
|---|---|
| Space! - Don't forget to add spaces whenever necessary, in particular around brace in function definition, or in test conditions for ifs. |
if -space- [ -space- -f /etc/foo -space- ]; then ... |
| Quote - Always quote parameters, variables passed to test in if ... then ... else: |
if [ "$name" -eq 5 ]; then ... |
| For loops with file - Use simply * to list files in for loops, not `ls *`: |
for file in *; cat "$file"; done # SUCCEEDS, even if white space
for file in `ls *`; cat "$file"; done # FAILS miserably
|
Incorrect variable definition
So it is MYVAR=value and not |
srcDir = $1 # WRONG - spaces around = sign
$srcDir=$1 # WRONG - $ prefix
maxW= $(sed -rn '/$^/Q' myfile.txt) # WRONG - SPACE!
srcDir=$1 # CORRECT
srcDir="$1" # BEST
|
| Semi-colon in find - Semi-colon in find commands must be escaped ! |
find . -exec echo {} ; # WRONG - semi-colon not escaped
find . -exec echo {} \; # CORRECT
|
| Variable not exported outside parens |
( read pid < $PID_FILE ) 2> /dev/null # WRONG - var pid not kept
read pid 2> /dev/null < $PID_FILE # CORRECT
|
| Wrong redirection order |
read pid < $PID_FILE 2> /dev/null # WRONG - error msg if $PID_FILE
# doesn't exist
read pid 2> /dev/null < $PID_FILE # CORRECT
|
| Using a bash built-in instead of external program Bash built-in commands override external commands with same name (eg. kill and echo) |
$ type kill # kill is a shell builtin
$ type /bin/kill # /bin/kill is /bin/kill
$ /bin/kill -v # kill (cygwin) 1.14
|
| Variables assigned by read built-in seems to be empty → use a surrounding parens! |
echo "1 2 3" | (read a b c; echo $a $b $c) # CORRECT
echo "1 2 3" | read a b c; echo $a $b $c # WRONG
|
| Don't quote tilde in if test block | if [ -a ~/bin/"my file" ]; then echo found; fi # CORRECT
if [ -a "~/bin/my file" ]; then echo found; fi # WRONG
|
| Need quoting when echoing a variable with embedded newlines. This is because echo takes newlines (like any blanks) as parameter separator |
HEADER=$(sed -rn '/$^/Q' myfile.txt)
echo "$HEADER" # CORRECT
echo $HEADER # WRONG - newline are removed
|