Bash Tips and Pitfalls: Difference between revisions

From miki
Jump to navigation Jump to search
Line 6: Line 6:
* [http://www.davidpashley.com/articles/writing-robust-shell-scripts.html#id2326620 Writing Robust Bash Shell Scripts]
* [http://www.davidpashley.com/articles/writing-robust-shell-scripts.html#id2326620 Writing Robust Bash Shell Scripts]
* [http://tldp.org/LDP/abs/html/gotchas.html Advanced Bash-Shell Scripting - Gotchas]
* [http://tldp.org/LDP/abs/html/gotchas.html Advanced Bash-Shell Scripting - Gotchas]

== Tips for Robust Scripts ==
TBC


== Tips ==
== Tips ==

Revision as of 09:46, 26 January 2011

Reference

Local page:

External links:

Tips for Robust Scripts

TBC

Tips

Parsing Command-Line Option Parameters

#!/bin/bash
# (old version)
args=`getopt abc: $*`
if test $? != 0
  then
    echo 'Usage: -a -b -c file'
    exit 1
fi
set -- $args
for i
do
  case "$i" in
    -c) shift;echo "flag c set to $1";shift;;
    -a) shift;echo "flag a set";;
    -b) shift;echo "flag b set";;
  esac
done
$ ./g -abc "foo"
flag a set
flag b set
flag c set to foo
#!/bin/bash
while getopts  "abc:" flag
do
  echo "$flag" $OPTIND $OPTARG
done
$ ./g -abc "foo"
a 1
b 1
c 3 foo
  • To parse option like --value=name ([1])
until [[ ! "$*" ]]; do
  if [[ ${1:0:2} = '--' ]]; then
    PAIR=${1:2}
    PARAMETER=$(echo ${PAIR%=*} | tr [:lower:]- [:upper:]_)
    eval P_$PARAMETER=${PAIR##*=}
  fi
  shift
done

Empty a file keeping permissions

Empty a file named filename, keeping the same permission and user/group:

>filename

Print multi-lines with echo

Print multi-lines text with echo:

$ echo -e "Some text\n...on 2 lines..."                    # Enable interpretation of backslash escapes (must be quoted!)
Some text
...on 2 lines...

Print multi-line variables with echo

One can save in a variable the multi-line output of a command. Later this variable can echoed while preserving the linefeeds if the variable is enclosed in quotes "...":

$ mymultilinevar=$(<myfile.txt sed -e'/first line/,/last line/')
$ echo "$mymultilinevar"
first line
second line
...
last line

Echo with colors

The command echo can display colors thanks to escape sequence commands [2]:

echo -e "\033[35;1m Shocking \033[0m"       #Display "shocking" in bright purple

The first character is the escape character 27 (033 in octal). One can also type directly ^[ (i.e. Ctrl-AltGr-[). The syntax is (where spaces were added for clarity)

\033 [ <command> m
\033 [ <command> ; <command> m

Note that commands can be chained. The set of commands is given in the color table below:

code style code foreground code foreground code background code background
0 default colour 90 dark grey 40 black 100 dark grey
1 bold 31 red 91 light red 41 red 101 light red
4 underlined 32 green 92 light green 42 green 102 light green
5 flashing text 33 orange 93 yellow 43 orange 103 yellow
7 reverse field 34 blue 94 light blue 44 blue 104 light blue
35 purple 95 light purple 45 purple 105 light purple
36 cyan 96 turquoise 46 cyan 106 turquoise
37 grey 47 grey

Get file size

The different ways to extract file size in a Bash script:

SIZE=$(stat -c%s "$FILENAME")                              # Using stat
SIZE=$(ls -l $FILENAME | awk -F" "'{ print $5 }')          # Using ls / awk
SIZE=$(du -b $FILENAME | sed 's/\([0-9]*\)\(.*\)/\1/')     # Using du
SIZE=$(cat $FILENAME | wc -c)                              # Using cat / wc
SIZE=$(ls -l $FILENAME | cut -d " " -f 6)                  # Using ls / cut

Read file content into env variable

Read the content of a file into an environment variable:

PID=`cat $PIDFILE`
read PID < $PIDFILE

Get the PID of a new process

Getting the pid of a new process (when other processes with same name are already running)

oldPID=`pidofproc /usr/bin/ssh`
/usr/bin/ssh -f -N -n -q -D 1080 noekeon
RETVAL=$?
newPID=`pidofproc /usr/bin/ssh`
uniqPID=`echo $oldPID $newPID|sed -e 's/ /\n/g'|sort|uniq -u`
echo $uniqPID

Get the PID of a running process

Getting the pid of a running process

pid=$(pidof -o $$ -o $PPID - o %PPID -x /bin/ssh)

Detect if a given process is running

This is actually a tricky one. Some good solutions, all giving answer in $?:

[ -e /proc/$pid ]               # PID  - nice, but is it portable?
pkill -0 $name                  # NAME - probably the best using command-name
ps -p $pid >/dev/null           # PID  - need redirect, otherwise ps will print the process found
/bin/kill -O $pid 2>/dev/null   # PID  - need redirect, otherwise kill will complain if no process found
                                #        ... also works with bash built-in kill

Some wrong / bad solutions:

ps -aef | grep $pid                   # --== FAIL ==-- Will match grep process itself + $pid as ppid
ps -aef | grep $name                  # --== FAIL ==-- Will match grep process itself
ps -aef | grep -v grep | grep $pid    # --== UGLY ==-- ... and slow. Better use ps -fp $(pgrep $pid)
ps -p $pid | grep $pid                # --== SLOW ==-- better test $? immediately

Don't use this method for locking in startup scripts. Be careful with race condition. The best solution is to use a mutex, or use an atomic command (like mkdir). See for example:

Launch a process in the background

Different ways to launch process in the background (unordered - might be useful one day...). The double ampersand trick comes from here.

myprocess.exe &
exec myprocess.exe
exec myprocess.exe &
( ( exec myprocess.exe & ) & )
nohup myprocess.exe &
( ( nohup myprocess.exe & ) & )

Display the name / body of functions

To list the functions declared in the current environment, or to list the body of a function:

declare -f                    # List all defined functions and their bodies
declare -f name               # List the body of function "name"
declare -F                    # List name of all defined functions

Return the subnet address

Solution from [3].

/sbin/ifconfig eth0 |
grep 'inet addr' | tr .: '  ' |
(read inet addr a b c d Bcast e f g h Mask i j k l;
echo $(( $a & $i )).$(( $b & $j )).$(( $c & $k )).$(( $d & $l )) )

Remove file name extensions

FILENAME="myfile.pdf"
echo ${FILENAME%%.pdf}          # only matches '.pdf', not '.PDF'
echo ${FILENAME%%.???}          # only matches 3-letter extension

Formatted output / printing using printf

printf is a Bash built-in function that allows printing formatted output much like the standard C printf instructions.

printf "%02d" 1                  # outputs '01'

Delete files with special characters

find . -inum [inode] -exec rm -i {} \;     # Use inode
rm -- -foo                                 # Special case for name with a heading dash
rm ./-foo

Pits

A list of frequent gotcha's !

Description Example
Space! - Don't forget to add spaces whenever necessary, in particular around brace in function definition, or in test conditions for ifs.

if -space- [ -space- -f /etc/foo -space- ]; then ...
function myfunc() { -space- echo Hello, World!; }

Quote - Always quote parameters, variables passed to test in if ... then ... else:

if [ "$name" -eq 5 ]; then ...

For loops with file - Use simply * to list files in for loops, not `ls *`:
for file in *; cat "$file"; done       # SUCCEEDS, even if white space
for file in `ls *`; cat "$file"; done  # FAILS miserably
Incorrect variable definition
  • NO space between the variable name and the subsequent equal sign
  • No dollar $ prefix!!!

So it is MYVAR=value and not MYVAR= value !!!

srcDir = $1                         # WRONG - spaces around = sign
$srcDir=$1                          # WRONG - $ prefix
maxW= $(sed -rn '/$^/Q' myfile.txt) # WRONG - SPACE!
srcDir=$1                           # CORRECT
srcDir="$1"                         # BEST
Semi-colon in find - Semi-colon in find commands must be escaped !
find . -exec echo {} ;        # WRONG - semi-colon not escaped
find . -exec echo {} \;       # CORRECT
Variable not exported outside parens
( read pid < $PID_FILE ) 2> /dev/null   # WRONG - var pid not kept
read pid 2> /dev/null < $PID_FILE       # CORRECT
Wrong redirection order
read pid < $PID_FILE 2> /dev/null  # WRONG - error msg if $PID_FILE
                                   #   doesn't exist
read pid 2> /dev/null < $PID_FILE  # CORRECT
Using a bash built-in instead of external program
Bash built-in commands override external commands with same name (eg. kill and echo)
$ type kill                 # kill is a shell builtin
$ type /bin/kill            # /bin/kill is /bin/kill
$ /bin/kill -v              # kill (cygwin) 1.14
Variables assigned by read built-in seems to be empty → use a surrounding parens!
echo "1 2 3" | (read a b c; echo $a $b $c)  # CORRECT
echo "1 2 3" | read a b c; echo $a $b $c    # WRONG
Don't quote tilde in if test block
if [ -a ~/bin/"my file" ]; then echo found; fi # CORRECT
if [ -a "~/bin/my file" ]; then echo found; fi # WRONG
Need quoting when echoing a variable with embedded newlines.
This is because echo takes newlines (like any blanks) as parameter separator
HEADER=$(sed -rn '/$^/Q' myfile.txt)
echo "$HEADER" # CORRECT
echo $HEADER   # WRONG - newline are removed