Bash

From miki
Jump to navigation Jump to search

Related pages

Other pages dedicated to Bash on this wiki:

References

External references: 

help getopts
help test
help if

About alternatives to Bash:

Shell

Keyboard shortcuts

Some of the most used keyboard shortcuts in Bash (excerpt from the manual pages).
C- refers to Ctrl key, M- refers to Meta key (typ. Alt+key, or Esc then key), S- refers to Shift key.
\e introduces an escape character (same as ^[ printed by verbatim command (Ctrl-V) in vim)

Customized keyboard shortcuts described below can be obtained by adding the following lines to e.g. your ~/.inputrc file: 

"\eOA":history-search-backward     #Up arrow
"\e[A":history-search-backward     #Up arrow
"\eOB":history-search-forward      #Down arrow
"\e[B":history-search-forward      #Down arrow
"\C- ":dynamic-complete-history    #Ctrl-space
"\e ":dynamic-complete-history     #Esc-space
"\e[2~":paste-from-clipboard       #Insert
tab:menu-complete                  #TAB
"\e[Z":menu-complete-backward      #Shift-TAB
"\C-y":menu-complete-backward      #Ctrl-Y
Note on defining new bindings
Binding by defaults are defined in file ~/.inputrc. Alternatively they can also be defined in the start file ~/.bashrc using command bind.
They are different method to define the binding keys
  • Specifying the key Keyname, without double quotes (but always within single quotes when using command bind)
  • Specifying the key Keyseq, with double quotes (*and* single quotes when using command bind)
  • Specifying the key Verbatim, with double quotes, and using control-V followed by key in Bash or Vi. Note: This is a good trick to know the key sequence of special keys like arrow key.
The binding keyseq differs depending on whether Readline is configured to support 8-bit input/output characters. Cygwin and openSUSE have different settings regarding this. To support 8-bit input/output (like in OpenSUSE), add the following to the ~/.inputrc:
set convert-meta off
set input-meta on
set meta-flag on
set output-meta on

bind 'tab:dynamic-complete-history'                #Keyname
bind '"\C-i":dynamic-complete-history'             #Keyseq
bind '"<Control-V><Tab>:dynamic-complete-history'  #Verbatim - <> means press the given key while editing

See also 

man readline
Using negative argument in readline command
[3], [4]. Examples:
"\e[Z":"\M--1\t"          # \M--1 is actually-1 argument for next command , \t i.e. TAB
"\e[Z":"\M--\t"           # We can skip the 1
"\e[Z":"\e--\C-i"         # Equivalent, but using \e instead of \M, and \C-i instead of \t

Commands for Moving

forward-word (M-f, C-→, M-→, S-→)
Move forward to the end of the next word. Words are composed of alphanumeric characters (letters and digits).
!!! C-→, M-→, S-→ not supported in Cygwin.
backward-word (M-b, C-←, M-←, S-←)
Move back to the start of the current or previous word. Words are composed of alphanumeric characters (letters and digits).
!!! C-←, M-←, S-← not supported in Cygwin.
clear-screen (C-l)
Clear the screen leaving the current line at the top of the screen. With an argument, refresh the current line without clearing the screen.

Commands for Manipulating the History

reverse-search-history (C-r)
Search backward starting at the current line and moving 'up' through the history as necessary. This is an incremental search.
This is equivalent to interactive grep in history — very powerful
forward-search-history (C-s)
Search forward starting at the current line and moving 'down' through the history as necessary. This is an incremental search.
!!! Not supported in Cygwin.
history-search-forward (↓)
Search forward through the history for the string of characters between the start of the current line and the point. This is a non-incremental search.
!!! Custom keyboard shortcut.
history-search-backward (↑)
Search backward through the history for the string of characters between the start of the current line and the point. This is a non-incremental search.
!!! Custom keyboard shortcut.
yank-last-arg (M-., M-_)
Insert the last argument to the previous command (the last word of the previous history entry). With an argument, behave exactly like yank-nth-arg. Successive calls to yank-last-arg move back through the history list, inserting the last argument of each line in turn. The history expansion facilities are used to extract the last argument, as if the "!$" history expansion had been specified.
history-expand-line (M-^)
Perform history expansion on the current line. See HISTORY EXPANSION below for a description of history expansion.
!!! AZERTY keyboard: press Alt-^ twice
operate-and-get-next (C-o)
Accept the current line for execution and fetch the next line relative to the current line from the history for editing. Any argument is ignored.
edit-and-execute-command (C-xC-e)
Invoke an editor on the current command line, and execute the result as shell commands. Bash attempts to invoke $VISUAL, $EDITOR, and emacs as the editor, in that order.

Commands for Changing Text

delete-char (C-d)
Delete the character at point. If point is at the beginning of the line, there are no characters in the line, and the last character typed was not bound to delete-char, then return EOF.
backward-delete-char (Rubout)
Delete the character behind the cursor. When given a numeric argument, save the deleted text on the kill ring.
quoted-insert (C-q, C-v)
Add the next character typed to the line verbatim. This is how to insert characters like C-q, for example.
transpose-chars (C-t)
Drag the character before point forward over the character at point, moving point forward as well. If point is at the end of the line, then this transposes the two characters before point. Negative arguments have no effect.
transpose-words (M-t)
Drag the word before point past the word after point, moving point over that word as well. If point is at the end of the line, this transposes the last two words on the line.
upcase-word (M-u)
Uppercase the current (or following) word. With a negative argument, uppercase the previous word, but do not move point.
downcase-word (M-l)
Lowercase the current (or following) word. With a negative argument, lowercase the previous word, but do not move point.
capitalize-word (M-c)
Capitalize the current (or following) word. With a negative argument, capitalize the previous word, but do not move point.
overwrite-mode
Toggle overwrite mode. With an explicit positive numeric argument, switches to overwrite mode. With an explicit non-positive numeric argument, switches to insert mode. This command affects only emacs mode; vi mode does overwrite differently. Each call to readline() starts in insert mode. In overwrite mode, characters bound to self-insert replace the text at point rather than pushing the text to the right. Characters bound to backward-delete-char replace the character before point with a space. By default, this command is unbound.

Killing and Yanking

kill-line (C-k)
Kill the text from point to the end of the line.
unix-line-discard (C-u)
Kill backward from point to the beginning of the line. The killed text is saved on the kill-ring.
kill-word (M-d)
Kill from point to the end of the current word, or if between words, to the end of the next word. Word boundaries are the same as those used by forward-word.
backward-kill-word (M-Rubout)
Kill the word behind point. Word boundaries are the same as those used by backward-word.
unix-word-rubout (C-w)
Kill the word behind point, using white space as a word boundary. The killed text is saved on the kill-ring.

Numeric Arguments

digit-argument (M-0, M-1, ..., M--)
Add this digit to the argument already accumulating, or start a new argument. M-- starts a negative argument.

Completing

complete (TAB)
Attempt to perform completion on the text before point. Bash attempts completion treating the text as a variable (if the text begins with $), username (if the text begins with ~), hostname (if the text begins with @), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted.
menu-complete (TAB) / menu-complete-backward (Shift-TAB, C-Y)
Similar to complete but replaces the word with next (previous) completion.
!!! Apparently Shift-Tab doesn't work in Cygwin. Use Ctrl-Y instead
possible-completions (M-?)
List the possible completions of the text before point.
insert-completions (M-*)
Insert all completions of the text before point that would have been generated by possible-completions.
dynamic-complete-history (M-SPACE,C-SPACE)
Attempt completion on the text before point, comparing the text against lines from the history list for possible completion matches.
!!! Must use combination Esc + Tab. Custom combination in Cygwin.

Miscellaneous

prefix-meta (ESC)
Metafy the next character typed. ESC f is equivalent to Meta-f.
undo (C-_, C-x C-u)
Incremental undo, separately remembered for each line.
!!! AZERTY keyboard: press Control--.
revert-line (M-r)
Undo all changes made to this line. This is like executing the undo command enough times to return the line to its initial state.
set-mark (C-@, M-<space>)
Set the mark to the point. If a numeric argument is supplied, the mark is set to that position.
exchange-point-and-mark (C-x C-x)
Swap the point with the mark. The current cursor position is set to the saved position, and the old cursor position is saved as the mark.

History

Some examples (assume that magic-space is mapped to Space) 

echo 12 34
!!                 # Execute previous command
!1                 # Execute command n°1
!-2                # Execute penultimate command
echo 12 34
^12^56^            # Execute previous command, replacing 12 with 56. Quick for !!:s/12/56/
!!<space>          # Example of magic space
echo !-2<space>    #   ... also works anywhere in the current line
for i in <C-xC-e>  # Shortcut to edit cmd in editor
fc                 # Edit previos command in editor

Script Quick Tutorial

Invocation

A bash shell can be (see man bash)

  • either interactive or not interactive
The shell is interactive when started with the -i option, or when not started without non-option arguments and without the -c option, and whose stdin/stderr are both connected to terminals.
  • either a login shell or not a login shell
A login shell is one whose first character of argument zero is a - (i.e. $0 == -bash or such), or one started with the --login or -l option.
Using su a login shell is started with option -, -l or --login.

So 

su - username               # LOGIN   and INTERACTIVE
bash --login
bash -l
su username                 # NOT login   and INTERACTIVE
bash
bash -i
su - username -c "..."      # LOGIN   and NOT interactive
bash --login -c "..."
su username -c "..."        # NOT login and NOT interactive
bash -c "..."
bash -c "..." --login       # idem (actually --login is passed to -c commad)
  • Use command shopt to know whether current shell is interactive / login.
  • Interactive shells have i defined in $-, and also define variable PS1.
  • Sourcing file can be skipped / enforced with --noprofile, --norc or --rcfile.

A simple script to detect the current invocation [5]: 

[[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'
shopt -q login_shell && echo 'Login shell' || echo 'Not login shell'

The following files are sourced by bash at invocation

NOT login LOGIN
INTERACTIVE
  • First /etc/bash.bashrc.
  • Then ~/.bashrc
  • First /etc/profile.
  • Then either ~/.bash_profile, ~/.bash_login, ~/.profile, whichever is found first.
NOT interactive

None

But on Ubuntu, /etc/profile automatically sources /etc/bash.bashrc, which then aborts if it detects that the shell is not interactive. Also ~/.profile sources ~/.bashrc, which also aborts if non-interactive. So in an out-of-the-box configuration, the following files are sourced:

NOT login LOGIN
INTERACTIVE

/etc/bash.bashrc
~/.bashrc

/etc/profile
/etc/bash.bashrc
~/.profile
~/.bashrc

NOT interactive

None

/etc/profile
~/.profile

Environment

export MYVAR=myvalue && command-to-execute         #MYVAR defined for the current shell and invoked shell
MYVAR=myvalue && command-to-execute                #MYVAR defined for the current shell
MYVAR=myvalue command-to-execute                   #MYVAR only defined for the subsequent command

As a rule of thumb, better define environment variables (PATH, etc) in ~/.profile, so that they are available to all program launched from a login shell (see [6], [7]).

Quoting

Escape \ 
cp file \'file\$\'
Preserve literal value of the next character
Single quote '...' 
echo 'it cost $10'
Preserve literal value of all characters
Dollar Single quote '...' 
echo $'it cost \'$10\''
Preserve literal value of all characters, except escaped character
Double quote "..."' 
cp file "file $var"
Preserve literal value of all characters, except $ ` \ !
Dollar Double quote $"..."' 
echo $"file not found"
Translate string to current locale (for internationalization

Compound Command

(list) 
if ( kill $pid; ps -p $pid )
Execute list in a subshell and set return status to last command returned status
{ list; } 
{ read v1; read v2; } <foo
Group command — Execute list in current environment. Note the ; and whitespaces!
((expr)) 
 if ((i<10)); ...
Arithmetic evalution. Set return status accordingly.
[[ expr ]] 
 if [[ -e foo && -e bar ]]; ...
Condition expression. Set return status accordingly.

Expansion and Substitution

This is only a subset of the expansion possibilities. See the man pages for more information

${!param} 
var=hello; ref=var; 
echo ${!ref}          # "hello"
foo[0]=foo0; foo[1]=foo1
ref=foo[0]            # ... also for arrays!
echo ${!ref}          # "foo0"
Indirect expansion — Equivalent to eval \$$ref, but cleaner and safer. It also works for arrays!
${param%word}
${param%%word} 
var1=file.ext; var2=dir//; 
echo ${var1%%.ext}    # "file"
echo ${var2%%/}       # "dir"
Parameter expansion — Suffix removal. Use a single % for shortest match.
Memo: 100% → suffix.
${param#word}
${param##word} 
var=__file
echo ${var##__}       # "file"
Parameter expansion — Prefix removal. Use a single # for shortest match.
memo: #comment → prefix.
${param/pat/str}
${param//pat/str} 
var=sometext
echo ${var/e/_}       # "som_text"
Parameter expansion — Search & replace. Use /pattern to replace all matches
$(command)
`command` 
echo $(basename $0)
Command substitution — Expands to the output of command
Trailing newlines are stripped!
$((expr))
$[expr] 
echo $((16 * 16))
Arithmetic expansion — Expands to the result of expression ($[...] is deprecated)
<(command)
>(command) 
diff <(ls dir1) <(ls dir2)
Process substitution — Substitute a process with a named pipe connected to the process (see tips)
* ? [...] 
echo *
Pathname expansion — Expands to file names matching given pattern (done after word splitting)
{0..19} 
echo {0..9}
# 0 1 2 3 4 5 6 7 8 9
Sequence expansion

Here summary of expansion result depending on whether parameter is unset, set and/or null:

set and not null null unset
${parameter:-word} substitute parameter substitute word substitute word
${parameter-word} substitute parameter substitute null substitute word
${parameter:=word} substitute parameter assign word assign word
${parameter=word} substitute parameter substitute null assign word
${parameter:?word} substitute parameter error, exit error, exit
${parameter?word} substitute parameter substitute null error, exit
${parameter:+word} substitute word substitute null substitute null
${parameter+word} substitute word substitute word substitute null

Positional Parameters

See section "Looping on positional parameters" for an illustration on how to process each parameters in a for loop. 

# !/bin/bash

function parsing ()
{
    echo "($1) ($2) ($3) ($4)"
}

echo -n 'parsing $*   -- ';  parsing $*
echo -n 'parsing $@   -- ';  parsing $@
echo -n 'parsing "$*" -- ';  parsing "$*"
echo -n 'parsing "$@" -- ';  parsing "$@"
# Echo number of parameters
echo $#

$ ./test.sh p1 p2 "p 3" 'p 4'
# parsing $*   -- (p1) (p2) (p) (3)
# parsing $@   -- (p1) (p2) (p) (3)
# parsing "$*" -- (p1 p2 p 3 p 4) () () ()
# parsing "$@" -- (p1) (p2) (p 3) (p 4)
# 4

To test if there are parameters: 

if [ $# -eq 0 ]; then
    echo No parameters!
    exit 1
fi

Operators

VAR+=value

if VAR is a string, append value
if VAR is an array, add element value
if VAR is a numerical value, add value 

STRING=
STRING+=" foo"    # Now string is "foo" (w/o space!)
STRING+=" bar"    # Now string is "foo bar"

String pattern matching

Several possibilities to do pattern matching in strings:

  • Using operator =~ and regular expression
  • Using operator == and Bash pattern
  • Using case

Using case is the fastest option for simple, single patterns (10% faster than ==). == is faster than =~ (for instance fooMATCHbar == *MATCH* is roughly 3x faster than fooMATCHbar =~ .*MATCH.* ).

Examples
if [[ fooMATCHbar =~ .*MATCH.* ]]; then echo found; done
if [[ fooMATCHbar == *MATCH* ]]; then echo found; done
case fooMATCHbar in *MATCH*) echo found;; esac

Redirection - Descriptors, Here Documents and Here Strings

&>word
>&word 		Redirect both stdout and stderr (semantically equivalent to >word 2>&1)
|& Pipe both stdout and stderr


Description Source Output
Descriptors - redirect input / output to files or file descriptor. These file names have special meaning: /dev/fd/fd (with 2nd fd an integer), /dev/stdin, /dev/stdout, /dev/stderr, /dev/tcp/host/port, /dev/udp/host/port
See Using Process Substitution tip and stderr redirection tip for more advanced redirection. 
ls > /dev/stderr    # WRONG
ls >> /dev/stderr   # BETTER
ls >&2              # BEST
ls 2>&1 > dirlist   # WRONG 
ls > dirlist 2>&1   # OK
ls >& dirlist       # BETTER
ls &> dirlist       # BETTER (PREFERRED)
./myapp |& tee foo  # Pipe stdout & stderr
Here Documents - Use <<word to declare here documents.
Check BASH manpage for more information.

We can also combine several here documents on the same line, or use pipelines [8] 

cat <<__EOF__
	First line
	Second line
	Third line
__EOF__

# here document with pipeline
cat <<__EOF__ |
	First line
	Second line
	Third line
__EOF__
tr [a-z] [A-Z]

# Multi here documents
cat <<eof1; cat <<eof2
Hi,
eof1
Helene.
eof2

	First line
	Second line
	Third line

	FIRST LINE
	SECOND LINE
	THIRD LINE

	Hi,
	Helene.
Here Documents - Use <<- to strip leading tab characters. 
cat <<-__EOF__
	First line
	Second line
	Third line
__EOF__

First line
Second line
Third line
Here Strings - Use <<<word to declare here strings.
 
cat <<<'Hello, World!'
TEXT='Hello, World!' cat <<<$TEXT
cat<<<"First line
> Second line
> Third line"

Hello, World!
Hello, World!
First line
Second line
Third line

If ... then ... [elif ...] ... else ... fi

(See [9] for in-depth discussion on Bash tests)

The basic syntax for if-then-else sequence control is 

if list; then list; [ elif list; then list; ] ... [ else list; ] fi

The if/then construct tests whether the exit status of a list of command is 0 (i.e. success), and if so, executes another list of command. Test conditions can be written using shell built-in command test, or equivalently the square brackets [...]. Other possibilities are using the extended test command [[...]] (that performs parameter expansion and command substitution as well as arithmetic evaluation, see [10]), or the double-parens ((...)) expression (see [11]) that evaluates an arithmetic expression (returs true if expression is non-zero). 

if mv $src $dst; then echo move is ok; else move is not ok; fi                 # Using list
if test -f /etc/foo; then echo file found; else echo file NOT found; fi        # test command
if [ -f /etc/foo ]; then echo file found; else echo file NOT found; fi         # [ is a synomym for test and a builtin
if [[ -f $file ]]; then echo file $file found; else echo file NOT found; fi    # Extended test construct - &&, ||, < and > works 
if (( 128 && 64 )); then echo result is non-zero; else echo result is zero; fi # Expands and evaluates arithmetic expression

Here's test's options (more info with help test): 

-d FILE        # Check if the file is a directory
-e FILE        # Check if the file exists
-f FILE        # Check if the file is a regular file
-g FILE        # Check if the file hash SGID permissions
-r FILE        # Check if the file is readable
-s FILE        # Check if the file's size is not 0
-u FILE        # Check if the file has SUID permissions
-w FILE        # Check if the file is writeable
-x FILE        # Check if the file is executable
NBR1 -eq NBR2  # Check is NBR1 is equals to NBR2
NBR1 -ne NBR2  # Check if NBR1 is not equals to NBR2
NBR1 -ge NBR2  # Check if NBR1 is greater than or equal to NBR2
NBR1 -gt NBR2  # Check if NBR1 is greater than NBR2
NBR1 -le NBR2  # Check if NBR1 is less than or equal to NBR2
NBR1 -lt NBR2  # Check if NBR1 is less than NBR2
STR1 = STR2    # Check if STR1 is the same as STR2
STR1 != STR2   # Check if STR1 is not the same as STR2
-n STR         # Evaluates to true if STR is not null (can be used to test whether a variable is defined)
-z STR         # Evaluates to true if STR is null.

Conditions can be combined with && and || or with test operator -a and -o. Use ! to negate a test: 

if [ $x -ge 5 ] && [ $x -le 10 ]; then ...; fi
if [ $x -gt 5 ] || [ $x -lt 10 ]; then ...; fi
if [ $x -ge 5 -a $x -le 10 ]; then ...; fi
if [ $x -gt 5 -o $x -lt 10 ]; then ...; fi
if ! [ -a /some/file.txt ]; then ...; fi                     #negation
if [ -n "$DEFINED" ]; then echo DEFINED is defined; fi
if [ "$DEFINED" ]; then echo DEFINED is defined; fi          #also works

Some examples: 

if [ "$name" -eq 5 ]; then echo one; else echo two; fi
if [ -d ~/var ]; then touch ~/var/done; else { mkdir ~/var; touch ~/var/done; } fi   # braces { ... } are optional here

Alternative: Using && and ||

Using && and ||, one can also build a if ... then ... else statement: 

[ -f /etc/hosts ] && { echo one; echo two; } || { echo three; echo four; }

Using [[expression]]

The conditinal expressions [[expression]] uses the same CONDITIONAL EXPRESSIONS as the regular [ ... ], but in addition accept the following construct (see manual pages for more):

value == pattern
value != pattern 

if [[ $NAME == *.c ]]; then echo C Source file; fi
String Pattern is considered a shell pattern

value =~ pattern 

if [[ $(ps $PID) =~ ssh-agent ]]; then echo $PID is ssh-agent pid; fi
String Pattern is considered an extended regular regex

Thanks to these, there is no need for constructs using grep: 

if ps aux | grep ssh-agent; then echo ssh-agent found; fi        # NOT EFFICIENT, 2 processes spawn

if [[ $(ps aux) =~ ssh-agent ]]; then echo ssh-agent foudn; fi   # BETTER!!!

Strings

Reference: Advanced Bash Scripting Guide 

len=${#line}         # Get length of string line
LANG=C LC_ALL=C
bytlen=${#line}      # Get *byte* length

stringZ=abcABC123ABCabc                      # 0-based indexing
echo ${stringZ:0}                            # abcABC123ABCabc
echo ${stringZ:1}                            # bcABC123ABCabc
echo ${stringZ:7}                            # 23ABCabc

echo ${stringZ:7:3}                          # 23A

# Using negative index
echo ${stringZ:-4}                           # abcABC123ABCabc - FAIL! because construct ${parameter:-default}.
echo ${stringZ:(-4)}                         # Cabc            - SUCCESS! 
echo ${stringZ: -4}                          # Cabc            - SUCCESS!

Arrays

See [12] for more detailed information. 

# Assigning array elements
area[11]=23        #  Array members need not be consecutive or contiguous.
area[13]=37
area[51]=UFOs

# APPEND to array
ARRAY=()
ARRAY+=('foo')
ARRAY+=('bar baz')

# LOOP on array element:
for i in ${!ARRAY[*]}; do echo ${ARRAY[$i]}; done

# Use curly brackets to read!
echo "\${area[11]}         = ${area[11]}"
echo "\${area[0]}  = $area = ${area[0]}"

# Use array to echo first word in a list:
make="/usr/bin/make -r --no-print-directory -j 2"
words=($make)
echo $words

# Store multiline in array
files=($(grep -l foo *))
echo "There are ${#files[@]} matching files, first is ${files[0]}"

# Other ways to assign an array
area2=( zero one two three four )       # zero-based indexing!
area3=([17]=seventeen [24]=twenty-four)

# Use quoting to insert white-spaces
array=( [0]="first element" [1]="second element" [3]="fourth element" )

# read file into array (one line, separated by blank)
read -a array < FILE

# Read file into array (line by line)
readarray -t array < FILE     # Bash 4.0
IFS=$'\r\n' read -d '' -r -a array < FILE
IFS=$'\r\n' GLOBIGNORE='*' command eval 'XYZ=($(cat FILE))'
                              # https://stackoverflow.com/questions/11393817/read-lines-from-a-file-into-a-bash-array
readarray -d '' array < <(find . -name "$input" -print0)
                              # https://stackoverflow.com/questions/23356779/how-can-i-store-the-find-command-results-as-an-array-in-bash
                              # See also tips page for more

# Some operators
${array:1}   == ${array[0]:1} # Param. expansion of 1st element, starting at pos #1 (2nd char)
${#array}    == ${#array[0]}  # Length of 1st element
${#array[*]} == ${#array[@]}  # Number of elements in array.
unset array[1]                # Remove 2nd element of array (but ***leave a HOLE in INDEX***)
array[1]=                     # ... idem
unset array                   # Delete entire array

# Re-INDEX an array (e.g. if some element deleted)
foo=(a b c)
declare -p foo                # declare -a foo=([0]="a" [1]="b" [2]="c")
unset foo[1]
declare -p foo                # declare -a foo=([0]="a" [2]="c")
foo=("${foo[@]}")             # Re-indexing
declare -p foo                # declare -a foo=([0]="a" [1]="c")

# CROP array
array=("${array[@]:1}")   # Crop 1st elm and reindex, using substring expansion
unset 'array[0]'          # Crop 1st elm, but doesn't reindex
array=("${array[@]}")     # ... to force reindex
# Keep only some of the last elements
# ... using a loop
while [ ${#array[@]} -gt 5 ] ; do
    array=("${array[@]:1}")
done
# ... using a test
[ ${#array[@]} -gt 5 ] && array=("${array[@]:$((${#array[@]} -5))}")

# Test arrays
[ ${#array[@]} -eq 0 ] && echo "array empty"
[[ -v array[@] ]] && echo "array is set"  # BASH 4.2 - But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY
[[ ${array@a} = a ]] && echo "array is ARRAY"  # BASH 5.x - ... ARRAY is DECLARED, but maybe not SET yet

# PRINT arrays
echo ${array[@]}              # Print elements separated by SPACE
echo ${array[*]}              # ... idem
echo "${array[*]}"            # ... idem
echo "${array[@]}"            # Print elements separated by 1st char in IFS
IFS='/'; echo "${array[*]}"   # MUST set IFS before!
IFS='/' echo "${array[*]}"    # Does NOT work because echo param got expanded BEFORE IFS assignment

unset array[1]
echo "${array[*]}"            # DELETED element are NOT printed

# COPY an array
copy=("${array[@]}")

# INDIRECT access to array element
REF=array[0]
# REF=array[@]    -- also works
# REF=!array[@]   -- DOES NOT work though (for keys)
echo ${!REF}

# ... better solution since bash 4.3+, use **namerefs**:
declare -A array=( [a]=A [b]=B )
declare -n REF=array          # ... or ...
local -n REF=array            # Better in functions
echo ${REF[a]}                # Element access
echo ${REF[@]}                # Return all values
echo ${!REF[@]}               # Return all keys

# INDIRECT length of an array or array element (w/o declare -n)
REF=array[@]                  # Our reference to an array
local current=("${!str}")     # Copy into an array using indirect ref

echo ${#current[*]}           # Now we can use regular array operator - array size
echo ${current[1]}            # Print 2nd element
echo ${#current[1]}           # ... and size of 2nd element

# Examine / serialize array
declare -p array              # declare -a array=([0]="two" [1]="one")

Associative Arrays

Associative arrays (so-called hash in Perl) are available in Bash since version 4.

  • Associative array are declared with declare -A ARRAY
  • Values are accessed with $ARRAY[key]
  • Keys are accessed with exclamation mark: ${!ARRAY[@]}
declare -A ARRAY                          # Declare 'ARRAY' as a (local) associative array
declare -gA ARRAY                         # Declare 'ARRAY' as a global associative array
                                          # ... handy when array is constructed in a function
ARRAY[foo]=FOO                            # Associate value 'FOO' to key 'foo'
ARRAY[bar]=BAR                            # Associate value 'BAR' to key 'bar'
declare -A ARRAY=( [foo]=FOO [bar]=BAR )  # ... idem, on one line
declare -p ARRAY                          # declare -A ARRAY=( [foo]=FOO [bar]=BAR )
echo ${ARRAY[foo]}                        # Print value associated with 'foo'
echo ${ARRAY[*]}                          # Print all values in ARRAY
echo ${ARRAY[@]}                          # ... idem
echo ${!ARRAY[*]}                         # Print all ***keys*** in ARRAY
echo "${ARRAY[*]}"                        # Same as for "$*"
echo "${ARRAY[@]}"                        # Same as for "$@"
${#ARRAY[*]} == ${#ARRAY[@]}              # Number of elements in ARRAY.
[ ${ARRAY[key]+x} ] && echo "exists"      # Check if 'key' exists, based on ${foo+x} returning x if foo set and nothing otherwise
[[ -v ARRAY[key] ]] && echo "exists"      # ... alternative (BASH 4.2)
[[ ${ARRAY@a} = A && -v ARRAY[key] ]]     # ... SAFER, test ARRAY is ASSOC first (BASH 5.0)
keys=(${ARRAY[@]})                        # All keys in a standard array

To delete an array or element: 

unset ARRAY                               # Empty the ARRAY (ARRAY is NOT ASSOCIATIVE anymore!)
unset ARRAY[foo]                          # Delete a KEY
unset ARRAY["my key"]                     # ... quote if KEY contains a space
K="my key"
unset ARRAY["$K"]                         # Delete a KEY contained in variable

To CLEAR an associative array in a function, UNSET it then declare it GLOBALLY with -gA. For instance: 

unset A
declare -A A
A[C]=2; 
a() 
{ 
  unset A        # Clear the array
  declare -gA A  # Declare it as ASSOCIATIVE and GLOBAL
  A[X]=1
}
a
declare -p A     # declare -A A=([X]="1")

The following test if ARRAY is set: 

# BASH 4.2
[[ -v ARRAY[@] ]] && echo "ARRAY is set"   # But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY

# BASH 5.x
[[ ${ARRAY@a} = A ]] && echo "ARRAY is ASSOC ARRAY"  # ... ARRAY is DECLARED, but maybe not SET yet

WHILE / UNTIL Loops

Syntax is 

while list; do list; done
until list; do list; done

Some examples of While loops: 

while true; do
   echo "Press CTRL-C to quit."
done

# Faster alternative using Bash built-in colon feature
while :; do
   echo "Press CTRL-C to quit."
done

# A more complete example
x=0;                                 # initialize x to 0
while [ "$x" -le 10 ]; do
    echo "Current value of x: $x"
    # increment the value of x:
    x=$(expr $x + 1)
    sleep 1
done

Some examples of until loops: 

x=0
until [ "$x" -ge 10 ]; do
    echo "Current value of x: $x"
    x=$(expr $x + 1)
    sleep 1
done

FOR Loops

There are 2 syntax for for loops: 

for name [ in word ] ; do list ; done
for (( expr1 ; expr2 ; expr3 )) ; do list ; done

The normal iteration sequence can be modified with keywords continue [n], break [n].

The first syntax enumerates a list, as in: 

# Counting from 1 to 10...
for dots in 1 2 3 4 5 6 7 8 9 10; do echo -n "$dots... "; done;
echo 'Done !'

# Enumerating a list
for fruit in Apple Pear Cherry; do
  echo "The value of variable fruit is: $fruit"
  sleep 1
done

# Acting on a directory list:
for file in *; do
  echo "Adding .html extension to $file..."
  mv $file $file.html
  sleep 1
done

# On several lines
for f in $(seq 33296 33330); do
  wget "http://i.techrepublic.com.com/gallery/${f}.jpg"
done

# On a single line
for i in `seq 278 328`; do wget http://i.techrepublic.com.com/gallery/33$i.jpg; done

Looping on positional parameters

Several solutions:

  • for i in "$@" (note the quotes), explicitly refering to $@ as source list, and which assign i to each positional parameter.
  • for i, which Bash interprets as an implicit in "$@").
  • Use a while [ $# -gt 0 ] loop and shift (note: testing [ -n "$1" ] is flawed since some param might be empty)
#Illustrate correct and incorrect ways to loop through positional parameters
echo There are $# parameters
echo -n 'for i in $*  : '; for i in $*         ; do echo -n '('$i') '; done; echo
echo -n 'for i in $@  : '; for i in $@         ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$*": '; for i in "$*"       ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$@": '; for i in "$@"       ; do echo -n '('$i') '; done; echo
echo -n 'for i        : '; for i               ; do echo -n '('$i') '; done; echo
echo -n 'while loop   : '; while [ $# -gt 0 ]; do echo -n '('$1') '; shift; done; echo

% loop one "or two"
There are 2 parameters
for i in $*  : (one) (or) (two)
for i in $@  : (one) (or) (two)
for i in "$*": (one or two)
for i in "$@": (one) (or two)
for i        : (one) (or two)
while loop   : (one) (or two)

Looping on file LINEs

The simplest, using read: 

# Reading line per line - from a file
while IFS= read -r LINE; do                         # IFS= to keep surrounding blanks, -r to keep backslashes
    echo "$LINE"
done < file                                         # Note: file could be <(command) to pipe from a command

# Reading line per line - command output / stdin
cat file | grep ... |
while IFS read -r LINE; do
    echo "$LINE"                                    # DO NOT ASSIGN VAR HERE, OR IT IS LOST (SUB-PROCESS)!!!
done

# Reading line per line - command output but *not* stdin
while IFS= read -u 3 -r LINE; do
    echo "$LINE"
    read -p -i Y "Do you want to continue? [Y/n]"   # We can still query the user
done 3< <(command)

Or modifying IFS: 

ORIGIFS=$IFS
IFS=$'\r'
for LINE in $(command); do
    echo "$LINE"
done
IFS=$ORIGIFS

case ... in ... esac

The syntax is 

case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac

Some example 

x=5     # initialize x to 5
# now check the value of x:
case $x in
   0) echo "Value of x is 0."
      ;;
   5) echo "Value of x is 5."
      ;;
   9) echo "Value of x is 9."
      ;;
   *) echo "Unrecognized value."
esac

Functions

Functions are like aliases, but accept parameters: 

function myfunc1() { echo "$2"; echo "$1"; }
myfunc2() { MYVAR="$1"; echo Another $MYVAR function; }       #The keyword ''function'' is optional

Interactivity

Read User's password

Reading input from user is done with the commands read. To prevent password echo on display, one can use the command stty: 

###### Preventing echo with stty ######
read -r -p "Username: " uname
stty -echo
read -r -p "Password: " passw; echo
stty echo

Preventing password echo using option -s: 

###### Using -s ######
PASS="abc123"
read -r -s -p "Password: " mypassword
echo ""
[ "$mypassword" == "$PASS" ] && echo "Password accepted" || echo "Access denied"

Complete solution: 

###### Complete solution ######
USER=wbi\\titeuf
echo "Mounting windows share... Please type password for user $USER..."
# time-out after 60sec, raw input no escaping, no echo, prompt
read -t 60 -r -s -p "Password: " PASSWORD
# delete prompt line
echo -e -n "\r"
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/C$ /mnt/c
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/D$ /mnt/d
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/F$ /mnt/f
# delete password variable
PASSWORD=------------------------------------
PASSWORD=abcdefghijklmnopqrstuvwxyz0123456789
PASSWORD=------------------------------------

Beware of pitfalls with read and piping.

Signal trapping

Use trap to trap signals send to a Bash script, and redirect execution to a given function/command. 

#Trap Ctrl-C (signal SIGINT) by executing function "sorry"
trap sorry INT
sorry() { echo "I'm sorry Dave. I can't do that."; sleep 3; }

Signals can also be ignored or reset. 

#reset the trap:
trap - INT
#do nothing when SIGINT is caught:
trap " INT

Signals can be very handy. See for instance the example on how to use ALRM signal to kill a process

Note that signal table is preserved by exec but not by fork

To force execution of a routine at exit, trap EXIT or equivalently 0 [13]: 

trap "rm $tmpfile" EXIT

See also Bash tips for more tricks.

Exit Status

Use $? to read the exit status of the last command executed. Value is 0 if last command ran without error, and an integer value between 1-255 on error. The command exit nnn can be used to terminate a script or function and to deliver a given exit status.

One can negate a value with operator !, but don't forget the space! 

true    # The "true" builtin.
! true  # Negating true
true
!true   # The '!' operator prefixing a command invokes the Bash history mechanism. It just repeats the previous command.

Some examples to test the exit status: 

if [ $? -eq 0 ]; then echo "exit status is 0"; else echo "exit status is non-zero"; fi

SUCCESS=0
grep -q "$word" "$filename"    #  "q" - quiet mode
if [ $? -eq $SUCCESS ]; then echo "$word was found"; else echo "not found"; fi

if ( ps -as | grep -q "ssh-agent" ); then echo ssh-agent started; else echo ssh-agent not started; fi

TCP/UDP devices

Bash creates special TCP/UDP devices (/dev/tcp/host/port and /dev/udp/host/port) that can be used to easily establish TCP/UDP connections in Bash scripts.

Some examples from [14]:

  • Getting time from NIST:
cat </dev/tcp/time.nist.gov/13
# 53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) *
  • Downloading an URL from a web server:
exec 5<>/dev/tcp/www.net.cn/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5
  • Send email with Bash, see SMTP.

Miscellaneous

Use eval to execute the output (stdout) of a process 
eval `dircolors`
Use $$ as file name suffix or prefix to create unique name for temporary file.
$$ is actually process PID. 
touch /tmp/mytmp.$$
echo some text>/tmp/mytmp.$$
cat /tmp/mytmp.$$

Bug

  • (version 4.1-2ubuntu3) Typo in manpage. menu-complete-backward is written menu-complete-krd. In the man-page, this is because of misplaced \ (menu\-complete-\backward instead of menu\-complete\-backward. Filed a bug report using bashbug
Retrieved from "https://miki.immie.org/mediawiki/index.php?title=Bash&oldid=17673"