Bash: Difference between revisions

From miki
Jump to navigation Jump to search
(→‎Warning / Common pitfalls: removed. moved to Bash hints & tips)
 
(138 intermediate revisions by the same user not shown)
Line 1: Line 1:
==Related pages==
Other pages dedicated to ''Bash'' on this wiki:
* [[Bash Tips and Pitfalls]]
* [[Linux Commands]]

==References==
==References==
External references:
* [http://linux.die.net/man/1/bash Bash Manual Page]
* [http://linux.die.net/man/1/bash Bash Manual Page]
* [http://www.gnu.org/software/bash/manual/bashref.html Bash Reference Manual]
* [http://www.gnu.org/software/bash/manual/bashref.html Bash Reference Manual]
Line 5: Line 11:
* http://www.ss64.com/bash/index.html (find Linux shell command if you know Windows shell command, and vice-versa)
* http://www.ss64.com/bash/index.html (find Linux shell command if you know Windows shell command, and vice-versa)
* http://tille.xalasys.com/training/bash
* http://tille.xalasys.com/training/bash
* Use command '''<tt>help</tt>''', to get help on built-in command:

==Shell==

=== Commands ===
Listing directory content:
<source lang="bash">
<source lang="bash">
help getopts
ls --full-time # To list files full date & time
help test
ls -d directory-name # List directory entries, not content
help if
# eg. ls -d /etc/rc*
ls -lS | head # List biggest files first - print only first 10 entries
ls -lt | head -n 20 # List most recent files first, print only 20 first entries
</source>
</source>
* ''Introduction to Bash Shell Scripting'': [http://www.filibeto.org/sun/lib/development/shell/intr_to_bash_scr.html], or with better formatting [http://www.justlinux.com/nhf/Programming/Introduction_to_bash_Shell_Scripting.html].
* [http://tldp.org/LDP/abs/html/index.html Advanced Bash-Scripting Guide - An in-depth exploration of the art of shell scripting]
* [http://www.davidpashley.com/articles/writing-robust-shell-scripts.html#id2326620 Writing Robust Bash Shell Scripts]

About alternatives to Bash:
* [https://www.arp242.net/why-zsh.html s/bash/zsh/g]

==Shell==


=== Keyboard shortcuts ===
=== Keyboard shortcuts ===
Some of the most used keyboard shortcuts in ''Bash'' (excerpt from [http://linux.die.net/man/1/bash the manual pages]).
Some of the most used keyboard shortcuts in ''Bash'' (excerpt from [http://linux.die.net/man/1/bash the manual pages]).
<br/>'''C-''' refers to ''Ctrl'' key, '''M-''' refers to ''Meta'' key (typ. ''Alt''+key, or ''Esc'' then key), '''S-''' refers to ''Shift'' key.
<br/>'''C-''' refers to ''Ctrl'' key, '''M-''' refers to ''Meta'' key (typ. ''Alt''+key, or ''Esc'' then key), '''S-''' refers to ''Shift'' key.
<br/>'''\e''' introduces an escape character (same as <font color="blue">'''^['''</font> printed by '''verbatim''' command (Ctrl-V) in '''vim''')


Customized keyboard shortcuts described below can be obtained by adding the following lines to e.g. your <tt>~/.inputrc</tt> file:
Customized keyboard shortcuts described below can be obtained by adding the following lines to e.g. your <tt>~/.inputrc</tt> file:
Line 31: Line 40:
"\C- ":dynamic-complete-history #Ctrl-space
"\C- ":dynamic-complete-history #Ctrl-space
"\e ":dynamic-complete-history #Esc-space
"\e ":dynamic-complete-history #Esc-space
"\e[2~":paste-from-clipboard # Insert
"\e[2~":paste-from-clipboard #Insert
tab:menu-complete #TAB
"\e[Z":menu-complete-backward #Shift-TAB
"\C-y":menu-complete-backward #Ctrl-Y
</source>
</source>


Line 37: Line 49:
:Binding by defaults are defined in file <tt>~/.inputrc</tt>. Alternatively they can also be defined in the start file <tt>~/.bashrc</tt> using command '''bind'''.
:Binding by defaults are defined in file <tt>~/.inputrc</tt>. Alternatively they can also be defined in the start file <tt>~/.bashrc</tt> using command '''bind'''.
:They are different method to define the binding keys
:They are different method to define the binding keys
:* Specifying the key '''Keyname''', without double quotes (but always within single quotes)
:* Specifying the key '''Keyname''', without double quotes (but always within single quotes when using command <code>bind</code>)
:* Specifying the key '''Keyseq''', with double quotes (and single quotes)
:* Specifying the key '''Keyseq''', with double quotes ('''*and*''' single quotes when using command <code>bind</code>)
:* Specifying the key '''Verbatim''', with double quotes, and using ''control-V'' followed by key in ''Bash'' or ''Vi''. ''Note: This is a good trick to know the key sequence of special keys like arrow key''.
:* Specifying the key '''Verbatim''', with double quotes, and using ''control-V'' followed by key in ''Bash'' or ''Vi''. ''Note: This is a good trick to know the key sequence of special keys like arrow key''.
:The binding ''keyseq'' differs depending on whether ''Readline'' is configured to support 8-bit input/output characters. ''Cygwin'' and ''openSUSE'' have different settings regarding this. To support 8-bit input/output (like in ''OpenSUSE''), add the following to the <tt>~/.inputrc</tt>:
:The binding ''keyseq'' differs depending on whether ''Readline'' is configured to support 8-bit input/output characters. ''Cygwin'' and ''openSUSE'' have different settings regarding this. To support 8-bit input/output (like in ''OpenSUSE''), add the following to the <tt>~/.inputrc</tt>:
<source lang="text">
<pre>
set convert-meta off
set convert-meta off
set input-meta on
set input-meta on
set meta-flag on
set meta-flag on
set output-meta on
set output-meta on
</pre>
</source>


<source lang="bash">
<source lang="bash">
Line 52: Line 64:
bind '"\C-i":dynamic-complete-history' #Keyseq
bind '"\C-i":dynamic-complete-history' #Keyseq
bind '"<Control-V><Tab>:dynamic-complete-history' #Verbatim - <> means press the given key while editing
bind '"<Control-V><Tab>:dynamic-complete-history' #Verbatim - <> means press the given key while editing
</source>

See also
<source lang="bash">man readline</source>

;Using negative argument in readline command
:[http://www.mail-archive.com/redhat-list@redhat.com/msg129032.html], [http://docs.freebsd.org/info/readline/readline.info.Readline_Arguments.html]. Examples:
<source lang="bash">
"\e[Z":"\M--1\t" # \M--1 is actually-1 argument for next command , \t i.e. TAB
"\e[Z":"\M--\t" # We can skip the 1
"\e[Z":"\e--\C-i" # Equivalent, but using \e instead of \M, and \C-i instead of \t
</source>
</source>


Line 65: Line 88:


==== Commands for Manipulating the History ====
==== Commands for Manipulating the History ====
;reverse-search-history (C-r)
;{{blue|reverse-search-history (C-r)}}
:Search backward starting at the current line and moving 'up' through the history as necessary. This is an incremental search.
:Search backward starting at the current line and moving 'up' through the history as necessary. This is an incremental search.
:''{{blue|This is equivalent to interactive grep in history &mdash; very powerful}}''
;forward-search-history (C-s)
;forward-search-history (C-s)
:Search forward starting at the current line and moving 'down' through the history as necessary. This is an incremental search.
:Search forward starting at the current line and moving 'down' through the history as necessary. This is an incremental search.
Line 82: Line 106:
:'''!!! AZERTY keyboard: press Alt-^ twice'''
:'''!!! AZERTY keyboard: press Alt-^ twice'''
;operate-and-get-next (C-o)
;operate-and-get-next (C-o)
:Accept the current line for execution and fetch the next line relative to the current line from the history for editing. Any argument is ignored.
:Accept the current line for execution and fetch the next line relative to the current line from the history for editing. Any argument is ignored.
;{{blue|edit-and-execute-command (C-xC-e)}}
:Invoke an editor on the current command line, and execute the result as shell commands. Bash attempts to invoke $VISUAL, $EDITOR, and emacs as the editor, in that order.


==== Commands for Changing Text ====
==== Commands for Changing Text ====
Line 121: Line 147:


==== Completing ====
==== Completing ====
;complete (TAB)
;complete (<font color="red"><s>TAB</s></font>)
:Attempt to perform completion on the text before point. Bash attempts completion treating the text as a variable (if the text begins with '''$'''), username (if the text begins with '''~'''), hostname (if the text begins with '''@'''), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted.
:Attempt to perform completion on the text before point. Bash attempts completion treating the text as a variable (if the text begins with '''$'''), username (if the text begins with '''~'''), hostname (if the text begins with '''@'''), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted.
;menu-complete (<font color="red">TAB</font>) / menu-complete-backward (<font color="red">Shift-TAB, C-Y</font>)
:Similar to complete but replaces the word with next (previous) completion.
:<font color="red">''!!! Apparently Shift-Tab doesn't work in Cygwin. Use '''Ctrl-Y''' instead''</font>
;possible-completions (M-?)
;possible-completions (M-?)
:List the possible completions of the text before point.
:List the possible completions of the text before point.
Line 148: Line 177:
<source lang="bash">
<source lang="bash">
echo 12 34
echo 12 34
!! #execute previous command
!! # Execute previous command
!1 #execute command n°1
!1 # Execute command n°1
!-2 #execute penultimate command
!-2 # Execute penultimate command
echo 12 34
echo 12 34
^12^56^ #execute previous command, replacing 12 with 56. Quick for !!:s/12/56/
^12^56^ # Execute previous command, replacing 12 with 56. Quick for !!:s/12/56/
!!<space> #Example of magic space
!!<space> # Example of magic space
ech !-2<space> #... also works anywhere in the current line
echo !-2<space> # ... also works anywhere in the current line
for i in <C-xC-e> # Shortcut to edit cmd in editor
fc # Edit previos command in editor
</source>
</source>


==Script Quick Tutorial==
==Script Quick Tutorial==


===Invocation===


A ''bash'' shell can be (see <code>man bash</code>)
* either '''interactive''' or '''not interactive'''
:The shell is '''interactive''' when started with the <code>-i</code> option, or when not started without non-option arguments and without the <code>-c</code> option, and whose stdin/stderr are both connected to terminals.
* either a '''login shell''' or '''not a login shell'''
:A '''login shell''' is one whose first character of argument zero is a <code>-</code> (i.e. <code>$0 == -bash</code> or such), or one started with the <code>--login</code> or <code>-l</code> option.
:Using '''<code>su</code>''' a login shell is started with option <code>-</code>, <code>-l</code> or <code>--login</code>.

So
<source lang="bash">
su - username # LOGIN and INTERACTIVE
bash --login
bash -l
su username # NOT login and INTERACTIVE
bash
bash -i
su - username -c "..." # LOGIN and NOT interactive
bash --login -c "..."
su username -c "..." # NOT login and NOT interactive
bash -c "..."
bash -c "..." --login # idem (actually --login is passed to -c commad)
</source>
* Use command '''shopt''' to know whether current shell is interactive / login.
* Interactive shells have '''i''' defined in '''<tt>$-</tt>''', and also define variable '''PS1'''.
* Sourcing file can be skipped / enforced with <code>--noprofile</code>, <code>--norc</code> or <code>--rcfile</code>.

A simple script to detect the current invocation [https://unix.stackexchange.com/questions/26676/how-to-check-if-a-shell-is-login-interactive-batch]:
<source lang="bash">
[[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'
shopt -q login_shell && echo 'Login shell' || echo 'Not login shell'
</source>

The following files are sourced by '''bash''' at invocation

{| class=wikitable
|-
! || NOT login || LOGIN
|-
! INTERACTIVE
|
* First {{file|/etc/bash.bashrc}}.
* Then {{file|~/.bashrc}}
|rowspan=2|
* First {{file|/etc/profile}}.
* Then either {{file|~/.bash_profile}}, {{file|~/.bash_login}}, {{file|~/.profile}}, whichever is found first.
|-
! NOT interactive
|
None
|}
But on Ubuntu, {{file|/etc/profile}} automatically sources {{file|/etc/bash.bashrc}}, which then aborts if it detects that the shell is not interactive. Also {{file|~/.profile}} sources {{file|~/.bashrc}}, which also aborts if non-interactive. So '''in an out-of-the-box configuration''', the following files are sourced:

{| class=wikitable
|-
! || NOT login || LOGIN
|-
! INTERACTIVE
|
{{file|/etc/bash.bashrc}}<br/>
{{file|~/.bashrc}}
|
{{file|/etc/profile}}<br/>
{{file|/etc/bash.bashrc}}<br/>
{{file|~/.profile}}<br/>
{{file|~/.bashrc}}
|-
! NOT interactive
|
None
|
{{file|/etc/profile}}<br/>
{{file|~/.profile}}
|}


===Environment===
===Environment===
Line 169: Line 273:
</source>
</source>


As a rule of thumb, better define environment variables (PATH, etc) in '''~/.profile''', so that they are available to all program launched from a login shell (see [http://superuser.com/questions/183845/which-setup-files-should-be-used-for-setting-up-environment-variables-with-bash/183956], [http://superuser.com/questions/183870/difference-between-bashrc-and-bash-profile/183980]).
===Quotation===

===Quoting===
{| class="wikitable"
{| class="wikitable"
|-
|-
|'''Escape \'''
|<source lang="bash" enclose="valid">echo "var is $var"</source>
|
|String enclosed in double quote is treated as a '''single''' argument. Variable and special characters are expanded.
{{nb|1=<source lang="bash">cp file \'file\$\'</source>}}
|Preserve literal value of the next character
|-
|-
|'''Single quote '...' '''
|<source lang="bash" enclose="valid">echo 'it costs $10'</source>
|
|Same as double quote, but variables are not expanded.
{{nb|1=<source lang="bash">echo 'it cost $10'</source>}}
|Preserve literal value of all characters
|-
|-
|'''Dollar Single quote '...' '''
|<source lang="bash" enclose="valid">x=`expr $x + 1`</source>
|
|Back-quote is replaced by the result of the invoked command.
{{nb|1=<source lang="bash">echo $'it cost \'$10\''</source>}}
|Preserve literal value of all characters, except escaped character
|-
|-
|'''Double quote "..."' '''
|<source lang="bash" enclose="valid">x=$(expr $x + 1)</source>
|
|Same as back-quote, but somewhat clearer
{{nb|1=<source lang="bash">cp file "file $var"</source>}}
|Preserve literal value of all characters, except <tt>'''$ ` \ !'''</tt>
|-
|-
|'''Dollar Double quote $"..."' '''
|<source lang="bash" enclose="valid">x=$(($x + 1))</source>
|
|Built-in Bash expression evaluator. Accepts operation are <tt>+, -, *, /, %</tt>.
{{nb|1=<source lang="bash">echo $"file not found"</source>}}
|Translate string to current locale (for internationalization
|}

===Compound Command===
{| class="wikitable"
|-
|'''(list)'''
|
{{nb|1=<source lang="bash">if ( kill $pid; ps -p $pid )</source>}}
|Execute list in a subshell and set return status to last command returned status
|-
|'''{ list; }'''
|
{{nb|1=<source lang="bash">{ read v1; read v2; } <foo</source>}}
|Group command &mdash; Execute list in current environment. Note the <tt>;</tt> and ''whitespaces''!
|-
|'''((expr))'''
|
{{nb|1=<source lang="bash"> if ((i<10)); ...</source>}}
|Arithmetic evalution. Set return status accordingly.
|-
|'''<nowiki>[[ expr ]]</nowiki>'''
|
{{nb|1=<source lang="bash"> if [[ -e foo && -e bar ]]; ...</source>}}
|Condition expression. Set return status accordingly.
|}
|}


===Variable Expansion===
=== Expansion and Substitution ===
This is only a subset of the expansion possibilities. See the man pages for more information
{| class="wikitable"
{| class="wikitable"
|-
|-
|'''<tt>${!param}</tt>'''
|<source lang="bash" enclose="valid">
|{{nb|1=<source lang="bash">
% VAR=myvar.pattern && echo ${VAR%%.pattern}
var=hello; ref=var;
myvar
echo ${!ref} # "hello"
foo[0]=foo0; foo[1]=foo1
ref=foo[0] # ... also for arrays!
echo ${!ref} # "foo0"
</source>
</source>
}}
|Use '''<tt>${i%%pattern}</tt>''' to delete given trailing pattern. Use a single % for shortest matching pattern.<br/>''memo: % at the end as in '''100%'''''.
|'''Indirect expansion''' &mdash; Equivalent to <code>eval \$$ref</code>, but cleaner and safer. It also works for '''arrays'''!
|-
|-
|'''<tt>${param%word}<br/>${param%%word}</tt>'''
|<source lang="bash" enclose="valid">
|{{nb|1=<source lang="bash">
% VAR=pattern.myvar && echo ${VAR##pattern.}
var1=file.ext; var2=dir//;
myvar
echo ${var1%%.ext} # "file"
echo ${var2%%/} # "dir"
</source>
</source>
}}
|Use '''<tt>${i##pattern}</tt>''' to delete given heading pattern. Use a single # for shortest matching pattern.<br/>''memo: # at the beginning as in '''#comment'''''.
|'''Parameter expansion''' &mdash; Suffix removal. Use a single % for shortest match.<br/>''Memo: '''100%''' &rarr; suffix''.
|-
|-
|'''<tt>${param#word}<br/>${param##word}</tt>'''
|<source lang="bash" enclose="valid">
|{{nb|1=<source lang="bash">
% VAR=some.find.text && echo ${VAR/find/replace}
var=__file
some.replace.text
echo ${var##__} # "file"
</source>}}
|'''Parameter expansion''' &mdash; Prefix removal. Use a single # for shortest match.<br/>''memo: '''#comment''' &rarr; prefix''.
|-
|'''<tt>${param/pat/str}<br/>${param//pat/str}</tt>'''
|{{nb|1=<source lang="bash">
var=sometext
echo ${var/e/_} # "som_text"
</source>}}
|'''Parameter expansion''' &mdash; Search & replace. Use <tt>/pattern</tt> to replace all matches
|-
|'''<tt>$(command)<br/>`command`</tt>'''
|{{nb|1=<source lang="bash">echo $(basename $0)</source>}}
|Command substitution &mdash; Expands to the output of command<br/>Trailing newlines are {{red|stripped!}}
|-
|'''<tt>$((expr))<br/>$[expr]</tt>'''
|{{nb|1=<source lang="bash">echo $((16 * 16))</source>}}
|'''Arithmetic expansion''' &mdash; Expands to the result of expression (<tt>$[...]</tt> is deprecated)
|-
|'''<tt>&lt;(command)<br/>&gt;(command)</tt>'''
|{{nb|1=<source lang="bash">diff <(ls dir1) <(ls dir2)</source>}}
|'''Process substitution''' &mdash; Substitute a process with a named pipe connected to the process (see [[Bash Tips and Pitfalls#Using Process Substitution|tips]])
|-
|'''<tt>* ? [...]</tt>'''
|{{nb|1=<source lang="bash">echo *</source>}}
|'''Pathname expansion''' &mdash; Expands to file names matching given pattern (done after word splitting)
|-
|'''<tt>{0..19}</tt>'''
|{{nb|1=<source lang="bash">
echo {0..9}
# 0 1 2 3 4 5 6 7 8 9</source>}}
|'''Sequence expansion'''
|}

Here summary of expansion result depending on whether <code>parameter</code> is '''unset''', '''set''' and/or '''null''':
{| class=wikitable
|-
|
| '''set''' and '''not null'''
| '''null'''
| '''unset'''
|-
! <code>${parameter:-word}</code>
| substitute <code>parameter</code> || substitute <code>word</code> || substitute <code>word</code>
|-
! <code>${parameter-word} </code>
| substitute <code>parameter</code> || substitute null || substitute <code>word</code>
|-
! <code>${parameter:=word}</code>
| substitute <code>parameter</code> || assign <code>word</code> || assign <code>word</code>
|-
! <code>${parameter=word} </code>
| substitute <code>parameter</code> || substitute null || assign <code>word</code>
|-
! <code>${parameter:?word}</code>
| substitute <code>parameter</code> || error, exit || error, exit
|-
! <code>${parameter?word} </code>
| substitute <code>parameter</code> || substitute null || error, exit
|-
! <code>${parameter:+word}</code>
| substitute <code>word</code> || substitute null || substitute null
|-
! <code>${parameter+word} </code>
| substitute <code>word</code> || substitute <code>word</code> || substitute null
|}

===Positional Parameters===

See [[#Looping on positional parameters|section "Looping on positional parameters"]] for an illustration on how to process each parameters in a <code>for</code> loop.

{| class="wikitable"
|-
|{{nb|1=<source lang="bash">
# !/bin/bash

function parsing ()
{
echo "($1) ($2) ($3) ($4)"
}

echo -n 'parsing $* -- '; parsing $*
echo -n 'parsing $@ -- '; parsing $@
echo -n 'parsing "$*" -- '; parsing "$*"
echo -n 'parsing "$@" -- '; parsing "$@"
# Echo number of parameters
echo $#
</source>}}
|{{nb|1=<source lang="bash">
$ ./test.sh p1 p2 "p 3" 'p 4'
# parsing $* -- (p1) (p2) (p) (3)
# parsing $@ -- (p1) (p2) (p) (3)
# parsing "$*" -- (p1 p2 p 3 p 4) () () ()
# parsing "$@" -- (p1) (p2) (p 3) (p 4)
# 4
</source>}}
|}

To test if there are parameters:
<source lang=bash>
if [ $# -eq 0 ]; then
echo No parameters!
exit 1
fi
</source>
</source>

|Use '''<tt>${i/pattern/text}</tt>''' to replace given pattern with given text.
=== Operators ===
{| class=wikitable
|-
|VAR+=value
|
if ''VAR'' is a string, append ''value''<br/>
if ''VAR'' is an array, add element ''value''<br/>
if ''VAR is a numerical value, add ''value''
|{{nb|1=<source lang=bash>
STRING=
STRING+=" foo" # Now string is "foo" (w/o space!)
STRING+=" bar" # Now string is "foo bar"
</source>}}
|}

=== String pattern matching ===
Several possibilities to do pattern matching in strings:
* Using operator <code>=~</code> and regular expression
* Using operator <code>==</code> and Bash pattern
* Using <code>case</code>

Using <code>case</code> is the fastest option for simple, single patterns (10% faster than <code>==</code>). <code>==</code> is faster than <code>=~</code> (for instance <code>[[ fooMATCHbar == *MATCH* ]]</code> is roughly 3x faster than <code>[[ fooMATCHbar =~ .*MATCH.* ]]</code>).

;Examples
<source lang="bash">
if [[ fooMATCHbar =~ .*MATCH.* ]]; then echo found; done
if [[ fooMATCHbar == *MATCH* ]]; then echo found; done
case fooMATCHbar in *MATCH*) echo found;; esac
</source>

===Redirection - Descriptors, Here Documents and Here Strings===

{| class=wikitable
|-
! &&gt;word<br/> &gt;&word
| Redirect '''both''' stdout and stderr (semantically equivalent to '''>word 2>&1''')
|-
! &#124;&
| Pipe '''both''' stdout and stderr
|}
|}


===Redirection - Here Documents and Here Strings===


{| class="wikitable"
{| class="wikitable"
Line 217: Line 508:
! Source
! Source
! Output
! Output
|-
|width=50%|'''Descriptors''' - redirect input / output to files or file descriptor. These file names have special meaning: <tt>/dev/fd/fd</tt> (with 2nd fd an integer), <tt>/dev/stdin</tt>, <tt>/dev/stdout</tt>, <tt>/dev/stderr</tt>, <tt>/dev/tcp/host/port</tt>, <tt>/dev/udp/host/port</tt><br/> See [[Bash Tips and Pitfalls#Using Process Substitution|Using Process Substitution]] tip and [[Bash Tips and Pitfalls#Redirecting stdout and stderr with tee and a pipe|stderr redirection]] tip for more advanced redirection.
|{{nb|1=<source lang="bash">
ls > /dev/stderr # WRONG
ls >> /dev/stderr # BETTER
ls >&2 # BEST
ls 2>&1 > dirlist # WRONG
ls > dirlist 2>&1 # OK
ls >& dirlist # BETTER
ls &> dirlist # BETTER (PREFERRED)
./myapp |& tee foo # Pipe stdout & stderr
</source>}}
|-
|-
|'''Here Documents''' - Use <tt>&lt;&lt;word</tt> to declare here documents.<br/>Check [http://www.gnu.org/software/bash/manual/bashref.html#Redirections BASH manpage] for more information.
|'''Here Documents''' - Use <tt>&lt;&lt;word</tt> to declare here documents.<br/>Check [http://www.gnu.org/software/bash/manual/bashref.html#Redirections BASH manpage] for more information.

|<source lang="bash">
We can also combine several here documents on the same line, or use pipelines [https://stackoverflow.com/questions/7046381/multiline-syntax-for-piping-a-heredoc-is-this-portable]
|{{nb|1=<source lang="bash">
cat <<__EOF__
cat <<__EOF__
First line
First line
Line 225: Line 530:
Third line
Third line
__EOF__
__EOF__
</source>}}

<source lang="bash">
# here document with pipeline
cat <<__EOF__ |
First line
Second line
Third line
__EOF__
tr [a-z] [A-Z]
</source>

<source lang="bash">
# Multi here documents
cat <<eof1; cat <<eof2
Hi,
eof1
Helene.
eof2
</source>
</source>
|
|
Line 232: Line 556:
Third line
Third line
</pre>
</pre>

<pre>
FIRST LINE
SECOND LINE
THIRD LINE
</pre>

<pre>
Hi,
Helene.
</pre>

|-
|-
|'''Here Documents''' - Use <tt>&lt;&lt;-</tt> to strip leading tab characters.
|'''Here Documents''' - Use <tt>&lt;&lt;-</tt> to strip leading tab characters.
|<source lang="bash">
|{{nb|1=<source lang="bash">
cat <<-__EOF__
cat <<-__EOF__
First line
First line
Line 240: Line 576:
Third line
Third line
__EOF__
__EOF__
</source>
</source>}}
|
|
<pre>
<pre>
Line 249: Line 585:
|-
|-
|'''Here Strings''' - Use <tt>&lt;&lt;&lt;word</tt> to declare here strings.<br/>
|'''Here Strings''' - Use <tt>&lt;&lt;&lt;word</tt> to declare here strings.<br/>
|<source lang="bash">% cat <<<'Hello, World!'
|{{nb|1=<source lang="bash">
% TEXT='Hello, World!' cat <<<$TEXT
cat <<<'Hello, World!'
TEXT='Hello, World!' cat <<<$TEXT
% cat<<<"First line
cat<<<"First line
> Second line
> Second line
> Third line"
> Third line"
</source>
</source>}}
|
|
<pre>Hello, World!
<pre>Hello, World!
Line 265: Line 602:


===If ... then ... [elif ...] ... else ... fi===
===If ... then ... [elif ...] ... else ... fi===
(See [http://tldp.org/LDP/abs/html/testconstructs.html] for in-depth discussion on Bash tests)

The basic syntax for if-then-else sequence control is
The basic syntax for if-then-else sequence control is
'''if''' list; '''then''' list; ''[ '''elif''' list; '''then''' list; ]'' ... ''[ '''else''' list; ]'' '''fi'''
'''if''' list; '''then''' list; ''[ '''elif''' list; '''then''' list; ]'' ... ''[ '''else''' list; ]'' '''fi'''
The ''if/then'' construct tests whether the exit status of a list of command is 0 (i.e. success), and if so, executes another list of command. Test conditions can be written using shell ''built-in'' command '''<tt>test</tt>''', or equivalently the square brackets '''<tt>[</tt>'''...'''<tt>]</tt>'''. Other possibilities are using the ''extended test command'' '''<tt>[[</tt>'''...'''<tt>]]</tt>''' (that performs parameter expansion and command substitution as well as arithmetic evaluation, see [http://tldp.org/LDP/abs/html/testconstructs.html#DBLBRACKETS]), or the ''double-parens'' '''<tt>((</tt>'''...'''<tt>))</tt>''' expression (see [http://tldp.org/LDP/abs/html/dblparens.html]) that evaluates an arithmetic expression (returs true if expression is non-zero).
Test conditions can be written using command <tt>test</tt>, or equivalently using the ''square brackets''.


<source lang="bash">
<source lang="bash">
if test -f /etc/foo; then echo file found; else echo file NOT found; fi
if mv $src $dst; then echo move is ok; else move is not ok; fi # Using list
if [ -f /etc/foo ]; then echo file found; else echo file NOT foudn; fi
if test -f /etc/foo; then echo file found; else echo file NOT found; fi # test command
if [ -f /etc/foo ]; then echo file found; else echo file NOT found; fi # [ is a synomym for test and a builtin
if [[ -f $file ]]; then echo file $file found; else echo file NOT found; fi # Extended test construct - &&, ||, < and > works
if (( 128 && 64 )); then echo result is non-zero; else echo result is zero; fi # Expands and evaluates arithmetic expression
</source>
</source>


Here's <tt>test</tt>'s options:
Here's <tt>test</tt>'s options (more info with '''<tt>help test</tt>'''):
<source lang="bash">
<source lang="bash">
-d FILE # Check if the file is a directory
-d FILE # Check if the file is a directory
Line 293: Line 635:
STR1 = STR2 # Check if STR1 is the same as STR2
STR1 = STR2 # Check if STR1 is the same as STR2
STR1 != STR2 # Check if STR1 is not the same as STR2
STR1 != STR2 # Check if STR1 is not the same as STR2
-n STR # Evaluates to true if STR is not null
-n STR # Evaluates to true if STR is not null (can be used to test whether a variable is defined)
-z STR # Evaluates to true if STR is null.
-z STR # Evaluates to true if STR is null.
</source>
</source>


Conditions can be combined with <tt>''&&''</tt> and <tt>''||''</tt>:
Conditions can be combined with <code>'''&&'''</code> and <code>'''||'''</code> or with <code>'''test'''</code> operator <code>'''-a'''</code> and <code>'''-o'''</code>. Use <code>'''!'''</code> to negate a test:
<source lang="bash">
<source lang="bash">
if [ $x -ge 5] && [ $x -le 10]; then ...
if [ $x -ge 5 ] && [ $x -le 10 ]; then ...; fi
if [ $x -gt 5] || [ $x -lt 10]; then ...
if [ $x -gt 5 ] || [ $x -lt 10 ]; then ...; fi
if [ $x -ge 5 -a $x -le 10 ]; then ...; fi
if [ $x -gt 5 -o $x -lt 10 ]; then ...; fi
if ! [ -a /some/file.txt ]; then ...; fi #negation
if [ -n "$DEFINED" ]; then echo DEFINED is defined; fi
if [ "$DEFINED" ]; then echo DEFINED is defined; fi #also works
</source>
</source>


Line 314: Line 661:
<source lang="bash">
<source lang="bash">
[ -f /etc/hosts ] && { echo one; echo two; } || { echo three; echo four; }
[ -f /etc/hosts ] && { echo one; echo two; } || { echo three; echo four; }
</source>

==== Using <code><nowiki>[[</nowiki>''expression''<nowiki>]]</nowiki></code> ====
The conditinal expressions <code><nowiki>[[</nowiki>''expression''<nowiki>]]</nowiki></code> uses the same ''CONDITIONAL EXPRESSIONS'' as the regular <code>[ ... ]</code>, but in addition accept the following construct (see manual pages for more):

{| class="wikitable"
|-
|
'''value == pattern'''<br/>
'''value != pattern'''
|
{{nb|1=<source lang="bash">if [[ $NAME == *.c ]]; then echo C Source file; fi</source>}}
|String ''Pattern'' is considered a '''shell pattern'''
|-
|
'''value =~ pattern'''
|
{{nb|1=<source lang="bash">if [[ $(ps $PID) =~ ssh-agent ]]; then echo $PID is ssh-agent pid; fi</source>}}
|String ''Pattern'' is considered an '''extended regular regex'''
|}

Thanks to these, there is no need for constructs using '''grep''':
<source lang=bash>
if ps aux | grep ssh-agent; then echo ssh-agent found; fi # NOT EFFICIENT, 2 processes spawn

if [[ $(ps aux) =~ ssh-agent ]]; then echo ssh-agent foudn; fi # BETTER!!!
</source>

=== Strings ===
Reference: [http://tldp.org/LDP/abs/html/string-manipulation.html Advanced Bash Scripting Guide]

<source lang=bash>
len=${#line} # Get length of string line
LANG=C LC_ALL=C
bytlen=${#line} # Get *byte* length

stringZ=abcABC123ABCabc # 0-based indexing
echo ${stringZ:0} # abcABC123ABCabc
echo ${stringZ:1} # bcABC123ABCabc
echo ${stringZ:7} # 23ABCabc

echo ${stringZ:7:3} # 23A

# Using negative index
echo ${stringZ:-4} # abcABC123ABCabc - FAIL! because construct ${parameter:-default}.
echo ${stringZ:(-4)} # Cabc - SUCCESS!
echo ${stringZ: -4} # Cabc - SUCCESS!
</source>

=== Arrays ===
See [http://tldp.org/LDP/abs/html/arrays.html] for more detailed information.

<source lang="bash">
# Assigning array elements
area[11]=23 # Array members need not be consecutive or contiguous.
area[13]=37
area[51]=UFOs

# APPEND to array
ARRAY=()
ARRAY+=('foo')
ARRAY+=('bar baz')

# LOOP on array element:
for i in ${!ARRAY[*]}; do echo ${ARRAY[$i]}; done

# Use curly brackets to read!
echo "\${area[11]} = ${area[11]}"
echo "\${area[0]} = $area = ${area[0]}"

# Use array to echo first word in a list:
make="/usr/bin/make -r --no-print-directory -j 2"
words=($make)
echo $words

# Store multiline in array
files=($(grep -l foo *))
echo "There are ${#files[@]} matching files, first is ${files[0]}"

# Other ways to assign an array
area2=( zero one two three four ) # zero-based indexing!
area3=([17]=seventeen [24]=twenty-four)

# Use quoting to insert white-spaces
array=( [0]="first element" [1]="second element" [3]="fourth element" )

# read file into array (one line, separated by blank)
read -a array < FILE

# Read file into array (line by line)
readarray -t array < FILE # Bash 4.0
IFS=$'\r\n' read -d '' -r -a array < FILE
IFS=$'\r\n' GLOBIGNORE='*' command eval 'XYZ=($(cat FILE))'
# https://stackoverflow.com/questions/11393817/read-lines-from-a-file-into-a-bash-array
readarray -d '' array < <(find . -name "$input" -print0)
# https://stackoverflow.com/questions/23356779/how-can-i-store-the-find-command-results-as-an-array-in-bash
# See also tips page for more

# Some operators
${array:1} == ${array[0]:1} # Param. expansion of 1st element, starting at pos #1 (2nd char)
${#array} == ${#array[0]} # Length of 1st element
${#array[*]} == ${#array[@]} # Number of elements in array.
unset array[1] # Remove 2nd element of array (but ***leave a HOLE in INDEX***)
array[1]= # ... idem
unset array # Delete entire array

# Re-INDEX an array (e.g. if some element deleted)
foo=(a b c)
declare -p foo # declare -a foo=([0]="a" [1]="b" [2]="c")
unset foo[1]
declare -p foo # declare -a foo=([0]="a" [2]="c")
foo=("${foo[@]}") # Re-indexing
declare -p foo # declare -a foo=([0]="a" [1]="c")

# CROP array
array=("${array[@]:1}") # Crop 1st elm and reindex, using substring expansion
unset 'array[0]' # Crop 1st elm, but doesn't reindex
array=("${array[@]}") # ... to force reindex
# Keep only some of the last elements
# ... using a loop
while [ ${#array[@]} -gt 5 ] ; do
array=("${array[@]:1}")
done
# ... using a test
[ ${#array[@]} -gt 5 ] && array=("${array[@]:$((${#array[@]} -5))}")

# Test arrays
[ ${#array[@]} -eq 0 ] && echo "array empty"
[[ -v array[@] ]] && echo "array is set" # BASH 4.2 - But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY
[[ ${array@a} = a ]] && echo "array is ARRAY" # BASH 5.x - ... ARRAY is DECLARED, but maybe not SET yet

# PRINT arrays
echo ${array[@]} # Print elements separated by SPACE
echo ${array[*]} # ... idem
echo "${array[*]}" # ... idem
echo "${array[@]}" # Print elements separated by 1st char in IFS
IFS='/'; echo "${array[*]}" # MUST set IFS before!
IFS='/' echo "${array[*]}" # Does NOT work because echo param got expanded BEFORE IFS assignment

unset array[1]
echo "${array[*]}" # DELETED element are NOT printed

# COPY an array
copy=("${array[@]}")

# INDIRECT access to array element
REF=array[0]
# REF=array[@] -- also works
# REF=!array[@] -- DOES NOT work though (for keys)
echo ${!REF}

# ... better solution since bash 4.3+, use **namerefs**:
declare -A array=( [a]=A [b]=B )
declare -n REF=array # ... or ...
local -n REF=array # Better in functions
echo ${REF[a]} # Element access
echo ${REF[@]} # Return all values
echo ${!REF[@]} # Return all keys

# INDIRECT length of an array or array element (w/o declare -n)
REF=array[@] # Our reference to an array
local current=("${!str}") # Copy into an array using indirect ref

echo ${#current[*]} # Now we can use regular array operator - array size
echo ${current[1]} # Print 2nd element
echo ${#current[1]} # ... and size of 2nd element

# Examine / serialize array
declare -p array # declare -a array=([0]="two" [1]="one")
</source>

=== Associative Arrays ===
Associative arrays (so-called ''hash'' in Perl) are available in Bash since version 4.
* Associative array are declared with <code>declare -A ARRAY</code>
* Values are accessed with <code>$ARRAY[key]</code>
* Keys are accessed with '''exclamation mark''': <code>${!ARRAY[@]}</code>

<source lang=bash>
declare -A ARRAY # Declare 'ARRAY' as a (local) associative array
declare -gA ARRAY # Declare 'ARRAY' as a global associative array
# ... handy when array is constructed in a function
ARRAY[foo]=FOO # Associate value 'FOO' to key 'foo'
ARRAY[bar]=BAR # Associate value 'BAR' to key 'bar'
declare -A ARRAY=( [foo]=FOO [bar]=BAR ) # ... idem, on one line
declare -p ARRAY # declare -A ARRAY=( [foo]=FOO [bar]=BAR )
echo ${ARRAY[foo]} # Print value associated with 'foo'
echo ${ARRAY[*]} # Print all values in ARRAY
echo ${ARRAY[@]} # ... idem
echo ${!ARRAY[*]} # Print all ***keys*** in ARRAY
echo "${ARRAY[*]}" # Same as for "$*"
echo "${ARRAY[@]}" # Same as for "$@"
${#ARRAY[*]} == ${#ARRAY[@]} # Number of elements in ARRAY.
[ ${ARRAY[key]+x} ] && echo "exists" # Check if 'key' exists, based on ${foo+x} returning x if foo set and nothing otherwise
[[ -v ARRAY[key] ]] && echo "exists" # ... alternative (BASH 4.2)
[[ ${ARRAY@a} = A && -v ARRAY[key] ]] # ... SAFER, test ARRAY is ASSOC first (BASH 5.0)
keys=(${ARRAY[@]}) # All keys in a standard array
</source>
To delete an array or element:
<source lang=bash>
unset ARRAY # Empty the ARRAY (ARRAY is NOT ASSOCIATIVE anymore!)
unset ARRAY[foo] # Delete a KEY
unset ARRAY["my key"] # ... quote if KEY contains a space
K="my key"
unset ARRAY["$K"] # Delete a KEY contained in variable
</source>

To CLEAR an associative array in a function, UNSET it then declare it GLOBALLY with <code>-gA</code>. For instance:
<source lang="bash">
unset A
declare -A A
A[C]=2;
a()
{
unset A # Clear the array
declare -gA A # Declare it as ASSOCIATIVE and GLOBAL
A[X]=1
}
a
declare -p A # declare -A A=([X]="1")
</source>

The following test if ARRAY is set:
<source lang="bash">
# BASH 4.2
[[ -v ARRAY[@] ]] && echo "ARRAY is set" # But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY

# BASH 5.x
[[ ${ARRAY@a} = A ]] && echo "ARRAY is ASSOC ARRAY" # ... ARRAY is DECLARED, but maybe not SET yet
</source>
</source>


Line 353: Line 928:


===FOR Loops===
===FOR Loops===

* For loops by examples: http://www.cyberciti.biz/faq/bash-for-loop/

There are 2 syntax for '''for''' loops:
There are 2 syntax for '''for''' loops:
'''for''' name ''[ '''in''' word ]'' ; '''do''' list ; '''done'''
'''for''' name ''[ '''in''' word ]'' ; '''do''' list ; '''done'''
'''for''' (( expr1 ; expr2 ; expr3 )) ; '''do''' list ; '''done'''
'''for''' (( expr1 ; expr2 ; expr3 )) ; '''do''' list ; '''done'''

The normal iteration sequence can be modified with keywords '''continue''' ''[n]'', '''break''' ''[n]''.


The first syntax enumerates a list, as in:
The first syntax enumerates a list, as in:
Line 389: Line 969:
* '''for i in "$@"''' (note the ''quotes''), explicitly refering to $@ as source list, and which assign i to each positional parameter.
* '''for i in "$@"''' (note the ''quotes''), explicitly refering to $@ as source list, and which assign i to each positional parameter.
* '''for i''', which Bash interprets as an implicit <tt>in "$@"</tt>).
* '''for i''', which Bash interprets as an implicit <tt>in "$@"</tt>).
* Use a '''while [ "$1" != "" ]''' loop and '''shift'''
* Use a '''while [ $# -gt 0 ]''' loop and '''shift''' (note: testing <code>[ -n "$1" ]</code> is flawed since some param might be empty)


{|
{|
Line 397: Line 977:
#Illustrate correct and incorrect ways to loop through positional parameters
#Illustrate correct and incorrect ways to loop through positional parameters
echo There are $# parameters
echo There are $# parameters
echo -n 'for i in $* : '; for i in $* ; do echo -n '-'$i'- '; done; echo
echo -n 'for i in $* : '; for i in $* ; do echo -n '('$i') '; done; echo
echo -n 'for i in $@ : '; for i in $@ ; do echo -n '-'$i'- '; done; echo
echo -n 'for i in $@ : '; for i in $@ ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$*": '; for i in "$*" ; do echo -n '-'$i'- '; done; echo
echo -n 'for i in "$*": '; for i in "$*" ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$@": '; for i in "$@" ; do echo -n '-'$i'- '; done; echo
echo -n 'for i in "$@": '; for i in "$@" ; do echo -n '('$i') '; done; echo
echo -n 'for i : '; for i ; do echo -n '-'$i'- '; done; echo
echo -n 'for i : '; for i ; do echo -n '('$i') '; done; echo
echo -n 'while loop : '; while [ "$1" != "" ]; do echo -n '-'$1'- '; shift; done; echo
echo -n 'while loop : '; while [ $# -gt 0 ]; do echo -n '('$1') '; shift; done; echo
</source>
</source>
|<source lang=text>
|<source lang=text>
% loop one "or two"
% loop one "or two"
There are 2 parameters
There are 2 parameters
for i in $* : -one- -or- -two-
for i in $* : (one) (or) (two)
for i in $@ : -one- -or- -two-
for i in $@ : (one) (or) (two)
for i in "$*": -one or two-
for i in "$*": (one or two)
for i in "$@": -one- -or two-
for i in "$@": (one) (or two)
for i : -one- -or two-
for i : (one) (or two)
while loop : -one- -or two-
while loop : (one) (or two)
</source>
</source>
|}
|}

=== Looping on file LINEs ===
The simplest, using '''read''':
<source lang="bash">
# Reading line per line - from a file
while IFS= read -r LINE; do # IFS= to keep surrounding blanks, -r to keep backslashes
echo "$LINE"
done < file # Note: file could be <(command) to pipe from a command

# Reading line per line - command output / stdin
cat file | grep ... |
while IFS read -r LINE; do
echo "$LINE" # DO NOT ASSIGN VAR HERE, OR IT IS LOST (SUB-PROCESS)!!!
done

# Reading line per line - command output but *not* stdin
while IFS= read -u 3 -r LINE; do
echo "$LINE"
read -p -i Y "Do you want to continue? [Y/n]" # We can still query the user
done 3< <(command)
</source>

Or modifying '''IFS''':
<source lang="bash">
ORIGIFS=$IFS
IFS=$'\r'
for LINE in $(command); do
echo "$LINE"
done
IFS=$ORIGIFS
</source>


===case ... in ... esac===
===case ... in ... esac===
Line 445: Line 1,056:
<source lang="bash">
<source lang="bash">
###### Preventing echo with stty ######
###### Preventing echo with stty ######
read -p "Username: " uname
read -r -p "Username: " uname
stty -echo
stty -echo
read -p "Password: " passw; echo
read -r -p "Password: " passw; echo
stty echo
stty echo
</source>
</source>
Line 455: Line 1,066:
###### Using -s ######
###### Using -s ######
PASS="abc123"
PASS="abc123"
read -s -p "Password: " mypassword
read -r -s -p "Password: " mypassword
echo ""
echo ""
[ "$mypassword" == "$PASS" ] && echo "Password accepted" || echo "Access denied"
[ "$mypassword" == "$PASS" ] && echo "Password accepted" || echo "Access denied"
Line 477: Line 1,088:
PASSWORD=------------------------------------
PASSWORD=------------------------------------
</source>
</source>

Beware of {{red|pitfalls}} with [[Bash Tips and Pitfalls#Pits|read and piping]].


===Signal trapping===
===Signal trapping===
Line 493: Line 1,106:
trap " INT
trap " INT
</source>
</source>

Signals can be very handy. See for instance the [[Bash Tips and Pitfalls#Kill on ALARM signal|example on how to use ALRM signal to kill a process]]

Note that signal table is preserved by <tt>exec</tt> but not by <tt>fork</tt>

To force execution of a routine at exit, trap EXIT or equivalently 0 [http://stackoverflow.com/questions/64786/error-handling-in-bash]:
<source lang=bash>
trap "rm $tmpfile" EXIT
</source>

See also [[Bash_Tips_and_Pitfalls#Use_signals_to_fail_cleanly|Bash tips]] for more tricks.

===Exit Status===
Use <tt>$?</tt> to read the exit status of the last command executed. Value is '''0''' if last command ran without error, and an integer value between '''1-255''' on error. The command <tt>exit ''nnn''</tt> can be used to terminate a script or function and to deliver a given exit status.

One can negate a value with operator !, but don't forget the space!

<source lang="bash">
true # The "true" builtin.
! true # Negating true
true
!true # The '!' operator prefixing a command invokes the Bash history mechanism. It just repeats the previous command.
</source>

Some examples to test the exit status:
<source lang="bash">
if [ $? -eq 0 ]; then echo "exit status is 0"; else echo "exit status is non-zero"; fi

SUCCESS=0
grep -q "$word" "$filename" # "q" - quiet mode
if [ $? -eq $SUCCESS ]; then echo "$word was found"; else echo "not found"; fi

if ( ps -as | grep -q "ssh-agent" ); then echo ssh-agent started; else echo ssh-agent not started; fi
</source>

===TCP/UDP devices===
Bash creates special TCP/UDP devices (<tt>/dev/tcp/host/port</tt> and <tt>/dev/udp/host/port</tt>) that can be used to easily establish TCP/UDP connections in Bash scripts.

Some examples from [http://tldp.org/LDP/abs/html/devref1.html]:
* Getting time from NIST:
<source lang="bash">
cat </dev/tcp/time.nist.gov/13
# 53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) *
</source>

* Downloading an URL from a web server:
<source lang="bash">
exec 5<>/dev/tcp/www.net.cn/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5
</source>

* Send email with Bash, see [[SMTP]].


===Miscellaneous===
===Miscellaneous===
Line 499: Line 1,165:
|-
|-
|Use <tt>eval</tt> to execute the output (stdout) of a process
|Use <tt>eval</tt> to execute the output (stdout) of a process
|<source lang="bash" enclose="valid">eval 'dircolors'</source>
|{{nb|1=<source lang="bash" enclose="valid">eval `dircolors`</source>}}
|-
|-
|Use <tt>$$</tt> as file name suffix or prefix to create unique name for temporary file.<br/><tt>$$</tt> is actually process PID.
|Use <tt>$$</tt> as file name suffix or prefix to create unique name for temporary file.<br/><tt>$$</tt> is actually process PID.
|<source lang="bash" enclose="valid">
|{{nb|1=<source lang="bash">
touch /tmp/mytmp.$$
touch /tmp/mytmp.$$
echo some text>/tmp/mytmp.$$
echo some text>/tmp/mytmp.$$
cat /tmp/mytmp.$$
cat /tmp/mytmp.$$
</source>
</source>}}
|}
|}


== References ==
==Bug==
* (version 4.1-2ubuntu3) Typo in manpage. <tt>menu-complete-backward</tt> is written <tt>menu-complete-krd</tt>. In the man-page, this is because of misplaced <tt>\</tt> (<tt>menu\-complete-\backward</tt> instead of <tt>menu\-complete\-backward</tt>. Filed a bug report using <code>bashbug</code>
* [http://www.filibeto.org/sun/lib/development/shell/intr_to_bash_scr.html], or with better formatting [http://www.justlinux.com/nhf/Programming/Introduction_to_bash_Shell_Scripting.html].

Latest revision as of 10:07, 17 July 2024

Related pages

Other pages dedicated to Bash on this wiki:

References

External references:

help getopts
help test
help if

About alternatives to Bash:

Shell

Keyboard shortcuts

Some of the most used keyboard shortcuts in Bash (excerpt from the manual pages).
C- refers to Ctrl key, M- refers to Meta key (typ. Alt+key, or Esc then key), S- refers to Shift key.
\e introduces an escape character (same as ^[ printed by verbatim command (Ctrl-V) in vim)

Customized keyboard shortcuts described below can be obtained by adding the following lines to e.g. your ~/.inputrc file:

"\eOA":history-search-backward     #Up arrow
"\e[A":history-search-backward     #Up arrow
"\eOB":history-search-forward      #Down arrow
"\e[B":history-search-forward      #Down arrow
"\C- ":dynamic-complete-history    #Ctrl-space
"\e ":dynamic-complete-history     #Esc-space
"\e[2~":paste-from-clipboard       #Insert
tab:menu-complete                  #TAB
"\e[Z":menu-complete-backward      #Shift-TAB
"\C-y":menu-complete-backward      #Ctrl-Y
Note on defining new bindings
Binding by defaults are defined in file ~/.inputrc. Alternatively they can also be defined in the start file ~/.bashrc using command bind.
They are different method to define the binding keys
  • Specifying the key Keyname, without double quotes (but always within single quotes when using command bind)
  • Specifying the key Keyseq, with double quotes (*and* single quotes when using command bind)
  • Specifying the key Verbatim, with double quotes, and using control-V followed by key in Bash or Vi. Note: This is a good trick to know the key sequence of special keys like arrow key.
The binding keyseq differs depending on whether Readline is configured to support 8-bit input/output characters. Cygwin and openSUSE have different settings regarding this. To support 8-bit input/output (like in OpenSUSE), add the following to the ~/.inputrc:
set convert-meta off
set input-meta on
set meta-flag on
set output-meta on
bind 'tab:dynamic-complete-history'                #Keyname
bind '"\C-i":dynamic-complete-history'             #Keyseq
bind '"<Control-V><Tab>:dynamic-complete-history'  #Verbatim - <> means press the given key while editing

See also

man readline
Using negative argument in readline command
[3], [4]. Examples:
"\e[Z":"\M--1\t"          # \M--1 is actually-1 argument for next command , \t i.e. TAB
"\e[Z":"\M--\t"           # We can skip the 1
"\e[Z":"\e--\C-i"         # Equivalent, but using \e instead of \M, and \C-i instead of \t

Commands for Moving

forward-word (M-f, C-→, M-→, S-→)
Move forward to the end of the next word. Words are composed of alphanumeric characters (letters and digits).
!!! C-→, M-→, S-→ not supported in Cygwin.
backward-word (M-b, C-←, M-←, S-←)
Move back to the start of the current or previous word. Words are composed of alphanumeric characters (letters and digits).
!!! C-←, M-←, S-← not supported in Cygwin.
clear-screen (C-l)
Clear the screen leaving the current line at the top of the screen. With an argument, refresh the current line without clearing the screen.

Commands for Manipulating the History

reverse-search-history (C-r)
Search backward starting at the current line and moving 'up' through the history as necessary. This is an incremental search.
This is equivalent to interactive grep in history — very powerful
forward-search-history (C-s)
Search forward starting at the current line and moving 'down' through the history as necessary. This is an incremental search.
!!! Not supported in Cygwin.
history-search-forward ()
Search forward through the history for the string of characters between the start of the current line and the point. This is a non-incremental search.
!!! Custom keyboard shortcut.
history-search-backward ()
Search backward through the history for the string of characters between the start of the current line and the point. This is a non-incremental search.
!!! Custom keyboard shortcut.
yank-last-arg (M-., M-_)
Insert the last argument to the previous command (the last word of the previous history entry). With an argument, behave exactly like yank-nth-arg. Successive calls to yank-last-arg move back through the history list, inserting the last argument of each line in turn. The history expansion facilities are used to extract the last argument, as if the "!$" history expansion had been specified.
history-expand-line (M-^)
Perform history expansion on the current line. See HISTORY EXPANSION below for a description of history expansion.
!!! AZERTY keyboard: press Alt-^ twice
operate-and-get-next (C-o)
Accept the current line for execution and fetch the next line relative to the current line from the history for editing. Any argument is ignored.
edit-and-execute-command (C-xC-e)
Invoke an editor on the current command line, and execute the result as shell commands. Bash attempts to invoke $VISUAL, $EDITOR, and emacs as the editor, in that order.

Commands for Changing Text

delete-char (C-d)
Delete the character at point. If point is at the beginning of the line, there are no characters in the line, and the last character typed was not bound to delete-char, then return EOF.
backward-delete-char (Rubout)
Delete the character behind the cursor. When given a numeric argument, save the deleted text on the kill ring.
quoted-insert (C-q, C-v)
Add the next character typed to the line verbatim. This is how to insert characters like C-q, for example.
transpose-chars (C-t)
Drag the character before point forward over the character at point, moving point forward as well. If point is at the end of the line, then this transposes the two characters before point. Negative arguments have no effect.
transpose-words (M-t)
Drag the word before point past the word after point, moving point over that word as well. If point is at the end of the line, this transposes the last two words on the line.
upcase-word (M-u)
Uppercase the current (or following) word. With a negative argument, uppercase the previous word, but do not move point.
downcase-word (M-l)
Lowercase the current (or following) word. With a negative argument, lowercase the previous word, but do not move point.
capitalize-word (M-c)
Capitalize the current (or following) word. With a negative argument, capitalize the previous word, but do not move point.
overwrite-mode
Toggle overwrite mode. With an explicit positive numeric argument, switches to overwrite mode. With an explicit non-positive numeric argument, switches to insert mode. This command affects only emacs mode; vi mode does overwrite differently. Each call to readline() starts in insert mode. In overwrite mode, characters bound to self-insert replace the text at point rather than pushing the text to the right. Characters bound to backward-delete-char replace the character before point with a space. By default, this command is unbound.

Killing and Yanking

kill-line (C-k)
Kill the text from point to the end of the line.
unix-line-discard (C-u)
Kill backward from point to the beginning of the line. The killed text is saved on the kill-ring.
kill-word (M-d)
Kill from point to the end of the current word, or if between words, to the end of the next word. Word boundaries are the same as those used by forward-word.
backward-kill-word (M-Rubout)
Kill the word behind point. Word boundaries are the same as those used by backward-word.
unix-word-rubout (C-w)
Kill the word behind point, using white space as a word boundary. The killed text is saved on the kill-ring.

Numeric Arguments

digit-argument (M-0, M-1, ..., M--)
Add this digit to the argument already accumulating, or start a new argument. M-- starts a negative argument.

Completing

complete (TAB)
Attempt to perform completion on the text before point. Bash attempts completion treating the text as a variable (if the text begins with $), username (if the text begins with ~), hostname (if the text begins with @), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted.
menu-complete (TAB) / menu-complete-backward (Shift-TAB, C-Y)
Similar to complete but replaces the word with next (previous) completion.
!!! Apparently Shift-Tab doesn't work in Cygwin. Use Ctrl-Y instead
possible-completions (M-?)
List the possible completions of the text before point.
insert-completions (M-*)
Insert all completions of the text before point that would have been generated by possible-completions.
dynamic-complete-history (M-SPACE,C-SPACE)
Attempt completion on the text before point, comparing the text against lines from the history list for possible completion matches.
!!! Must use combination Esc + Tab. Custom combination in Cygwin.

Miscellaneous

prefix-meta (ESC)
Metafy the next character typed. ESC f is equivalent to Meta-f.
undo (C-_, C-x C-u)
Incremental undo, separately remembered for each line.
!!! AZERTY keyboard: press Control--.
revert-line (M-r)
Undo all changes made to this line. This is like executing the undo command enough times to return the line to its initial state.
set-mark (C-@, M-<space>)
Set the mark to the point. If a numeric argument is supplied, the mark is set to that position.
exchange-point-and-mark (C-x C-x)
Swap the point with the mark. The current cursor position is set to the saved position, and the old cursor position is saved as the mark.

History

Some examples (assume that magic-space is mapped to Space)

echo 12 34
!!                 # Execute previous command
!1                 # Execute command n°1
!-2                # Execute penultimate command
echo 12 34
^12^56^            # Execute previous command, replacing 12 with 56. Quick for !!:s/12/56/
!!<space>          # Example of magic space
echo !-2<space>    #   ... also works anywhere in the current line
for i in <C-xC-e>  # Shortcut to edit cmd in editor
fc                 # Edit previos command in editor

Script Quick Tutorial

Invocation

A bash shell can be (see man bash)

  • either interactive or not interactive
The shell is interactive when started with the -i option, or when not started without non-option arguments and without the -c option, and whose stdin/stderr are both connected to terminals.
  • either a login shell or not a login shell
A login shell is one whose first character of argument zero is a - (i.e. $0 == -bash or such), or one started with the --login or -l option.
Using su a login shell is started with option -, -l or --login.

So

su - username               # LOGIN   and INTERACTIVE
bash --login
bash -l
su username                 # NOT login   and INTERACTIVE
bash
bash -i
su - username -c "..."      # LOGIN   and NOT interactive
bash --login -c "..."
su username -c "..."        # NOT login and NOT interactive
bash -c "..."
bash -c "..." --login       # idem (actually --login is passed to -c commad)
  • Use command shopt to know whether current shell is interactive / login.
  • Interactive shells have i defined in $-, and also define variable PS1.
  • Sourcing file can be skipped / enforced with --noprofile, --norc or --rcfile.

A simple script to detect the current invocation [5]:

[[ $- == *i* ]] && echo 'Interactive' || echo 'Not interactive'
shopt -q login_shell && echo 'Login shell' || echo 'Not login shell'

The following files are sourced by bash at invocation

NOT login LOGIN
INTERACTIVE
  • First /etc/bash.bashrc.
  • Then ~/.bashrc
  • First /etc/profile.
  • Then either ~/.bash_profile, ~/.bash_login, ~/.profile, whichever is found first.
NOT interactive

None

But on Ubuntu, /etc/profile automatically sources /etc/bash.bashrc, which then aborts if it detects that the shell is not interactive. Also ~/.profile sources ~/.bashrc, which also aborts if non-interactive. So in an out-of-the-box configuration, the following files are sourced:

NOT login LOGIN
INTERACTIVE

/etc/bash.bashrc
~/.bashrc

/etc/profile
/etc/bash.bashrc
~/.profile
~/.bashrc

NOT interactive

None

/etc/profile
~/.profile

Environment

export MYVAR=myvalue && command-to-execute         #MYVAR defined for the current shell and invoked shell
MYVAR=myvalue && command-to-execute                #MYVAR defined for the current shell
MYVAR=myvalue command-to-execute                   #MYVAR only defined for the subsequent command

As a rule of thumb, better define environment variables (PATH, etc) in ~/.profile, so that they are available to all program launched from a login shell (see [6], [7]).

Quoting

Escape \
cp file \'file\$\'
Preserve literal value of the next character
Single quote '...'
echo 'it cost $10'
Preserve literal value of all characters
Dollar Single quote '...'
echo $'it cost \'$10\''
Preserve literal value of all characters, except escaped character
Double quote "..."'
cp file "file $var"
Preserve literal value of all characters, except $ ` \ !
Dollar Double quote $"..."'
echo $"file not found"
Translate string to current locale (for internationalization

Compound Command

(list)
if ( kill $pid; ps -p $pid )
Execute list in a subshell and set return status to last command returned status
{ list; }
{ read v1; read v2; } <foo
Group command — Execute list in current environment. Note the ; and whitespaces!
((expr))
 if ((i<10)); ...
Arithmetic evalution. Set return status accordingly.
[[ expr ]]
 if [[ -e foo && -e bar ]]; ...
Condition expression. Set return status accordingly.

Expansion and Substitution

This is only a subset of the expansion possibilities. See the man pages for more information

${!param}
var=hello; ref=var; 
echo ${!ref}          # "hello"
foo[0]=foo0; foo[1]=foo1
ref=foo[0]            # ... also for arrays!
echo ${!ref}          # "foo0"
Indirect expansion — Equivalent to eval \$$ref, but cleaner and safer. It also works for arrays!
${param%word}
${param%%word}
var1=file.ext; var2=dir//; 
echo ${var1%%.ext}    # "file"
echo ${var2%%/}       # "dir"
Parameter expansion — Suffix removal. Use a single % for shortest match.
Memo: 100% → suffix.
${param#word}
${param##word}
var=__file
echo ${var##__}       # "file"
Parameter expansion — Prefix removal. Use a single # for shortest match.
memo: #comment → prefix.
${param/pat/str}
${param//pat/str}
var=sometext
echo ${var/e/_}       # "som_text"
Parameter expansion — Search & replace. Use /pattern to replace all matches
$(command)
`command`
echo $(basename $0)
Command substitution — Expands to the output of command
Trailing newlines are stripped!
$((expr))
$[expr]
echo $((16 * 16))
Arithmetic expansion — Expands to the result of expression ($[...] is deprecated)
<(command)
>(command)
diff <(ls dir1) <(ls dir2)
Process substitution — Substitute a process with a named pipe connected to the process (see tips)
* ? [...]
echo *
Pathname expansion — Expands to file names matching given pattern (done after word splitting)
{0..19}
echo {0..9}
# 0 1 2 3 4 5 6 7 8 9
Sequence expansion

Here summary of expansion result depending on whether parameter is unset, set and/or null:

set and not null null unset
${parameter:-word} substitute parameter substitute word substitute word
${parameter-word} substitute parameter substitute null substitute word
${parameter:=word} substitute parameter assign word assign word
${parameter=word} substitute parameter substitute null assign word
${parameter:?word} substitute parameter error, exit error, exit
${parameter?word} substitute parameter substitute null error, exit
${parameter:+word} substitute word substitute null substitute null
${parameter+word} substitute word substitute word substitute null

Positional Parameters

See section "Looping on positional parameters" for an illustration on how to process each parameters in a for loop.

# !/bin/bash

function parsing ()
{
    echo "($1) ($2) ($3) ($4)"
}

echo -n 'parsing $*   -- ';  parsing $*
echo -n 'parsing $@   -- ';  parsing $@
echo -n 'parsing "$*" -- ';  parsing "$*"
echo -n 'parsing "$@" -- ';  parsing "$@"
# Echo number of parameters
echo $#
$ ./test.sh p1 p2 "p 3" 'p 4'
# parsing $*   -- (p1) (p2) (p) (3)
# parsing $@   -- (p1) (p2) (p) (3)
# parsing "$*" -- (p1 p2 p 3 p 4) () () ()
# parsing "$@" -- (p1) (p2) (p 3) (p 4)
# 4

To test if there are parameters:

if [ $# -eq 0 ]; then
    echo No parameters!
    exit 1
fi

Operators

VAR+=value

if VAR is a string, append value
if VAR is an array, add element value
if VAR is a numerical value, add value

STRING=
STRING+=" foo"    # Now string is "foo" (w/o space!)
STRING+=" bar"    # Now string is "foo bar"

String pattern matching

Several possibilities to do pattern matching in strings:

  • Using operator =~ and regular expression
  • Using operator == and Bash pattern
  • Using case

Using case is the fastest option for simple, single patterns (10% faster than ==). == is faster than =~ (for instance fooMATCHbar == *MATCH* is roughly 3x faster than fooMATCHbar =~ .*MATCH.* ).

Examples
if [[ fooMATCHbar =~ .*MATCH.* ]]; then echo found; done
if [[ fooMATCHbar == *MATCH* ]]; then echo found; done
case fooMATCHbar in *MATCH*) echo found;; esac

Redirection - Descriptors, Here Documents and Here Strings

&>word
>&word
Redirect both stdout and stderr (semantically equivalent to >word 2>&1)
|& Pipe both stdout and stderr


Description Source Output
Descriptors - redirect input / output to files or file descriptor. These file names have special meaning: /dev/fd/fd (with 2nd fd an integer), /dev/stdin, /dev/stdout, /dev/stderr, /dev/tcp/host/port, /dev/udp/host/port
See Using Process Substitution tip and stderr redirection tip for more advanced redirection.
ls > /dev/stderr    # WRONG
ls >> /dev/stderr   # BETTER
ls >&2              # BEST
ls 2>&1 > dirlist   # WRONG 
ls > dirlist 2>&1   # OK
ls >& dirlist       # BETTER
ls &> dirlist       # BETTER (PREFERRED)
./myapp |& tee foo  # Pipe stdout & stderr
Here Documents - Use <<word to declare here documents.
Check BASH manpage for more information.

We can also combine several here documents on the same line, or use pipelines [8]

cat <<__EOF__
	First line
	Second line
	Third line
__EOF__
# here document with pipeline
cat <<__EOF__ |
	First line
	Second line
	Third line
__EOF__
tr [a-z] [A-Z]
# Multi here documents
cat <<eof1; cat <<eof2
Hi,
eof1
Helene.
eof2
	First line
	Second line
	Third line
	FIRST LINE
	SECOND LINE
	THIRD LINE
	Hi,
	Helene.
Here Documents - Use <<- to strip leading tab characters.
cat <<-__EOF__
	First line
	Second line
	Third line
__EOF__
First line
Second line
Third line
Here Strings - Use <<<word to declare here strings.
cat <<<'Hello, World!'
TEXT='Hello, World!' cat <<<$TEXT
cat<<<"First line
> Second line
> Third line"
Hello, World!
Hello, World!
First line
Second line
Third line

If ... then ... [elif ...] ... else ... fi

(See [9] for in-depth discussion on Bash tests)

The basic syntax for if-then-else sequence control is

if list; then list; [ elif list; then list; ] ... [ else list; ] fi

The if/then construct tests whether the exit status of a list of command is 0 (i.e. success), and if so, executes another list of command. Test conditions can be written using shell built-in command test, or equivalently the square brackets [...]. Other possibilities are using the extended test command [[...]] (that performs parameter expansion and command substitution as well as arithmetic evaluation, see [10]), or the double-parens ((...)) expression (see [11]) that evaluates an arithmetic expression (returs true if expression is non-zero).

if mv $src $dst; then echo move is ok; else move is not ok; fi                 # Using list
if test -f /etc/foo; then echo file found; else echo file NOT found; fi        # test command
if [ -f /etc/foo ]; then echo file found; else echo file NOT found; fi         # [ is a synomym for test and a builtin
if [[ -f $file ]]; then echo file $file found; else echo file NOT found; fi    # Extended test construct - &&, ||, < and > works 
if (( 128 && 64 )); then echo result is non-zero; else echo result is zero; fi # Expands and evaluates arithmetic expression

Here's test's options (more info with help test):

-d FILE        # Check if the file is a directory
-e FILE        # Check if the file exists
-f FILE        # Check if the file is a regular file
-g FILE        # Check if the file hash SGID permissions
-r FILE        # Check if the file is readable
-s FILE        # Check if the file's size is not 0
-u FILE        # Check if the file has SUID permissions
-w FILE        # Check if the file is writeable
-x FILE        # Check if the file is executable
NBR1 -eq NBR2  # Check is NBR1 is equals to NBR2
NBR1 -ne NBR2  # Check if NBR1 is not equals to NBR2
NBR1 -ge NBR2  # Check if NBR1 is greater than or equal to NBR2
NBR1 -gt NBR2  # Check if NBR1 is greater than NBR2
NBR1 -le NBR2  # Check if NBR1 is less than or equal to NBR2
NBR1 -lt NBR2  # Check if NBR1 is less than NBR2
STR1 = STR2    # Check if STR1 is the same as STR2
STR1 != STR2   # Check if STR1 is not the same as STR2
-n STR         # Evaluates to true if STR is not null (can be used to test whether a variable is defined)
-z STR         # Evaluates to true if STR is null.

Conditions can be combined with && and || or with test operator -a and -o. Use ! to negate a test:

if [ $x -ge 5 ] && [ $x -le 10 ]; then ...; fi
if [ $x -gt 5 ] || [ $x -lt 10 ]; then ...; fi
if [ $x -ge 5 -a $x -le 10 ]; then ...; fi
if [ $x -gt 5 -o $x -lt 10 ]; then ...; fi
if ! [ -a /some/file.txt ]; then ...; fi                     #negation
if [ -n "$DEFINED" ]; then echo DEFINED is defined; fi
if [ "$DEFINED" ]; then echo DEFINED is defined; fi          #also works

Some examples:

if [ "$name" -eq 5 ]; then echo one; else echo two; fi
if [ -d ~/var ]; then touch ~/var/done; else { mkdir ~/var; touch ~/var/done; } fi   # braces { ... } are optional here

Alternative: Using && and ||

Using && and ||, one can also build a if ... then ... else statement:

[ -f /etc/hosts ] && { echo one; echo two; } || { echo three; echo four; }

Using [[expression]]

The conditinal expressions [[expression]] uses the same CONDITIONAL EXPRESSIONS as the regular [ ... ], but in addition accept the following construct (see manual pages for more):

value == pattern
value != pattern

if [[ $NAME == *.c ]]; then echo C Source file; fi
String Pattern is considered a shell pattern

value =~ pattern

if [[ $(ps $PID) =~ ssh-agent ]]; then echo $PID is ssh-agent pid; fi
String Pattern is considered an extended regular regex

Thanks to these, there is no need for constructs using grep:

if ps aux | grep ssh-agent; then echo ssh-agent found; fi        # NOT EFFICIENT, 2 processes spawn

if [[ $(ps aux) =~ ssh-agent ]]; then echo ssh-agent foudn; fi   # BETTER!!!

Strings

Reference: Advanced Bash Scripting Guide

len=${#line}         # Get length of string line
LANG=C LC_ALL=C
bytlen=${#line}      # Get *byte* length

stringZ=abcABC123ABCabc                      # 0-based indexing
echo ${stringZ:0}                            # abcABC123ABCabc
echo ${stringZ:1}                            # bcABC123ABCabc
echo ${stringZ:7}                            # 23ABCabc

echo ${stringZ:7:3}                          # 23A

# Using negative index
echo ${stringZ:-4}                           # abcABC123ABCabc - FAIL! because construct ${parameter:-default}.
echo ${stringZ:(-4)}                         # Cabc            - SUCCESS! 
echo ${stringZ: -4}                          # Cabc            - SUCCESS!

Arrays

See [12] for more detailed information.

# Assigning array elements
area[11]=23        #  Array members need not be consecutive or contiguous.
area[13]=37
area[51]=UFOs

# APPEND to array
ARRAY=()
ARRAY+=('foo')
ARRAY+=('bar baz')

# LOOP on array element:
for i in ${!ARRAY[*]}; do echo ${ARRAY[$i]}; done

# Use curly brackets to read!
echo "\${area[11]}         = ${area[11]}"
echo "\${area[0]}  = $area = ${area[0]}"

# Use array to echo first word in a list:
make="/usr/bin/make -r --no-print-directory -j 2"
words=($make)
echo $words

# Store multiline in array
files=($(grep -l foo *))
echo "There are ${#files[@]} matching files, first is ${files[0]}"

# Other ways to assign an array
area2=( zero one two three four )       # zero-based indexing!
area3=([17]=seventeen [24]=twenty-four)

# Use quoting to insert white-spaces
array=( [0]="first element" [1]="second element" [3]="fourth element" )

# read file into array (one line, separated by blank)
read -a array < FILE

# Read file into array (line by line)
readarray -t array < FILE     # Bash 4.0
IFS=$'\r\n' read -d '' -r -a array < FILE
IFS=$'\r\n' GLOBIGNORE='*' command eval 'XYZ=($(cat FILE))'
                              # https://stackoverflow.com/questions/11393817/read-lines-from-a-file-into-a-bash-array
readarray -d '' array < <(find . -name "$input" -print0)
                              # https://stackoverflow.com/questions/23356779/how-can-i-store-the-find-command-results-as-an-array-in-bash
                              # See also tips page for more

# Some operators
${array:1}   == ${array[0]:1} # Param. expansion of 1st element, starting at pos #1 (2nd char)
${#array}    == ${#array[0]}  # Length of 1st element
${#array[*]} == ${#array[@]}  # Number of elements in array.
unset array[1]                # Remove 2nd element of array (but ***leave a HOLE in INDEX***)
array[1]=                     # ... idem
unset array                   # Delete entire array

# Re-INDEX an array (e.g. if some element deleted)
foo=(a b c)
declare -p foo                # declare -a foo=([0]="a" [1]="b" [2]="c")
unset foo[1]
declare -p foo                # declare -a foo=([0]="a" [2]="c")
foo=("${foo[@]}")             # Re-indexing
declare -p foo                # declare -a foo=([0]="a" [1]="c")

# CROP array
array=("${array[@]:1}")   # Crop 1st elm and reindex, using substring expansion
unset 'array[0]'          # Crop 1st elm, but doesn't reindex
array=("${array[@]}")     # ... to force reindex
# Keep only some of the last elements
# ... using a loop
while [ ${#array[@]} -gt 5 ] ; do
    array=("${array[@]:1}")
done
# ... using a test
[ ${#array[@]} -gt 5 ] && array=("${array[@]:$((${#array[@]} -5))}")

# Test arrays
[ ${#array[@]} -eq 0 ] && echo "array empty"
[[ -v array[@] ]] && echo "array is set"  # BASH 4.2 - But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY
[[ ${array@a} = a ]] && echo "array is ARRAY"  # BASH 5.x - ... ARRAY is DECLARED, but maybe not SET yet

# PRINT arrays
echo ${array[@]}              # Print elements separated by SPACE
echo ${array[*]}              # ... idem
echo "${array[*]}"            # ... idem
echo "${array[@]}"            # Print elements separated by 1st char in IFS
IFS='/'; echo "${array[*]}"   # MUST set IFS before!
IFS='/' echo "${array[*]}"    # Does NOT work because echo param got expanded BEFORE IFS assignment

unset array[1]
echo "${array[*]}"            # DELETED element are NOT printed

# COPY an array
copy=("${array[@]}")

# INDIRECT access to array element
REF=array[0]
# REF=array[@]    -- also works
# REF=!array[@]   -- DOES NOT work though (for keys)
echo ${!REF}

# ... better solution since bash 4.3+, use **namerefs**:
declare -A array=( [a]=A [b]=B )
declare -n REF=array          # ... or ...
local -n REF=array            # Better in functions
echo ${REF[a]}                # Element access
echo ${REF[@]}                # Return all values
echo ${!REF[@]}               # Return all keys

# INDIRECT length of an array or array element (w/o declare -n)
REF=array[@]                  # Our reference to an array
local current=("${!str}")     # Copy into an array using indirect ref

echo ${#current[*]}           # Now we can use regular array operator - array size
echo ${current[1]}            # Print 2nd element
echo ${#current[1]}           # ... and size of 2nd element

# Examine / serialize array
declare -p array              # declare -a array=([0]="two" [1]="one")

Associative Arrays

Associative arrays (so-called hash in Perl) are available in Bash since version 4.

  • Associative array are declared with declare -A ARRAY
  • Values are accessed with $ARRAY[key]
  • Keys are accessed with exclamation mark: ${!ARRAY[@]}
declare -A ARRAY                          # Declare 'ARRAY' as a (local) associative array
declare -gA ARRAY                         # Declare 'ARRAY' as a global associative array
                                          # ... handy when array is constructed in a function
ARRAY[foo]=FOO                            # Associate value 'FOO' to key 'foo'
ARRAY[bar]=BAR                            # Associate value 'BAR' to key 'bar'
declare -A ARRAY=( [foo]=FOO [bar]=BAR )  # ... idem, on one line
declare -p ARRAY                          # declare -A ARRAY=( [foo]=FOO [bar]=BAR )
echo ${ARRAY[foo]}                        # Print value associated with 'foo'
echo ${ARRAY[*]}                          # Print all values in ARRAY
echo ${ARRAY[@]}                          # ... idem
echo ${!ARRAY[*]}                         # Print all ***keys*** in ARRAY
echo "${ARRAY[*]}"                        # Same as for "$*"
echo "${ARRAY[@]}"                        # Same as for "$@"
${#ARRAY[*]} == ${#ARRAY[@]}              # Number of elements in ARRAY.
[ ${ARRAY[key]+x} ] && echo "exists"      # Check if 'key' exists, based on ${foo+x} returning x if foo set and nothing otherwise
[[ -v ARRAY[key] ]] && echo "exists"      # ... alternative (BASH 4.2)
[[ ${ARRAY@a} = A && -v ARRAY[key] ]]     # ... SAFER, test ARRAY is ASSOC first (BASH 5.0)
keys=(${ARRAY[@]})                        # All keys in a standard array

To delete an array or element:

unset ARRAY                               # Empty the ARRAY (ARRAY is NOT ASSOCIATIVE anymore!)
unset ARRAY[foo]                          # Delete a KEY
unset ARRAY["my key"]                     # ... quote if KEY contains a space
K="my key"
unset ARRAY["$K"]                         # Delete a KEY contained in variable

To CLEAR an associative array in a function, UNSET it then declare it GLOBALLY with -gA. For instance:

unset A
declare -A A
A[C]=2; 
a() 
{ 
  unset A        # Clear the array
  declare -gA A  # Declare it as ASSOCIATIVE and GLOBAL
  A[X]=1
}
a
declare -p A     # declare -A A=([X]="1")

The following test if ARRAY is set:

# BASH 4.2
[[ -v ARRAY[@] ]] && echo "ARRAY is set"   # But ARRAY can be a VARIABLE, an ARRAY, or ASSOC. ARRAY

# BASH 5.x
[[ ${ARRAY@a} = A ]] && echo "ARRAY is ASSOC ARRAY"  # ... ARRAY is DECLARED, but maybe not SET yet

WHILE / UNTIL Loops

Syntax is

while list; do list; done
until list; do list; done

Some examples of While loops:

while true; do
   echo "Press CTRL-C to quit."
done

# Faster alternative using Bash built-in colon feature
while :; do
   echo "Press CTRL-C to quit."
done

# A more complete example
x=0;                                 # initialize x to 0
while [ "$x" -le 10 ]; do
    echo "Current value of x: $x"
    # increment the value of x:
    x=$(expr $x + 1)
    sleep 1
done

Some examples of until loops:

x=0
until [ "$x" -ge 10 ]; do
    echo "Current value of x: $x"
    x=$(expr $x + 1)
    sleep 1
done

FOR Loops

There are 2 syntax for for loops:

for name [ in word ] ; do list ; done
for (( expr1 ; expr2 ; expr3 )) ; do list ; done

The normal iteration sequence can be modified with keywords continue [n], break [n].

The first syntax enumerates a list, as in:

# Counting from 1 to 10...
for dots in 1 2 3 4 5 6 7 8 9 10; do echo -n "$dots... "; done;
echo 'Done !'

# Enumerating a list
for fruit in Apple Pear Cherry; do
  echo "The value of variable fruit is: $fruit"
  sleep 1
done

# Acting on a directory list:
for file in *; do
  echo "Adding .html extension to $file..."
  mv $file $file.html
  sleep 1
done

# On several lines
for f in $(seq 33296 33330); do
  wget "http://i.techrepublic.com.com/gallery/${f}.jpg"
done

# On a single line
for i in `seq 278 328`; do wget http://i.techrepublic.com.com/gallery/33$i.jpg; done

Looping on positional parameters

Several solutions:

  • for i in "$@" (note the quotes), explicitly refering to $@ as source list, and which assign i to each positional parameter.
  • for i, which Bash interprets as an implicit in "$@").
  • Use a while [ $# -gt 0 ] loop and shift (note: testing [ -n "$1" ] is flawed since some param might be empty)
#Illustrate correct and incorrect ways to loop through positional parameters
echo There are $# parameters
echo -n 'for i in $*  : '; for i in $*         ; do echo -n '('$i') '; done; echo
echo -n 'for i in $@  : '; for i in $@         ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$*": '; for i in "$*"       ; do echo -n '('$i') '; done; echo
echo -n 'for i in "$@": '; for i in "$@"       ; do echo -n '('$i') '; done; echo
echo -n 'for i        : '; for i               ; do echo -n '('$i') '; done; echo
echo -n 'while loop   : '; while [ $# -gt 0 ]; do echo -n '('$1') '; shift; done; echo
% loop one "or two"
There are 2 parameters
for i in $*  : (one) (or) (two)
for i in $@  : (one) (or) (two)
for i in "$*": (one or two)
for i in "$@": (one) (or two)
for i        : (one) (or two)
while loop   : (one) (or two)

Looping on file LINEs

The simplest, using read:

# Reading line per line - from a file
while IFS= read -r LINE; do                         # IFS= to keep surrounding blanks, -r to keep backslashes
    echo "$LINE"
done < file                                         # Note: file could be <(command) to pipe from a command

# Reading line per line - command output / stdin
cat file | grep ... |
while IFS read -r LINE; do
    echo "$LINE"                                    # DO NOT ASSIGN VAR HERE, OR IT IS LOST (SUB-PROCESS)!!!
done

# Reading line per line - command output but *not* stdin
while IFS= read -u 3 -r LINE; do
    echo "$LINE"
    read -p -i Y "Do you want to continue? [Y/n]"   # We can still query the user
done 3< <(command)

Or modifying IFS:

ORIGIFS=$IFS
IFS=$'\r'
for LINE in $(command); do
    echo "$LINE"
done
IFS=$ORIGIFS

case ... in ... esac

The syntax is

case word in [ [(] pattern [ | pattern ] ... ) list ;; ] ... esac

Some example

x=5     # initialize x to 5
# now check the value of x:
case $x in
   0) echo "Value of x is 0."
      ;;
   5) echo "Value of x is 5."
      ;;
   9) echo "Value of x is 9."
      ;;
   *) echo "Unrecognized value."
esac

Functions

Functions are like aliases, but accept parameters:

function myfunc1() { echo "$2"; echo "$1"; }
myfunc2() { MYVAR="$1"; echo Another $MYVAR function; }       #The keyword ''function'' is optional

Interactivity

Read User's password

Reading input from user is done with the commands read. To prevent password echo on display, one can use the command stty:

###### Preventing echo with stty ######
read -r -p "Username: " uname
stty -echo
read -r -p "Password: " passw; echo
stty echo

Preventing password echo using option -s:

###### Using -s ######
PASS="abc123"
read -r -s -p "Password: " mypassword
echo ""
[ "$mypassword" == "$PASS" ] && echo "Password accepted" || echo "Access denied"

Complete solution:

###### Complete solution ######
USER=wbi\\titeuf
echo "Mounting windows share... Please type password for user $USER..."
# time-out after 60sec, raw input no escaping, no echo, prompt
read -t 60 -r -s -p "Password: " PASSWORD
# delete prompt line
echo -e -n "\r"
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/C$ /mnt/c
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/D$ /mnt/d
mount -t smbfs -o username="$USER",password="$PASSWORD",iocharset=iso8859-1,codepage=cp437 //windows-host/F$ /mnt/f
# delete password variable
PASSWORD=------------------------------------
PASSWORD=abcdefghijklmnopqrstuvwxyz0123456789
PASSWORD=------------------------------------

Beware of pitfalls with read and piping.

Signal trapping

Use trap to trap signals send to a Bash script, and redirect execution to a given function/command.

#Trap Ctrl-C (signal SIGINT) by executing function "sorry"
trap sorry INT
sorry() { echo "I'm sorry Dave. I can't do that."; sleep 3; }

Signals can also be ignored or reset.

#reset the trap:
trap - INT
#do nothing when SIGINT is caught:
trap " INT

Signals can be very handy. See for instance the example on how to use ALRM signal to kill a process

Note that signal table is preserved by exec but not by fork

To force execution of a routine at exit, trap EXIT or equivalently 0 [13]:

trap "rm $tmpfile" EXIT

See also Bash tips for more tricks.

Exit Status

Use $? to read the exit status of the last command executed. Value is 0 if last command ran without error, and an integer value between 1-255 on error. The command exit nnn can be used to terminate a script or function and to deliver a given exit status.

One can negate a value with operator !, but don't forget the space!

true    # The "true" builtin.
! true  # Negating true
true
!true   # The '!' operator prefixing a command invokes the Bash history mechanism. It just repeats the previous command.

Some examples to test the exit status:

if [ $? -eq 0 ]; then echo "exit status is 0"; else echo "exit status is non-zero"; fi

SUCCESS=0
grep -q "$word" "$filename"    #  "q" - quiet mode
if [ $? -eq $SUCCESS ]; then echo "$word was found"; else echo "not found"; fi

if ( ps -as | grep -q "ssh-agent" ); then echo ssh-agent started; else echo ssh-agent not started; fi

TCP/UDP devices

Bash creates special TCP/UDP devices (/dev/tcp/host/port and /dev/udp/host/port) that can be used to easily establish TCP/UDP connections in Bash scripts.

Some examples from [14]:

  • Getting time from NIST:
cat </dev/tcp/time.nist.gov/13
# 53082 04-03-18 04:26:54 68 0 0 502.3 UTC(NIST) *
  • Downloading an URL from a web server:
exec 5<>/dev/tcp/www.net.cn/80
echo -e "GET / HTTP/1.0\n" >&5
cat <&5
  • Send email with Bash, see SMTP.

Miscellaneous

Use eval to execute the output (stdout) of a process
eval `dircolors`
Use $$ as file name suffix or prefix to create unique name for temporary file.
$$ is actually process PID.
touch /tmp/mytmp.$$
echo some text>/tmp/mytmp.$$
cat /tmp/mytmp.$$

Bug

  • (version 4.1-2ubuntu3) Typo in manpage. menu-complete-backward is written menu-complete-krd. In the man-page, this is because of misplaced \ (menu\-complete-\backward instead of menu\-complete\-backward. Filed a bug report using bashbug