Awk

From miki
Revision as of 06:08, 2 November 2017 by Mip (talk | contribs) (→‎References)
Jump to navigation Jump to search

References

Language reference

Awk program structure

@include "script1"    # gawk extension
pattern {action}
pattern {action}
# ...
function name (args) { ... }

A rule is a pattern and action. Either pattern or action can be omitted.

Patterns

/regular expression/ {  }   # match when input records fits reg. exp.
expression           {  }   # match when expression is nonzero
begpat, endpat       {  }
BEGIN                {  }   # match program begin. All BEGIN rules are merged.
END                  {  }   # match program end. All END rules are merged.
BEGINFILE            {  }   # match begin of each file (merged)
ENDFILE              {  }   # match end of each file (merged)
                     {  }   # empty pattern. Match every input record

Control statement

Block and sequences
Instructions are grouped with braces { ... } and separated by newlines or semi-colons ;;
{ if (NR) { print NR; print "not null" } }
If statement
# multiline
if (x % 2)
    print "x is even"
else
    print "x is odd"

# single line
if (x % 2) print "x is even"; else print "x is odd"
While statement
i = 1; while (i <= 3) { print $i; i++ }
For statement
for (i = 1; i <= 3; i++) print $i

Awk Program Examples

ps al | awk '{print $2}'                                         # Print second field of ps output
arp -n 10.137.3.129|awk '/ether/{print $3}'                      # Print third field of arp output, if line contains 'ether' somewhere
getent hosts unix.stackexchange.com | awk '{ print $1 ; exit }'  # Print only first line, then exit
find /proc -type l | awk -F"/" '{print $3}'                      # Print second folder name (i.e. process pid)

Example of parsing an XML file (and comparing with perl):

cat FILE
#        <configuration buildProperties="" description="" id="some_id.1525790178" name="some_name" parent="some_parent">
awk -F "[= <>\"]+" '/<configuration / { if ($8 == "some_name") print $6 }' FILE
# some_id.1525790178
perl -lne 'print $1 if /<configuration .* id="([^"]*)" name="some_name"/' FILE
# some_id.1525790178

How-To

Execute a system command and capture its output

To run a system command, we use system("cmd"). However to capture its output, we use cmd | getline value [1]. However, we must also close the command, otherwise awk will complain / will not reexecute the command / will produce strange resuts:

Example of program:

/\/\/ test password/ {
    cmd = "openssl rand -hex 16"; 
    cmd | getline r; 
    gsub(/[0-9a-f][0-9a-f]/,"0x&, ",r); 
    print "    { ", r, "}, // test password - DO NOT EDIT THIS COMMENT"; 
    close(cmd); 
    next;
}
{print}'

Tips

  • Defining environment variable - Using an Awk script and Bash builtin eval
eval $(awk 'BEGIN{printf "MY_VAR=value";}')
echo $MY_VAR
  • Hexadecimal conversion - Use strtonum to convert parameter:
{
    print strtonum($1);       # decimal, octal or hexa (guessed from prefix)
    print strtonum("0"$2);    # To force octal
    print strtonum("0x"$3);   # To force hexadecimal
}
  • Using environment variables - Use ENvIRON["NAME"]:
{ print strtonum("0x"ENVIRON["STARTADDR"]); }
  • Pass command-line parameters - Awk variables can be defined directly on the invocation line:
awk -v myvar=123 'BEGIN { printf "myvar is %d\n",myvar }'     # Use -v (before program text) for var used in BEGIN section
echo foo | awk '{ printf "myvar is %d\n",myvar }' myvar=123   # Otherwise specify var after program text
  • Pass command-line parameters - Awk defines the variables ARGC and ARGV:
BEGIN {
  for (i = 0; i < ARGC; i++)
  print ARGV[i]
}
  • $0 is the whole line
# Concatenate DNS
/^A\?/{print record; record=$0} 
/^A /{record=record " " $0;} 
END {print record}
  • String concatenation — simply line up the string without operator.
print "The result is " result;
  • Next line on pattern match — Only match one pattern in a pattern list
/PATTERN1/ {print $1; next}
/PATTERN2/ {print $2; next}
{print $3}