`awk` Process text by fields and patterns

Reads text line by line, splits each line into fields, and runs a small program on the lines you choose. Reach for it when grep and cut are not enough, for example to add up a column, filter on a number or reformat a table.

Synopsis

awk [-F fs] [-v var=value] 'program' [FILE]...
awk [-F fs] -f progfile [FILE]...

Common options

Option or form	What it does
`-F fs`	Set the field separator. Use -F, for comma-separated files. The default is runs of spaces and tabs
`-v var=value`	Set a variable before the program starts. Handy for passing shell values into the program
`-f progfile`	Read the program from a file instead of the command line
`$1, $2, ...`	Not a flag. These are the first field, second field and so on of the current line. $0 is the whole line
`NR`	Not a flag. A built-in variable holding the current line number, counted across all input. NR>1 skips a header row
`NF`	Not a flag. A built-in variable holding the number of fields on the current line. $NF is the last field
`BEGIN, END`	Not flags. BEGIN { ... } runs once before any input. END { ... } runs once after the last line, which is where totals are printed

Examples

Print the first column of a CSV file

awk -F, '{print $1}' data.csv

name
ana
bo
cy
dee

Print two columns and skip the header row

awk -F, 'NR>1 {print $1, $3}' data.csv

ana 7
bo 3
cy 9
dee 3

Show only rows where the score is above 5

awk -F, 'NR>1 && $3>5' data.csv

ana,red,7
cy,red,9

Add up the score column

awk -F, 'NR>1 {sum+=$3} END {print sum}' data.csv

Total score for each team

awk -F, 'NR>1 {total[$2]+=$3} END {for (t in total) print t, total[t]}' data.csv | sort

blue 6
red 16

Pass a shell value in with -v

awk -F, -v min=5 'NR>1 && $3>min {print $1}' data.csv

ana
cy

Notes

Put the program in single quotes. Otherwise the shell expands $1 and other symbols before awk sees them.
A bare pattern with no braces prints matching lines. Braces with no pattern run on every line.
-F, splits on every comma, so quoted CSV fields that contain commas will be split wrongly. awk does not understand CSV quoting.
Comparisons treat fields that look like numbers as numbers, and everything else as text. A header such as score is text, which is why NR>1 is used to skip it.
The awk on macOS is the BSD (one true) awk. GNU awk (gawk) on Linux adds extras such as -i inplace and gensub(). Stick to the basics if a script must run on both.

awk Process text by fields and patterns

Synopsis

Common options

Examples

Notes

See also

`awk` Process text by fields and patterns