iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

awk

By · Reviewed & last updated · Part of Linux & Bash Tutorial

awk is a tiny programming language for line-by-line text processing. Rules of the form pattern { action } run against each input line; fields are \$1, \$2, …, separated by \$IFS (default: whitespace).

Filter, transform, summarise in one line

EXAMPLE
# Sample data: orders.csv
# id,user,total,status
# 1,ada,9.99,paid
# 2,bo,29.99,refunded
# 3,cy,19.50,paid

# 1) Print a field
awk -F, '{ print $2 }' orders.csv         # 2nd column

# 2) Skip the header — NR is line number
awk -F, 'NR > 1 { print $2, $3 }' orders.csv

# 3) Filter rows with a condition
awk -F, 'NR > 1 && $4 == "paid" { print }' orders.csv
awk -F, 'NR > 1 && $3 > 20 { print $1, $3 }' orders.csv

# 4) Sum a column
awk -F, 'NR > 1 { total += $3 } END { print "sum=" total }' orders.csv

# 5) Group + sum (a poor-man's GROUP BY)
awk -F, 'NR > 1 { agg[$4] += $3 }
         END { for (s in agg) print s, agg[s] }' orders.csv
# paid 29.49
# refunded 29.99

# 6) Average
awk -F, 'NR > 1 { sum += $3; n++ } END { printf "avg=%.2f\n", sum/n }' orders.csv

# 7) Transform — emit JSON-ish output
awk -F, 'NR > 1 {
    printf "{\"id\":%s,\"user\":\"%s\",\"total\":%s}\n", $1, $2, $3
}' orders.csv

# 8) Common log parsing — Apache/Nginx access log
awk '{ print $1 }' access.log | sort | uniq -c | sort -rn | head
# top-IP report

# 9) Status-code frequency
awk '{ codes[$9]++ } END { for (c in codes) print c, codes[c] }' access.log

# 10) Multi-rule script — patterns + END
awk '
    /ERROR/    { errors++ }
    /WARN/     { warns++ }
    END {
        printf "errors: %d, warns: %d\n", errors, warns
    }
' /var/log/app.log

# 11) Built-in variables
# NR  : current line number
# NF  : number of fields on current line
# FS  : input field separator (default: whitespace)
# OFS : output field separator (default: space)
# RS  : input record separator (default: newline)
# ORS : output record separator
awk 'BEGIN { FS=","; OFS="|" } { $1=$1; print }' orders.csv

# 12) Pull a specific field range
awk '{ for (i=3; i<=NF; i++) printf "%s%s", $i, (i<NF ? OFS : ORS) }' file

# 13) Compare to alternatives
# cut    : faster, fixed-width / single-delimiter
# perl   : full Perl regex + more language
# python : you've got bigger logic — reach for a script

Why it matters

awk is the Unix scalpel for “sum/count/group by a column” tasks. Knowing 5 idioms (NR, NF, -F, group-into-array, END) covers 80% of log-processing one-liners you’ll write.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
awk -F, '{ print $1, $3 }' data.csv
awk 'NR > 1 { sum += $2 } END { print sum }' file
Try it Yourself »

Discussion

Loading…