Awk turns text into columns and rows without learning a new language

Awk is a command-line tool that reads text files line by line and reorganizes them into table format. You give it rules about where to split each line, and it outputs the pieces in columns. Unlike spreadsheet software, awk works on plain text files and runs from the terminal — useful when you have thousands of log entries, network data, or configuration files that need organizing into readable columns.

The basic idea is straightforward: awk finds a separator (usually a space or comma), splits each line at that separator, and lets you print specific pieces in whatever order you want. If you have a file where each line contains a name, an IP address, and a timestamp all jumbled together, awk can extract just the IP address and name, arrange them side by side, and add headers so the output looks like a real table.

Key Takeaways

  • Awk reads text files line by line, splits each line at a separator you specify, and prints the pieces as columns.
  • The basic syntax is awk 'pattern { action }' filename, where the pattern decides which lines to process and the action decides what to print.
  • Use the -F flag to tell awk what character separates the fields in your file, such as a comma or colon.
  • Built-in variables like NF (number of fields) and NR (number of records) let you count columns and rows without writing extra code.
  • The printf function formats output into aligned columns with fixed widths, making tables readable.

Understanding fields, records, and separators

Awk thinks of your file as a collection of records (lines) and fields (pieces within each line). By default, awk treats any whitespace — spaces, tabs, or newlines — as a field separator. So if a line reads 192.168.1.5 eth0 active, awk sees three fields: the IP address, the interface name, and the status.

When your data uses a different separator, you tell awk with the -F flag. If your file is comma-separated (like a CSV export), you run awk -F, '{ print $1, $3 }' filename to print the first and third fields from each line. The dollar sign ($) means "the field at this position" — $1 is the first field, $2 is the second, and so on. $0 means the entire line.

A practical example: suppose you have a file called network.log where each line contains an IP address, a hostname, and a timestamp, separated by colons. Running awk -F: '{ print $1, $2 }' network.log prints the IP and hostname from each line, separated by a space in the output.

The basic awk syntax for printing columns

The simplest awk command has the structure awk 'action' filename. The action goes in single quotes and tells awk what to do with each line. The most common action is print, which outputs fields or text.

To print specific fields in a specific order, list them separated by commas inside the print statement. Commas tell awk to separate the output with spaces (or whatever you set as the output field separator). For example, awk '{ print $2, $1, $3 }' data.txt prints the second field first, then the first, then the third — reversing the order of the first two columns.

You can also add literal text by putting it in quotes. The command awk '{ print "IP: " $1 " Status: " $3 }' network.log prints the first field with the label "IP: " before it and the third field with "Status: " before it. This makes the output more readable when you are building a table by hand.

Adding headers and formatting columns with printf

A table needs headers — the column names at the top. Awk lets you run code before processing any lines using the BEGIN block. The command awk 'BEGIN { print "IP Address Hostname" } { print $1, $2 }' network.log prints the header once, then prints the data from each line.

For aligned, readable columns, use printf instead of print. Printf lets you specify the width of each column and how to align the text. The format string %-15s %s\n means "print the first field left-aligned in 15 characters, then the second field, then a newline." The percent sign (%) starts a format specifier, the dash (-) means left-align, the number is the width, and the letter (s for string, d for number) is the data type.

A complete example:

awk 'BEGIN { printf "%-15s %-20s %s\n", "IP Address", "Hostname", "Status" } { printf "%-15s %-20s %s\n", $1, $2, $3 }' network.log

This prints a header row with three columns of fixed widths, then prints each line of data aligned in the same columns. The output looks like a proper table, not a jumbled list.

Filtering lines with patterns

You often want to print only certain lines — for example, only the rows where the status is "active" or only lines containing a specific IP address. Awk lets you add a pattern before the action to decide which lines to process.

The pattern goes before the curly braces. awk '$3 == "active" { print $1, $2 }' network.log prints only lines where the third field equals "active". The double equals (==) means "compare for equality." You can also use != (not equal), > (greater than), < (less than), and ~ (matches a pattern).

To match text patterns, use the tilde (~). The command awk '$2 ~ /eth/ { print }' network.log prints only lines where the second field contains the text "eth" — useful for filtering hostnames or interface names. The forward slashes (/) mark the beginning and end of the pattern to search for.

Counting rows and using built-in variables

Awk provides built-in variables that track information about your data. NR is the record number (which line you are on), and NF is the number of fields in the current line. These are useful for adding row numbers to your table or for checking that every line has the expected number of columns.

To add a row number to each line, use awk '{ print NR, $1, $2 }' network.log. This prints the line number, then the first field, then the second field. To print only lines that have exactly three fields, use awk 'NF == 3 { print }' network.log.

You can also count totals. The command awk 'END { print NR }' network.log prints the total number of lines in the file. The END block runs after all lines have been processed, so NR at that point holds the final count. This is useful for adding a summary row to your table showing how many records were processed.

Combining multiple conditions and actions

Real-world data often requires more complex rules. You can combine patterns with logical operators: && (and), || (or), and ! (not). The command awk '$3 == "active" && $1 ~ /192/ { print }' network.log prints only lines where the third field is "active" AND the first field contains "192".

You can also run multiple actions on the same line. Use semicolons to separate them, or put each action in its own block. For example, awk '{ count++ } END { print "Total records:", count }' network.log increments a counter for each line, then prints the total at the end. The variable count starts at zero automatically (awk initializes undefined variables to 0 or empty string).

A practical example that combines several techniques:

awk -F: 'BEGIN { printf "%-15s %-20s\n", "IP", "Host" } $3 == "active" { printf "%-15s %-20s\n", $1, $2 } END { print "---"; print NR " total lines processed" }' network.log

This reads a colon-separated file, prints a header, prints only active records in aligned columns, and finishes with a summary line showing how many lines were read.

Frequently Asked Questions

What is the difference between print and printf in awk?

Print outputs fields separated by spaces and adds a newline at the end automatically. Printf gives you control over spacing, alignment, and column width using format strings like %-15s. Use print for quick output and printf when you need a formatted table.

How do I handle files where the separator is a space but some fields contain spaces?

If your data is inconsistent, use a more specific separator with -F. For example, if fields are separated by multiple spaces or tabs, use -F'[ \t]+' to treat any run of spaces or tabs as a single separator. This requires awk to support regular expressions in the -F flag, which most modern versions do.

Can awk read from multiple files at once?

Yes. List the filenames at the end of the command, and awk processes them in order as if they were one file. The command awk '{ print }' file1.txt file2.txt file3.txt prints all three files. The NR variable keeps counting across files, but the FNR variable resets for each file if you need to track position within a single file.

How do I save the table output to a new file?

Use the greater-than symbol (>) to redirect output. The command awk '{ print $1, $2 }' network.log > output.txt writes the table to a file called output.txt. Use >> to append to an existing file instead of overwriting it.

What if my data has inconsistent formatting or missing fields?

Check the number of fields with NF before printing. The command awk 'NF >= 3 { print $1, $2, $3 }' data.txt prints only lines with at least three fields, skipping broken or incomplete lines. You can also use conditional logic to print a placeholder like "N/A" when a field is missing.