Shell by Example: Reading Files POSIX + Bash

The simplest way to read a file is with cat, short for “concatenate”. Despite its name, cat is most often used to display a single file — but its real purpose is joining multiple files together.

Edit
#!/bin/sh
# Create sample files for demonstration
cat >/tmp/greeting.txt <<'EOF'
Hello, World!
Welcome to shell scripting.
EOF

cat >/tmp/farewell.txt <<'EOF'
Goodbye for now.
See you next time!
EOF

# Display a single file
echo "=== Single file ==="
cat /tmp/greeting.txt

# Concatenate two files into one stream
echo ""
echo "=== Two files combined ==="
cat /tmp/greeting.txt /tmp/farewell.txt
Output:
=== Single file ===
Hello, World!
Welcome to shell scripting.

=== Two files combined ===
Hello, World!
Welcome to shell scripting.
Goodbye for now.
See you next time!

Scripts become much more useful when they read data from files instead of hardcoding values. This lets you change behavior without editing the script itself.

A common pattern is reading configuration from a key=value file — the same format used by .env files, systemd units, and many Unix tools.

Edit
#!/bin/sh
# Create a config file
cat >/tmp/app.conf <<'EOF'
APP_NAME=My Application
APP_PORT=8080
APP_DEBUG=false
EOF

# Bad: hardcoded values buried in the script
echo "=== Hardcoded (fragile) ==="
echo "Starting My Application on port 8080..."

# Good: read values from the config file
echo ""
echo "=== From config file (flexible) ==="
while IFS='=' read -r key value; do
    case "$key" in
        APP_NAME)  app_name="$value" ;;
        APP_PORT)  app_port="$value" ;;
        APP_DEBUG) app_debug="$value" ;;
    esac
done </tmp/app.conf

echo "Starting $app_name on port $app_port (debug=$app_debug)..."
Output:
=== Hardcoded (fragile) ===
Starting My Application on port 8080...

=== From config file (flexible) ===
Starting My Application on port 8080 (debug=false)...

You often need just part of a file — the first few lines, the last few, or a specific range. Three tools cover these cases: head, tail, and sed -n.

Edit
#!/bin/sh
cat >/tmp/sample.txt <<'EOF'
Line 1: Introduction
Line 2: Background
Line 3: Methods
Line 4: Results
Line 5: Discussion
Line 6: Conclusion
EOF

# head -n N prints the first N lines
echo "=== First 2 lines (head) ==="
head -n 2 /tmp/sample.txt

# tail -n N prints the last N lines
echo ""
echo "=== Last 2 lines (tail) ==="
tail -n 2 /tmp/sample.txt

# sed -n 'X,Yp' prints lines X through Y
# The -n flag suppresses default output, and p prints
# only the matched range.
echo ""
echo "=== Lines 3-5 (sed) ==="
sed -n '3,5p' /tmp/sample.txt
Output:
=== First 2 lines (head) ===
Line 1: Introduction
Line 2: Background

=== Last 2 lines (tail) ===
Line 5: Discussion
Line 6: Conclusion

=== Lines 3-5 (sed) ===
Line 3: Methods
Line 4: Results
Line 5: Discussion

When files are large, you rarely want to read everything. grep searches for matching lines and wc counts lines, words, and characters — letting you inspect a file without reading it all.

Edit
#!/bin/sh
cat >/tmp/log.txt <<'EOF'
INFO  Server started
ERROR Cannot connect to database
INFO  Request received from 10.0.0.1
WARN  Slow query detected (3.2s)
ERROR Timeout waiting for response
INFO  Request completed
EOF

# grep prints lines matching a pattern
echo "=== Errors only (grep) ==="
grep "ERROR" /tmp/log.txt

# grep -c counts matches instead of printing them
echo ""
echo "=== Error count ==="
echo "$(grep -c "ERROR" /tmp/log.txt) errors found"

# wc gives line, word, and byte counts
echo ""
echo "=== File statistics (wc) ==="
lines=$(wc -l </tmp/log.txt)
words=$(wc -w </tmp/log.txt)
chars=$(wc -c </tmp/log.txt)
echo "Lines: $lines"
echo "Words: $words"
echo "Bytes: $chars"
Output:
=== Errors only (grep) ===
ERROR Cannot connect to database
ERROR Timeout waiting for response

=== Error count ===
2 errors found

=== File statistics (wc) ===
Lines: 6
Words: 26
Bytes: 183

The standard pattern for processing a file one line at a time is while IFS= read -r line. The two flags matter:

  • IFS= prevents stripping leading/trailing whitespace
  • -r prevents backslash sequences like \n from being interpreted

Without them, data can silently change as you read it.

Edit
#!/bin/sh
cat >/tmp/items.txt <<'EOF'
  indented line
line with \backslashes\
normal line
EOF

# Without the flags — whitespace stripped, backslashes eaten
echo "=== Without IFS= and -r (broken) ==="
while read line; do
    echo "  [$line]"
done </tmp/items.txt

# With the flags — data preserved exactly
echo ""
echo "=== With IFS= read -r (correct) ==="
while IFS= read -r line; do
    echo "  [$line]"
done </tmp/items.txt

# Adding line numbers is a common variant
echo ""
echo "=== With line numbers ==="
n=1
while IFS= read -r line; do
    echo "  $n: $line"
    n=$((n + 1))
done </tmp/items.txt
Output:
=== Without IFS= and -r (broken) ===
  [indented line]
  [line with backslashesnormal line]

=== With IFS= read -r (correct) ===
  [  indented line]
  [line with \backslashes\]
  [normal line]

=== With line numbers ===
  1:   indented line
  2: line with \backslashes\
  3: normal line

Many Unix files use delimiters to separate fields — /etc/passwd uses colons, CSVs use commas. Setting IFS to the delimiter lets read split each line into separate variables automatically.

Edit
#!/bin/sh
cat >/tmp/staff.csv <<'EOF'
alice:30:engineer
bob:25:designer
carol:35:manager
EOF

# IFS=: tells read to split on colons
echo "=== Parsing colon-delimited data ==="
while IFS=: read -r name age role; do
    echo "  $name is $age, works as $role"
done </tmp/staff.csv

# A subtle bug: if the file's last line has no trailing
# newline, `read` returns false and the line is skipped.
# Adding `|| [ -n "$line" ]` catches this case.
echo ""
echo "=== Handling missing final newline ==="
printf "red:apple\ngreen:pear" >/tmp/colors.txt
while IFS=: read -r color fruit || [ -n "$color" ]; do
    echo "  $color -> $fruit"
done </tmp/colors.txt
Output:
=== Parsing colon-delimited data ===
  alice is 30, works as engineer
  bob is 25, works as designer
  carol is 35, works as manager

=== Handling missing final newline ===
  red -> apple
  green -> pear

Sometimes you need the entire file in a variable rather than processing it line by line. Command substitution with cat is the standard approach.

Note: command substitution strips trailing newlines, so this works best for content you will process further rather than reproduce byte-for-byte.

Edit
#!/bin/sh
cat >/tmp/message.txt <<'EOF'
Hello from the file.
This is line two.
EOF

content=$(cat /tmp/message.txt)
echo "=== File in a variable ==="
echo "$content"

# POSIX sh has no arrays, but you can store lines in
# positional parameters using `set --`. This gives you
# `$1`, `$2`, etc. for each line.
echo ""
echo "=== Lines as positional parameters ==="
printf "apple\nbanana\ncherry\n" >/tmp/fruits.txt

set --
while IFS= read -r line; do
    set -- "$@" "$line"
done </tmp/fruits.txt

echo "Count: $#"
echo "First: $1"
echo "Last:  $3"
echo "All:   $*"
Output:
=== File in a variable ===
Hello from the file.
This is line two.

=== Lines as positional parameters ===
Count: 3
First: apple
Last:  cherry
All:   apple banana cherry

Before reading a file, check that it exists. Wrapping the check in a function makes it reusable across your script.

For more control, file descriptors let you open a file once and read from it at your own pace — useful when you need the first few lines without processing the whole file.

Edit
#!/bin/sh
# A reusable "safe read" function
read_file() {
    if [ ! -f "$1" ]; then
        echo "Error: $1 not found" >&2
        return 1
    fi
    cat "$1"
}

cat >/tmp/data.txt <<'EOF'
first
second
third
EOF

echo "=== Safe read (file exists) ==="
read_file /tmp/data.txt

echo ""
echo "=== Safe read (file missing) ==="
read_file /tmp/no_such_file.txt 2>&1 || true

# File descriptors: open with exec, read lines, then close
echo ""
echo "=== File descriptors ==="
exec 3</tmp/data.txt
read -r line1 <&3
read -r line2 <&3
exec 3<&-
echo "First line:  $line1"
echo "Second line: $line2"
Output:
=== Safe read (file exists) ===
first
second
third

=== Safe read (file missing) ===
Error: /tmp/no_such_file.txt not found

=== File descriptors ===
First line:  first
Second line: second

Not all input comes from files. Programs often read from standard input (stdin), which can come from a pipe, a redirect, or the terminal.

read -r reads one line from stdin. cat (with no arguments) or cat - copies all of stdin to stdout, which is useful in pipelines.

Edit
#!/bin/sh
# Reading piped input line by line
echo "=== Pipe into while-read ==="
printf "one\ntwo\nthree\n" | while IFS= read -r line; do
    echo "  got: $line"
done

# read -r grabs a single line from stdin
echo ""
echo "=== Single read from a pipe ==="
result=$(echo "hello from pipe" | { read -r val; echo "$val"; })
echo "  $result"

# cat - reads all of stdin (the dash is optional but
# makes the intent explicit in scripts)
echo ""
echo "=== cat - passes stdin through ==="
echo "piped through cat" | cat -
Output:
=== Pipe into while-read ===
  got: one
  got: two
  got: three

=== Single read from a pipe ===
  hello from pipe

=== cat - passes stdin through ===
piped through cat

Bash

Bash extends the read builtin with useful flags not available in POSIX sh:

  • read -t N sets a timeout (seconds)
  • read -n N reads exactly N characters
  • read -p "text" displays a prompt

Bash also provides mapfile (alias readarray) to load an entire file into an array in one step.

Edit
#!/bin/bash
# read -t: timeout after N seconds
echo "=== read -t (timeout) ==="
read -r -t 5 input <<<"hello from timeout"
echo "  Got: $input"

# read -n: read exactly N characters
echo ""
echo "=== read -n (single character) ==="
read -r -n 1 char <<<"y"
echo ""
echo "  Got: $char"

# read -p: display a prompt string
echo ""
echo "=== read -p (prompt) ==="
read -r -p "Enter value: " value <<<"42"
echo "  Got: $value"

# mapfile loads a whole file into an array at once —
# faster and simpler than a while-read loop
echo ""
echo "=== mapfile (array loading) ==="
printf "alpha\nbeta\ngamma\n" >/tmp/greek.txt
mapfile -t lines </tmp/greek.txt
echo "  Count: ${#lines[@]}"
echo "  First: ${lines[0]}"
echo "  Last:  ${lines[2]}"
Output:
=== read -t (timeout) ===
  Got: hello from timeout

=== read -n (single character) ===

  Got: y

=== read -p (prompt) ===
  Got: 42

=== mapfile (array loading) ===
  Count: 3
  First: alpha
  Last:  gamma

« Environment Variables | Writing Files »