Shell Scripting for Embedded Linux

Time: 60 min | Prerequisites: Exploring Linux (Section 0 — you must have completed the scripting exercises: shebang, variables, conditionals, loops, functions, grep/awk, arguments, exit codes) | Theory companion: Linux Fundamentals, Section 10: Shell Scripting

---

Learning Objectives

By the end of this lab you will be able to:

• Build a continuous sensor logger that writes CSV data
• Handle errors and cleanup gracefully with trap
• Parse kernel logs to diagnose driver and hardware issues
• Create a systemd timer to run scripts on a schedule
• Use signals (SIGUSR1) for communication between running scripts

Before You Start

This tutorial builds on the scripting skills from Exploring Linux, Section 0. You should already be comfortable with: shebang (#!/bin/bash), set -euo pipefail, variables, command substitution $(), conditionals (if/elif/else), for and while loops, functions, source, arguments ($1, $@, $#), exit codes ($?), and basic grep/awk.

All exercises below run on the Raspberry Pi via SSH.

---

1. Continuous Sensor Logger

In Exploring Linux you wrote scripts that run once and exit. Real embedded systems need scripts that run continuously — polling sensors, logging data, and reacting to changes.

Step 1: Temperature Logger with CSV Output

cat << 'EOF' > ~/scripts/temp_logger.sh
#!/bin/bash
set -euo pipefail

LOGFILE=~/temp_log.csv
INTERVAL=1

# Write CSV header
echo "timestamp,epoch,temp_mC,temp_C" > "$LOGFILE"
echo "Logging to $LOGFILE every ${INTERVAL}s (Ctrl+C to stop)..."

while true; do
    EPOCH=$(date +%s)
    TIMESTAMP=$(date +%H:%M:%S)
    TEMP=$(cat /sys/class/thermal/thermal_zone0/temp)
    TEMP_C=$(( TEMP / 1000 ))
    echo "${TIMESTAMP},${EPOCH},${TEMP},${TEMP_C}" >> "$LOGFILE"
    echo "${TIMESTAMP} ${TEMP_C}°C"
    sleep "$INTERVAL"
done
EOF
chmod +x ~/scripts/temp_logger.sh

Run it, let it log for 10–15 seconds, then press Ctrl+C:

~/scripts/temp_logger.sh

Check the log:

cat ~/temp_log.csv
wc -l ~/temp_log.csv

You should see a CSV with one row per second. Each row has a timestamp, epoch time, raw temperature in millidegrees, and temperature in Celsius.

Step 2: Iterate Over Hardware Devices

# List all I2C buses with permissions
for dev in /dev/i2c-*; do
    echo "Found: $dev (permissions: $(stat -c '%a' "$dev"))"
done

# List all thermal zones
for zone in /sys/class/thermal/thermal_zone*/temp; do
    LABEL=$(dirname "$zone")
    echo "$(basename "$LABEL"): $(( $(cat "$zone") / 1000 ))°C"
done

Step 3: Temperature Threshold Alert

cat << 'EOF' > ~/scripts/temp_alert.sh
#!/bin/bash
set -euo pipefail

THRESHOLD=70000  # 70°C in millidegrees

TEMP=$(cat /sys/class/thermal/thermal_zone0/temp)
TEMP_C=$(( TEMP / 1000 ))

if [[ $TEMP -gt $THRESHOLD ]]; then
    echo "WARNING: CPU temperature is ${TEMP_C}°C (threshold: $(( THRESHOLD / 1000 ))°C)" >&2
    exit 1
elif [[ $TEMP -gt $(( THRESHOLD - 10000 )) ]]; then
    echo "CAUTION: CPU temperature approaching threshold: ${TEMP_C}°C"
else
    echo "OK: CPU temperature is ${TEMP_C}°C"
fi
EOF
chmod +x ~/scripts/temp_alert.sh

~/scripts/temp_alert.sh
echo $?             # 0 = OK or CAUTION, 1 = WARNING

Checkpoint 1

Question	Your Answer
Lines in your CSV after 10 seconds
Number of I2C buses found
Current CPU temperature and alert status

---

2. Error Handling with trap

The logger from Step 1 has a problem: when you press Ctrl+C, it just dies. There is no cleanup, no summary, no graceful shutdown. In embedded systems, abrupt termination can corrupt log files or leave hardware in a bad state.

Step 4: Robust Logger with Cleanup

trap lets you run a function when a script receives a signal (like Ctrl+C) or exits:

cat << 'EOF' > ~/scripts/temp_logger_safe.sh
#!/bin/bash
set -euo pipefail

LOGFILE=~/temp_log_safe.csv
SENSOR=/sys/class/thermal/thermal_zone0/temp
INTERVAL=1
RUNNING=true

cleanup() {
    RUNNING=false
    local LINES
    LINES=$(wc -l < "$LOGFILE" 2>/dev/null || echo 0)
    echo ""
    echo "Stopped. Logged $LINES entries to $LOGFILE"
}
trap cleanup EXIT

# Pre-flight checks
if [[ ! -r "$SENSOR" ]]; then
    echo "ERROR: Cannot read $SENSOR" >&2
    exit 1
fi

echo "timestamp,temp_C" > "$LOGFILE"
echo "Logging to $LOGFILE (Ctrl+C to stop)..."

while $RUNNING; do
    TEMP=$(cat "$SENSOR" 2>/dev/null) || { echo "Sensor read failed" >&2; sleep "$INTERVAL"; continue; }
    echo "$(date +%H:%M:%S),$(( TEMP / 1000 ))" >> "$LOGFILE"
    sleep "$INTERVAL"
done
EOF
chmod +x ~/scripts/temp_logger_safe.sh

Run it for a few seconds, then press Ctrl+C:

~/scripts/temp_logger_safe.sh

Notice how the trap fires on exit and reports the line count — no data corruption, no silent death.

Note

How trap works:

Syntax	Meaning
trap cleanup EXIT	Run cleanup when the script exits (any reason)
trap cleanup INT	Run cleanup on Ctrl+C (SIGINT) only
trap cleanup INT TERM EXIT	Run cleanup on interrupt, kill, or exit

EXIT is the safest choice — it fires regardless of how the script ends (normal exit, error from set -e, or signal). Use trap whenever your script creates temporary files, opens hardware, or writes log files.

Section 5 extends trap to inter-script communication using SIGUSR1.

Checkpoint 2

Question	Your Answer
What happens when you press Ctrl+C on the safe logger?
What does trap cleanup EXIT do?
What happens if the sensor file is not readable?

---

3. Log Analysis

Kernel logs (dmesg) are your primary debugging tool on embedded Linux. Knowing how to filter and parse them quickly is essential.

Step 5: Parse Kernel Messages

# Find all driver probe messages
dmesg | grep -i "probe" | head -10

# Extract just the module names
dmesg | grep -i "probe" | awk '{for(i=1;i<=NF;i++) if($i ~ /probe/) print $(i-1)}' | sort -u

# Count messages by facility
dmesg | awk '{print $2}' | sort | uniq -c | sort -rn | head -10

Step 6: Build a Log Analyzer Script

cat << 'EOF' > ~/scripts/log_analyzer.sh
#!/bin/bash
set -euo pipefail

echo "=== Kernel Log Analysis ==="
echo ""

# Error summary
ERROR_COUNT=$(dmesg | grep -ci "error" || true)
WARN_COUNT=$(dmesg | grep -ci "warn" || true)
echo "Errors:   $ERROR_COUNT"
echo "Warnings: $WARN_COUNT"
echo ""

# Driver probes
echo "--- Driver Probes ---"
dmesg | grep -i "registered\|probe" | sed 's/.*\] //' | head -10
echo ""

# Recent messages (last 10)
echo "--- Last 10 Messages ---"
dmesg | tail -10 | sed 's/.*\] //'
EOF
chmod +x ~/scripts/log_analyzer.sh

~/scripts/log_analyzer.sh

You should see error/warning counts from the kernel log and the most recent messages. On a healthy Pi, error count is typically 0–5 (some are harmless boot-time messages).

Tip

The || true after grep -c prevents set -e from killing the script when grep finds zero matches (grep returns exit code 1 for no matches).

Checkpoint 3

Question	Your Answer
Error count in dmesg
Warning count in dmesg
How many unique drivers probed?

---

4. systemd Timers

Instead of running a logger in an infinite loop, you can use systemd to execute a script on a schedule — more reliable, survives reboots, and integrates with the system journal.

Step 7: Create a Service Unit

sudo bash -c 'cat << EOF > /etc/systemd/system/temp-log.service
[Unit]
Description=Log CPU temperature to CSV

[Service]
Type=oneshot
ExecStart=/bin/bash -c '"'"'echo "\$(date +%%H:%%M:%%S),\$(( \$(cat /sys/class/thermal/thermal_zone0/temp) / 1000 ))" >> /home/'"$(whoami)"'/temp_timer.csv'"'"'
User='"$(whoami)"'
EOF'

Tip

The User= line ensures the script runs as your user, not root. The command uses %% to escape % in systemd unit files.

Step 8: Create a Timer Unit

sudo bash -c 'cat << EOF > /etc/systemd/system/temp-log.timer
[Unit]
Description=Log CPU temperature every 5 minutes

[Timer]
OnBootSec=1min
OnUnitActiveSec=5min
AccuracySec=1s

[Install]
WantedBy=timers.target
EOF'

Step 9: Enable and Test

# Reload systemd to pick up new files
sudo systemctl daemon-reload

# Start the timer
sudo systemctl start temp-log.timer

# Check it is active
systemctl list-timers | grep temp-log

# Manually trigger the service to verify it works
sudo systemctl start temp-log.service

# Check the log
cat ~/temp_timer.csv

You should see at least one line with timestamp and temperature. The timer will add a new line every 5 minutes automatically.

Note

systemd timers are more reliable than cron for embedded systems — they integrate with the journal, support boot-relative timing, and handle missed runs (e.g., if the device was powered off).

Step 10: Clean Up (Optional)

When you are done experimenting:

sudo systemctl stop temp-log.timer
sudo systemctl disable temp-log.timer
sudo rm /etc/systemd/system/temp-log.service /etc/systemd/system/temp-log.timer
sudo systemctl daemon-reload

Checkpoint 4

Question	Your Answer
Timer shows in list-timers? (yes/no)
Entries in temp_timer.csv after manual trigger
What does OnUnitActiveSec=5min mean?

---

5. Signals Between Scripts

So far, trap has handled cleanup on exit. But signals can also be used for communication between running scripts — one script sends a signal to another, triggering a specific action without restarting.

Step 11: Signal-Driven Config Reload

A temperature logger that re-reads its configuration when it receives SIGUSR1:

cat << 'EOF' > ~/scripts/logger.conf
INTERVAL=2
EOF

cat << 'EOF' > ~/scripts/signal_logger.sh
#!/bin/bash
set -euo pipefail

CONF=~/scripts/logger.conf
LOGFILE=~/signal_log.csv
PIDFILE=/tmp/signal_logger.pid

# Write PID file so other scripts can find us
echo $$ > "$PIDFILE"
trap 'rm -f "$PIDFILE"; echo "Stopped."; exit 0' EXIT TERM

# Load config
source "$CONF"
echo "Started (PID $$), interval=${INTERVAL}s"

# Reload config on SIGUSR1
reload_config() {
    source "$CONF"
    echo "Config reloaded: interval=${INTERVAL}s"
}
trap reload_config USR1

echo "timestamp,temp_C" > "$LOGFILE"

while true; do
    TEMP=$(cat /sys/class/thermal/thermal_zone0/temp 2>/dev/null || echo 0)
    echo "$(date +%H:%M:%S),$(( TEMP / 1000 ))" >> "$LOGFILE"
    sleep "$INTERVAL"
done
EOF
chmod +x ~/scripts/signal_logger.sh

Test the Reload Pattern

# Start the logger in the background
~/scripts/signal_logger.sh &

# Let it log a few entries at 2-second intervals
sleep 6

# Change the interval
sed -i 's/INTERVAL=2/INTERVAL=5/' ~/scripts/logger.conf

# Send SIGUSR1 to reload config (no restart needed!)
kill -USR1 $(cat /tmp/signal_logger.pid)

# Let it log at the new interval
sleep 12

# Stop it cleanly
kill -TERM $(cat /tmp/signal_logger.pid)

# Check the log
cat ~/signal_log.csv

You should see entries 2 seconds apart, then 5 seconds apart after the reload — without any gap or restart.

Key Patterns

Pattern	How	Why
PID file	echo $$ > /tmp/app.pid	Other scripts can find and signal this process
Check if alive	kill -0 $PID	Signal 0 sends nothing but checks if the PID exists
Reload config	kill -USR1 $PID	Reconfigure without restart or data loss

Signal Table for Scripts

Signal	Typical Meaning	Example
SIGUSR1	Custom: reload config	kill -USR1 $(cat /tmp/app.pid)
SIGUSR2	Custom: rotate log file	kill -USR2 $(cat /tmp/app.pid)
SIGHUP	Traditional: daemon reload	systemctl reload nginx sends this
SIGTERM	Clean shutdown	systemctl stop sends this

Tip

This is the daemon reload pattern. systemctl reload nginx sends SIGHUP to the nginx master process, which re-reads its config without dropping connections. Now you understand what happens underneath.

Checkpoint 5

Question	Your Answer
PID of the signal_logger process
Log interval before and after SIGUSR1
What does kill -0 $PID do?

---

What Just Happened?

You built on the scripting fundamentals from Exploring Linux and applied them to real embedded tasks — continuous sensor logging, robust error handling, log analysis, scheduled automation with systemd, and inter-process signal communication. These patterns appear in every embedded Linux project:

• Continuous loggers → data collection and monitoring
• trap cleanup → scripts that fail safely when running unattended
• Log analysis → debugging driver issues and system problems
• systemd timers → reliable scheduled tasks without cron
• Signal communication → scripts that reconfigure without restart (daemon pattern)

---

Challenges

Challenge 1: Hardware Health Report

Write ~/scripts/health_report.sh that outputs a formatted report including:

• Hostname, kernel version, uptime
• CPU temperature with warning levels (OK / CAUTION / WARNING)
• Memory usage (total, free, percentage)
• Disk usage for the root partition (df -h /)
• Count of running processes and systemd services
• List of detected I2C and SPI devices
• Summary: total checks passed / total checks

Use functions for each section. Output should look professional (aligned columns, clear headers).

Challenge 2: Log Rotation Script

Write ~/scripts/rotate_logs.sh that:

• Finds all .csv files in ~/ older than 7 days
• Compresses them with gzip
• Deletes compressed files older than 30 days
• Logs what it did to ~/scripts/rotation.log

Hints: find ~/ -name "*.csv" -mtime +7, gzip, find -name "*.gz" -mtime +30 -delete.

Challenge 3: Watchdog Kicker

Write ~/scripts/watchdog_kicker.sh that:

• Opens /dev/watchdog (if it exists)
• Writes to it every 10 seconds to prevent system reset
• Reads and logs CPU temperature each kick
• Exits cleanly on Ctrl+C (closing the watchdog properly)

Warning

Only attempt Challenge 3 if you understand the hardware watchdog — opening /dev/watchdog starts a countdown. If your script crashes without closing it, the system will reboot. Test carefully.

---

Deliverable

• ~/scripts/temp_logger_safe.sh — robust sensor logger with error handling
• ~/scripts/log_analyzer.sh — kernel log analysis script
• temp_timer.csv — at least one entry from the systemd timer
• (Optional) challenge scripts

---

Next Steps

With scripting skills in hand, you can automate any embedded Linux task. Next, put pixels on screen: Tutorial: Framebuffer Basics, or build a complete appliance: Tutorial: Data Logger Appliance.

---

Back to Course Overview