Why Build Your Own Linux?
Goal: Understand the engineering reasons to build a custom Linux image in embedded systems.
For Hands-On Practice
Related tutorials: Buildroot Mini-Linux | Buildroot SDL2 Image
Prototype on stock distro is fine, but product constraints appear:
- boot too slow
- image too large
- too many unnecessary services
- hard to reproduce field behavior
This is where custom Linux images become necessary.
1. Boot Time
A stock Raspberry Pi OS takes 15–35 seconds to boot because it loads hundreds of packages, dozens of services, and a full desktop environment. Most of that is irrelevant to an embedded product that runs a single application. By building a custom image that contains only what your product needs, you can cut boot time to 3–10 seconds — often without any kernel-level optimization, just by removing unnecessary user-space software.
For appliances, boot time is not a convenience metric — it is a product requirement. A point-of-sale terminal that takes 30 seconds to boot after a power cut frustrates customers. A factory display that takes a minute to recover after a watchdog reboot means a minute of lost monitoring. The Buildroot pipeline below shows how a custom image is assembled from source.
Buildroot Pipeline
graph LR
A[defconfig] -->|make config| B[.config]
B -->|make menuconfig| B
B -->|make| C[Download Sources]
C --> D[Build Toolchain]
D --> E[Build Packages]
E --> F[Build Kernel]
F --> G[Build Root Filesystem]
G --> H[sdcard.img]
style H fill:#4CAF50,color:#fffThe output sdcard.img contains everything: bootloader, kernel, device tree, and root filesystem. Flash it to an SD card and boot.
2. Reliability
Embedded devices live in environments where power can disappear at any moment — a factory power glitch, a blown fuse, a user pulling the plug. If the filesystem was being written to when power dropped, the on-disk data may be inconsistent. On a stock distribution with a writable root filesystem, this can corrupt the OS itself, leaving the device unable to boot. A read-only root filesystem eliminates this risk entirely: if the root filesystem is never written to, it cannot be corrupted.
Read‑only rootfs combined with overlayfs provides the best of both worlds. The base system image is immutable and always bootable. Runtime writes (logs, temp files, configuration changes) land in a RAM-backed overlay that is discarded on reboot, restoring the device to a known-good state. Persistent data that must survive reboots goes to a dedicated writable partition with explicit fsync() calls to ensure it is flushed to storage before power could be lost.
3. Real‑Time and Determinism
A stock distribution kernel is configured for general-purpose use: it prioritizes throughput and fairness, not timing guarantees. For a data logger that samples a sensor every millisecond, this means occasional scheduling delays where the kernel handles other work (garbage collection, network interrupts, filesystem flushes) before returning to your sampling loop. These delays cause jitter — variation in the time between samples — which degrades data quality and can break control algorithms that assume uniform sampling.
A custom kernel with PREEMPT_RT patches converts interrupt handlers into schedulable threads and replaces spinlocks with sleeping mutexes, allowing your high-priority task to preempt almost anything in the kernel. This reduces worst-case scheduling latency from milliseconds to tens of microseconds. Building a custom kernel is the only way to enable PREEMPT_RT — stock distribution kernels do not include it.
4. Footprint and Security
Image size is not just a storage concern — it directly impacts update speed, security surface, and auditability. A 4 GB stock distribution contains roughly 1,500 packages. Each package is a potential source of vulnerabilities, and each vulnerability requires tracking, patching, and testing. A 50 MB Buildroot image with 20 packages is dramatically easier to audit, faster to push over the air, and presents far fewer targets to an attacker.
Smaller images are:
- Faster to update — a 50 MB OTA update over a cellular connection takes minutes, not hours
- Easier to audit — you can list every binary on the device and explain why it is there
- Less exposed to vulnerabilities — packages you do not install cannot be exploited
Concrete Size Comparison
| Component | Raspberry Pi OS | Buildroot Minimal |
|---|---|---|
| Total image | ~4.0 GB | ~50 MB |
| Kernel | ~25 MB | ~8 MB |
| Root filesystem | ~3.5 GB | ~30 MB |
| Installed packages | ~1500 | ~20 |
| Running processes at idle | ~120 | ~5 |
The 80x reduction in image size directly translates to: - Faster SD card writes during manufacturing - Faster over-the-air updates - Smaller attack surface (fewer packages = fewer vulnerabilities) - Faster boot (less to load and initialize)
5. Build vs Install on Target
Not everything needs to be baked into the image. During development, you often need tools like gdb, strace, or tcpdump that have no place in a production build. The question is: which software should be part of the immutable image, and which can be installed temporarily on a running device?
Build into image when: - required every boot - security/reproducibility matters - startup budget is strict
Install on target when: - experimenting - debugging temporary tools - short-lived diagnostics are needed
Good products do both, with a clear boundary. The image contains everything needed for the product to function. Developer tools are installed on-demand via SSH and removed before field deployment. If you find yourself installing the same tool on every device at every deploy, it should be in the image.
6. Build Systems in Practice
Two major open-source build systems dominate embedded Linux: Buildroot and Yocto. Both take a configuration file as input and produce a complete flashable image as output — but they differ significantly in complexity, flexibility, and the size of team they are designed for. Choosing between them is one of the first architectural decisions in a new embedded Linux project, and switching later is expensive.
- Buildroot: simpler path to minimal image — ideal for learning, prototypes, and single-product teams
- Yocto: more flexible and scalable for complex products — supports multiple product variants, per-package customization, and large teams
Choose by team capability, maintenance horizon, and product complexity.
Buildroot vs Yocto: Detailed Comparison
| Aspect | Buildroot | Yocto/OpenEmbedded |
|---|---|---|
| Learning curve | Low — make menuconfig is familiar |
High — BitBake, recipes, layers |
| Build time (first) | 30-90 minutes | 2-8 hours |
| Image size (minimal) | 30-80 MB | 50-150 MB |
| Package management | None (by design) | Optional (opkg, dpkg, rpm) |
| Customization | Kconfig + overlay directories | Recipes + layers (very flexible) |
| Partial rebuilds | Limited | Excellent (shared state cache) |
| Community/support | Smaller, focused | Large, enterprise-backed |
| Best for | Learning, simple appliances | Complex products, multi-team |
Rule of thumb: Start with Buildroot for learning and prototypes. Move to Yocto when you need per-package customization, multiple product variants, or team collaboration on the build system.
7. The Toolchain
The first thing Buildroot builds is the toolchain — the set of programs that turns source code into binaries for your target hardware.
Components
| Component | What It Does | Example |
|---|---|---|
| Compiler | Translates C/C++ to target machine code | arm-linux-gnueabihf-gcc |
| Linker | Combines object files + libraries into an executable | arm-linux-gnueabihf-ld |
| C library | Provides libc functions + syscall wrappers | glibc, musl, uClibc-ng |
| Binutils | Assembler, objdump, readelf, strip | arm-linux-gnueabihf-objdump |
| Sysroot | Headers + libraries for the target (not the host) | /usr/arm-linux-gnueabihf/ |
The arm-linux-gnueabihf- prefix is the target triplet — it tells you the architecture (ARM), OS (Linux), and ABI (GNU EABI, hard-float). Every tool in the cross-toolchain carries this prefix so it doesn't conflict with your host system's native tools.
Native vs Cross Toolchain
| Native (compile on target) | Cross (compile on host for target) | |
|---|---|---|
| Speed | Slow (Pi has limited CPU/RAM) | Fast (your laptop's full power) |
| Setup | Simple — apt install gcc |
Requires cross-toolchain |
| Use case | Quick tests, scripting languages | Kernel, Buildroot, production builds |
| Industry standard | No | Yes |
For this course, Buildroot provides the cross-toolchain automatically. In industry, cross-compilation is the default — nobody compiles a production kernel on a Raspberry Pi.
C Library Trade-offs
The C library is the most impactful toolchain choice because every user-space program links against it:
| glibc | musl | uClibc-ng | |
|---|---|---|---|
| Size | ~10 MB | ~1 MB | ~600 KB |
| POSIX conformance | Full | Nearly full | Partial |
| Thread safety | Full | Full | Partial |
| Default in | Debian, Ubuntu, Buildroot (default) | Alpine, some Buildroot configs | Legacy Buildroot |
| Best for | Full-featured systems | Minimal containers, embedded | Legacy/tiny systems |
Tip
When your Buildroot image uses musl instead of glibc, some programs may behave differently (locale handling, DNS resolution, NSS modules). Always test your application on the actual target image — not just on your Debian host.
Quick Checks (In Practice)
- can you reproduce the same image bit-for-bit (or config-identical)?
- do you track image/package versions per release?
- can you explain every service enabled at boot?
- is there a rollback plan for bad images?
Mini Exercise
You are building a door access control system that reads RFID cards and controls an electric lock. The system must: - Boot in under 5 seconds - Survive power cuts - Receive remote configuration updates
List: - 3 packages to bake into the image (and why each is essential) - 3 tools to install only during development (and why they don't belong in production) - One measurable success criterion with a specific target number
Key Takeaways
- Building Linux is an engineering decision, not a hobby.
- The reasons are performance, reliability, and security.
- Custom images are often required in real products.
Hands-On
Try this in practice: Tutorial: Buildroot — build a minimal Linux image and compare it to stock Raspberry Pi OS.