45,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Master the techniques needed to build great, efficient embedded devices on Linux
About This Book
- Discover how to build and configure reliable embedded Linux devices
- This book has been updated to include Linux 4.9 and Yocto Project 2.2 (Morty)
- This comprehensive guide covers the remote update of devices in the field and power management
Who This Book Is For
If you are an engineer who wishes to understand and use Linux in embedded devices, this book is for you. It is also for Linux developers and system programmers who are familiar with embedded systems and want to learn and program the best in class devices. It is appropriate for students studying embedded techniques, for developers implementing embedded Linux devices, and engineers supporting existing Linux devices.
What You Will Learn
- Evaluate the Board Support Packages offered by most manufacturers of a system on chip or embedded module
- Use Buildroot and the Yocto Project to create embedded Linux systems quickly and efficiently
- Update IoT devices in the field without compromising security
- Reduce the power budget of devices to make batteries last longer
- Interact with the hardware without having to write kernel device drivers
- Debug devices remotely using GDB, and see how to measure the performance of the systems using powerful tools such as perk, ftrace, and valgrind
- Find out how to configure Linux as a real-time operating system
In Detail
Embedded Linux runs many of the devices we use every day, from smart TVs to WiFi routers, test equipment to industrial controllers - all of them have Linux at their heart. Linux is a core technology in the implementation of the inter-connected world of the Internet of Things.
The comprehensive guide shows you the technologies and techniques required to build Linux into embedded systems. You will begin by learning about the fundamental elements that underpin all embedded Linux projects: the toolchain, the bootloader, the kernel, and the root filesystem. You'll see how to create each of these elements from scratch, and how to automate the process using Buildroot and the Yocto Project.
Moving on, you'll find out how to implement an effective storage strategy for flash memory chips, and how to install updates to the device remotely once it is deployed. You'll also get to know the key aspects of writing code for embedded Linux, such as how to access hardware from applications, the implications of writing multi-threaded code, and techniques to manage memory in an efficient way. The final chapters show you how to debug your code, both in applications and in the Linux kernel, and how to profile the system so that you can look out for performance bottlenecks.
By the end of the book, you will have a complete overview of the steps required to create a successful embedded Linux system.
Style and approach
This book is an easy-to-follow and pragmatic guide with in-depth analysis of the implementation of embedded devices. It follows the life cycle of a project from inception through to completion, at each stage giving both the theory that underlies the topic and practical step-by-step walkthroughs of an example implementation.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 657
Veröffentlichungsjahr: 2017
Ähnliche
Mastering Embedded Linux Programming
Second Edition
BIRMINGHAM - MUMBAI
Mastering Embedded Linux Programming
Second Edition
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: December 2015
Second edition: June 2017
Production reference: 1280617
ISBN 978-1-78728-328-2
www.packtpub.com
Credits
Author
Chris Simmonds
Copy Editors
Madhusudan Uchil
Stuti Shrivastava
Reviewers
Daiane Angolini
Otavio Salvador
Alex Tereschenko
Project Coordinator
Virginia Dias
Commissioning Editor
Kartikey Pandey
Proofreader
Safis Editing
Acquisition Editor
Prateek Bharadwaj
Indexer
Rekha Nair
Content Development Editor
Sharon Raj
Graphics
Kirk D'Penha
Technical Editor
Vishal Kamal Mewada
Production Coordinator
Melwyn Dsa
About the Author
Chris Simmonds is a software consultant and trainer living in southern England. He has almost two decades of experience in designing and building open-source embedded systems. He is the founder and chief consultant at 2net Ltd, which provides professional training and mentoring services in embedded Linux, Linux device drivers, and Android platform development. He has trained engineers at many of the biggest companies in the embedded world, including ARM, Qualcomm, Intel, Ericsson, and General Dynamics. He is a frequent presenter at open source and embedded conferences, including the Embedded Linux Conference and Embedded World. You can see some of his work on the Inner Penguin blog at www.2net.co.uk.
About the Reviewers
Daiane Angolini has been working with embedded Linux since 2008. She has been working as an application engineer at NXP, acting oninternal development, porting custom applications from Android, and on-customer support for i.MX architectures in areas such as Linux kernel,u-boot, Android, Yocto Project, and user-space applications. However, it was on the Yocto Project that she found her place. She has coauthored the books Embedded Linux Development with Yocto Project and Heading for the Yocto Project, and learned a lot in the process.
Otavio Salvador loves technology and started his free software activities in 1999. In 2002, he founded O.S. Systems, a company focused on embedded system development services and consultancy worldwide, creating and maintaining customized BSPs, and helping companies with their product's development challenges. This resulted in him joining the OpenEmbedded community in 2008, when he became an active contributor to the OpenEmbedded project. He has coauthored the books Embedded Linux Development with Yocto Project and Heading for the Yocto Project.
Alex Tereschenko is an embedded systems engineer by day, and an avid maker by night, who is convinced that computers can do a lot of good for people when they are interfaced with real-world objects, as opposed to just crunching data in a dusty corner. That's what's driving him in his projects, and this is why embedded systems and the Internet of Things are the topics he enjoys the most.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.comand as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787283283.
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
Starting Out
Selecting the right operating system
The players
Project life cycle
The four elements of embedded Linux
Open source
Licenses
Hardware for embedded Linux
Hardware used in this book
The BeagleBone Black
QEMU
Software used in this book
Summary
Learning About Toolchains
Introducing toolchains
Types of toolchains
CPU architectures
Choosing the C library
Finding a toolchain
Building a toolchain using crosstool-NG
Installing crosstool-NG
Building a toolchain for BeagleBone Black
Building a toolchain for QEMU
Anatomy of a toolchain
Finding out about your cross compiler
The sysroot, library, and header files
Other tools in the toolchain
Looking at the components of the C library
Linking with libraries – static and dynamic linking
Static libraries
Shared libraries
Understanding shared library version numbers
The art of cross compiling
Simple makefiles
Autotools
An example: SQLite
Package configuration
Problems with cross compiling
Summary
All About Bootloaders
What does a bootloader do?
The boot sequence
Phase 1 – ROM code
Phase 2 – secondary program loader
Phase 3 – TPL
Booting with UEFI firmware
Moving from bootloader to kernel
Introducing device trees
Device tree basics
The reg property
Labels and interrupts
Device tree include files
Compiling a device tree
Choosing a bootloader
U-Boot
Building U-Boot
Installing U-Boot
Using U-Boot
Environment variables
Boot image format
Loading images
Booting Linux
Automating the boot with U-Boot scripts
Porting U-Boot to a new board
Board-specific files
Configuring header files
Building and testing
Falcon mode
Barebox
Getting barebox
Building barebox
Using barebox
Summary
Configuring and Building the Kernel
What does the kernel do?
Choosing a kernel
Kernel development cycle
Stable and long term support releases
Vendor support
Licensing
Building the kernel
Getting the source
Understanding kernel configuration – Kconfig
Using LOCALVERSION to identify your kernel
Kernel modules
Compiling – Kbuild
Finding out which kernel target to build
Build artifacts
Compiling device trees
Compiling modules
Cleaning kernel sources
Building a kernel for the BeagleBone Black
Building a kernel for QEMU
Booting the kernel
Booting the BeagleBone Black
Booting QEMU
Kernel panic
Early user space
Kernel messages
Kernel command line
Porting Linux to a new board
A new device tree
Setting the board compatible property
Additional reading
Summary
Building a Root Filesystem
What should be in the root filesystem?
The directory layout
The staging directory
POSIX file access permissions
File ownership permissions in the staging directory
Programs for the root filesystem
The init program
Shell
Utilities
BusyBox to the rescue!
Building BusyBox
ToyBox – an alternative to BusyBox
Libraries for the root filesystem
Reducing the size by stripping
Device nodes
The proc and sysfs filesystems
Mounting filesystems
Kernel modules
Transferring the root filesystem to the target
Creating a boot initramfs
Standalone initramfs
Booting the initramfs
Booting with QEMU
Booting the BeagleBone Black
Mounting proc
Building an initramfs into the kernel image
Building an initramfs using a device table
The old initrd format
The init program
Starting a daemon process
Configuring user accounts
Adding user accounts to the root filesystem
A better way of managing device nodes
An example using devtmpfs
An example using mdev
Are static device nodes so bad after all?
Configuring the network
Network components for glibc
Creating filesystem images with device tables
Booting the BeagleBone Black
Mounting the root filesystem using NFS
Testing with QEMU
Testing with the BeagleBone Black
Problems with file permissions
Using TFTP to load the kernel
Additional reading
Summary
Selecting a Build System
Build systems
Package formats and package managers
Buildroot
Background
Stable releases and long-term support
Installing
Configuring
Running
Creating a custom BSP
U-Boot
Linux
Build
Adding your own code
Overlays
Adding a package
License compliance
The Yocto Project
Background
Stable releases and supports
Installing the Yocto Project
Configuring
Building
Running the QEMU target
Layers
BitBake and recipes
Customizing images via local.conf
Writing an image recipe
Creating an SDK
The license audit
Further reading
Summary
Creating a Storage Strategy
Storage options
NOR flash
NAND flash
Managed flash
MultiMediaCard and Secure Digital cards
eMMC
Other types of managed flash
Accessing flash memory from the bootloader
U-Boot and NOR flash
U-Boot and NAND flash
U-Boot and MMC, SD, and eMMC
Accessing flash memory from Linux
Memory technology devices
MTD partitions
MTD device drivers
The MTD character device, mtd
The MTD block device, mtdblock
Logging kernel oops to MTD
Simulating NAND memory
The MMC block driver
Filesystems for flash memory
Flash translation layers
Filesystems for NOR and NAND flash memory
JFFS2
Summary nodes
Clean markers
Creating a JFFS2 filesystem
YAFFS2
Creating a YAFFS2 filesystem
UBI and UBIFS
UBI
UBIFS
Filesystems for managed flash
Flashbench
Discard and TRIM
Ext4
F2FS
FAT16/32
Read-only compressed filesystems
squashfs
Temporary filesystems
Making the root filesystem read-only
Filesystem choices
Further reading
Summary
Updating Software in the Field
What to update?
Bootloader
Kernel
Root filesystem
System applications
Device-specific data
Components that need to be updated
The basics of software update
Making updates robust
Making updates fail-safe
Making updates secure
Types of update mechanism
Symmetric image update
Asymmetric image update
Atomic file updates
OTA updates
Using Mender for local updates
Building the Mender client
Installing an update
Using Mender for OTA updates
Summary
Interfacing with Device Drivers
The role of device drivers
Character devices
Block devices
Network devices
Finding out about drivers at runtime
Getting information from sysfs
The devices: /sys/devices
The drivers: /sys/class
The block drivers: /sys/block
Finding the right device driver
Device drivers in user space
GPIO
Handling interrupts from GPIO
LEDs
I2C
Serial Peripheral Interface (SPI)
Writing a kernel device driver
Designing a character driver interface
The anatomy of a device driver
Compiling kernel modules
Loading kernel modules
Discovering the hardware configuration
Device trees
The platform data
Linking hardware with device drivers
Additional reading
Summary
Starting Up – The init Program
After the kernel has booted
Introducing the init programs
BusyBox init
Buildroot init scripts
System V init
inittab
The init.d scripts
Adding a new daemon
Starting and stopping services
systemd
Building systemd with the Yocto Project and Buildroot
Introducing targets, services, and units
Units
Services
Targets
How systemd boots the system
Adding your own service
Adding a watchdog
Implications for embedded Linux
Further reading
Summary
Managing Power
Measuring power usage
Scaling the clock frequency
The CPUFreq driver
Using CPUFreq
Selecting the best idle state
The CPUIdle driver
Tickless operation
Powering down peripherals
Putting the system to sleep
Power states
Wakeup events
Timed wakeups from the real-time clock
Further reading
Summary
Learning About Processes and Threads
Process or thread?
Processes
Creating a new process
Terminating a process
Running a different program
Daemons
Inter-process communication
Message-based IPC
Unix (or local) sockets
FIFOs and named pipes
POSIX message queues
Summary of message-based IPC
Shared memory-based IPC
POSIX shared memory
Threads
Creating a new thread
Terminating a thread
Compiling a program with threads
Inter-thread communication
Mutual exclusion
Changing conditions
Partitioning the problem
Scheduling
Fairness versus determinism
Time-shared policies
Niceness
Real-time policies
Choosing a policy
Choosing a real-time priority
Further reading
Summary
Managing Memory
Virtual memory basics
Kernel space memory layout
How much memory does the kernel use?
User space memory layout
The process memory map
Swapping
Swapping to compressed memory (zram)
Mapping memory with mmap
Using mmap to allocate private memory
Using mmap to share memory
Using mmap to access device memory
How much memory does my application use?
Per-process memory usage
Using top and ps
Using smem
Other tools to consider
Identifying memory leaks
mtrace
Valgrind
Running out of memory
Further reading
Summary
Debugging with GDB
The GNU debugger
Preparing to debug
Debugging applications
Remote debugging using gdbserver
Setting up the Yocto Project for remote debugging
Setting up Buildroot for remote debugging
Starting to debug
Connecting GDB and gdbserver
Setting the sysroot
GDB command files
Overview of GDB commands
Breakpoints
Running and stepping
Getting information
Running to a breakpoint
Native debugging
The Yocto Project
Buildroot
Just-in-time debugging
Debugging forks and threads
Core files
Using GDB to look at core files
GDB user interfaces
Terminal user interface
Data display debugger
Eclipse
Debugging kernel code
Debugging kernel code with kgdb
A sample debug session
Debugging early code
Debugging modules
Debugging kernel code with kdb
Looking at an Oops
Preserving the Oops
Further reading
Summary
Profiling and Tracing
The observer effect
Symbol tables and compile flags
Beginning to profile
Profiling with top
Poor man's profiler
Introducing perf
Configuring the kernel for perf
Building perf with the Yocto Project
Building perf with Buildroot
Profiling with perf
Call graphs
perf annotate
Other profilers – OProfile and gprof
Tracing events
Introducing Ftrace
Preparing to use Ftrace
Using Ftrace
Dynamic Ftrace and trace filters
Trace events
Using LTTng
LTTng and the Yocto Project
LTTng and Buildroot
Using LTTng for kernel tracing
Using Valgrind
Callgrind
Helgrind
Using strace
Summary
Real-Time Programming
What is real time?
Identifying sources of non-determinism
Understanding scheduling latency
Kernel preemption
The real-time Linux kernel (PREEMPT_RT)
Threaded interrupt handlers
Preemptible kernel locks
Getting the PREEMPT_RT patches
The Yocto Project and PREEMPT_RT
High-resolution timers
Avoiding page faults
Interrupt shielding
Measuring scheduling latencies
cyclictest
Using Ftrace
Combining cyclictest and Ftrace
Further reading
Summary
Preface
Linux has been the mainstay of embedded computing for many years. And yet, there are remarkably few books that cover the topic as a whole: this book is intended to fill that gap. The term embedded Linux is not well-defined, and can be applied to the operating system inside a wide range of devices ranging from thermostats to Wi-Fi routers to industrial control units. However, they are all built on the same basic open source software. Those are the technologies that I describe in this book, based on my experience as an engineer and the materials I have developed for my training courses.
Technology does not stand still. The industry based around embedded computing is just as susceptible to Moore's law as mainstream computing. The exponential growth that this implies has meant that a surprisingly large number of things have changed since the first edition of this book was published. This second edition is fully revised to use the latest versions of the major open source components, which include Linux 4.9, Yocto Project 2.2 Morty, and Buildroot 2017.02. Since it is clear that embedded Linux will play an important part in the Internet of Things, there is a new chapter on the updating of devices in the field, including Over the Air updates. Another trend is the quest to reduce power consumption, both to extend the battery life of mobile devices and to reduce energy costs. The chapter on power management shows how this is done.
Mastering Embedded Linux Programming covers the topics in roughly the order that you will encounter them in a real-life project. The first 6 chapters are concerned with the early stages of the project, covering basics such as selecting the toolchain, the bootloader, and the kernel. At the conclusion of this this section, I introduce the idea of using an embedded build tool, using Buildroot and the Yocto Project as examples.
The middle part of the book, chapters 7 through to 13, will help you in the implementation phase of the project. It covers the topics of filesystems, the init program, multithreaded programming, software update, and power management. The third section, chapters 14 and 15, show you how to make effective use of the many debug and profiling tools that Linux has to offer in order to detect problems and identify bottlenecks. The final chapter brings together several threads to explain how Linux can be used in real-time applications.
Each chapter introduces a major area of embedded Linux. It describes the background so that you can learn the general principles, but it also includes detailed worked examples that illustrate each of these areas. You can treat this as a book of theory, or a book of examples. It works best if you do both: understand the theory and try it out in real life.
What this book covers
Chapter 1, Starting Out, sets the scene by describing the embedded Linux ecosystem and the choices available to you as you start your project.
Chapter 2, Learning About Toolchains, describes the components of a toolchain and shows you how to create a toolchain for cross-compiling code for the target board. It describes where to get a toolchain and provides details on how to build one from the source code.
Chapter 3, All About Bootloaders, explains the role of the bootloader in loading the Linux kernel into memory, and uses U-Boot and Bareboot as examples. It also introduces device trees as the mechanism used to encode the details of hardware in almost all embedded Linux systems.
Chapter 4, Configuring and Building the Kernel, provides information on how to select a Linux kernel for an embedded system and configure it for the hardware within the device. It also covers how to port Linux to the new hardware.
Chapter 5, Building a Root Filesystem, introduces the ideas behind the user space part of an embedded Linux implementation by means of a step-by-step guide on how to configure a root filesystem.
Chapter 6, Selecting a Build System, covers two commonly used embedded Linux build systems, Buildroot and Yocto Project, which automate the steps described in the previous four chapters.
Chapter 7, Creating a Storage Strategy, discusses the challenges created by managing flash memory, including raw flash chips and embedded MMC (eMMC) packages. It describes the filesystems that are applicable to each type of technology.
Chapter 8, Updating Software in the Field, examines various ways of updating the software after the device has been deployed, and includes fully managed Over the Air (OTA) updates. The key topics under discussion are reliability and security.
Chapter 9, Interfacing with Device Drivers, describes how kernel device drivers interact with the hardware with worked examples of a simple driver. It also describes the various ways of calling device drivers from the user space.
Chapter 10, Starting Up – The Init Program, shows how the first user space program--init--starts the rest of the system. It describes the three versions of the init program, each suitable for a different group of embedded systems, ranging from the simplicity of the BusyBox init, through System V init, to the current state-of-the-art, systemd.
Chapter 11, Managing Power, considers the various ways that Linux can be tuned to reduce power consumption, including Dynamic Frequency and Voltage scaling, selecting deeper idle states, and system suspend. The aim is to make devices that run for longer on a battery charge and also run cooler.
Chapter 12, Learning About Processes and Threads, describes embedded systems from the point of view of the application programmer. This chapter looks at processes and threads, inter-process communications, and scheduling policies
Chapter 13, Managing Memory, introduces the ideas behind virtual memory and how the address space is divided into memory mappings. It also describes how to measure memory usage accurately and how to detect memory leaks.
Chapter 14, Debugging with GDB, shows you how to use the GNU debugger, GDB, together with the debug agent, gdbserver, to debug applications running remotely on the target device. It goes on to show how you can extend this model to debug kernel code, making use of the kernel debug stubs, KGDB.
Chapter 15, Profiling and Tracing, covers the techniques available to measure the system performance, starting from whole system profiles and then zeroing in on particular areas where bottlenecks are causing poor performance. It also describes how to use Valgrind to check the correctness of an application's use of thread synchronization and memory allocation.
Chapter 16, Real-Time Programming, provides a detailed guide to real-time programming on Linux, including the configuration of the kernel and the PREEMPT_RT real-time kernel patch. The kernel trace tool, Ftrace, is used to measure kernel latencies and show the effect of the various kernel configurations.
What you need for this book
The software used in this book is entirely open source. In almost all cases, I have used the latest stable versions available at the time of writing. While I have tried to describe the main features in a manner that is not version-specific, it is inevitable that some of the examples will need adaptation to work with later software.
Embedded development involves two systems: the host, which is used for developing the programs, and the target, which runs them. For the host system, I have used Ubuntu 16.04, but most Linux distributions will work with just a little modification. You may decide to run Linux as a guest in a virtual machine, but you should be aware that some tasks, such as building a distribution using the Yocto Project, are quite demanding and are better run on a native installation of Linux.
I chose two exemplar targets: the QEMU emulator and the BeagleBone Black. Using QEMU means that you can try out most of the examples without having to invest in any additional hardware. On the other hand, some things work better if you do have real hardware, for which, I have chosen the BeagleBone Black because it is not expensive, it is widely available, and it has very good community support. Of course, you are not limited to just these two targets. The idea behind the book is to provide you with general solutions to problems so that you can apply them to a wide range of target boards.
Who this book is for
This book is written for developers who have an interest in embedded computing and Linux, and want to extend their knowledge into the various branches of the subject. In writing the book, I assume a basic understanding of the Linux command line, and in the programming examples, a working knowledge of the C language. Several chapters focus on the hardware that goes into an embedded target board, and, so, a familiarity with hardware and hardware interfaces will be a definite advantage in these cases.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. You can download the code files by following these steps:
Log in or register to our website using your e-mail address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
TAR for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Embedded-Linux-Programming-Second-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/MasteringEmbeddedLinuxProgrammingSecondEdition_ColorImages.pdf.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title. To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at [email protected] with a link to the suspected pirated material. We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Starting Out
You are about to begin working on your next project, and this time it is going to be running Linux. What should you think about before you put finger to keyboard? Let's begin with a high-level look at embedded Linux and see why it is popular, what are the implications of open source licenses, and what kind of hardware you will need to run Linux.
Linux first became a viable choice for embedded devices around 1999. That was when Axis (https://www.axis.com), released their first Linux-powered network camera and TiVo (https://business.tivo.com/) their first Digital Video Recorder (DVR). Since 1999, Linux has become ever more popular, to the point that today it is the operating system of choice for many classes of product. At the time of writing, in 2017, there are about two billion devices running Linux. That includes a large number of smartphones running Android, which uses a Linux kernel, and hundreds of millions of set-top-boxes, smart TVs, and Wi-Fi routers, not to mention a very diverse range of devices such as vehicle diagnostics, weighing scales, industrial devices, and medical monitoring units that ship in smaller volumes.
So, why does your TV run Linux? At first glance, the function of a TV is simple: it has to display a stream of video on a screen. Why is a complex Unix-like operating system like Linux necessary?
The simple answer is Moore's Law: Gordon Moore, co-founder of Intel, observed in 1965 that the density of components on a chip will double approximately every two years. That applies to the devices that we design and use in our everyday lives just as much as it does to desktops, laptops, and servers. At the heart of most embedded devices is a highly integrated chip that contains one or more processor cores and interfaces with main memory, mass storage, and peripherals of many types. This is referred to as a System on Chip, or SoC, and SoCs are increasing in complexity in accordance with Moore's Law. A typical SoC has a technical reference manual that stretches to thousands of pages. Your TV is not simply displaying a video stream as the old analog sets used to do.
The stream is digital, possibly encrypted, and it needs processing to create an image. Your TV is (or soon will be) connected to the Internet. It can receive content from smartphones, tablets, and home media servers. It can be (or soon will be) used to play games. And so on and so on. You need a full operating system to manage this degree of complexity.
Here are some points that drive the adoption of Linux:
Linux has the necessary functionality. It has a good scheduler, a good network stack, support for USB, Wi-Fi, Bluetooth, many kinds of storage media, good support for multimedia devices, and so on. It ticks all the boxes.
Linux has been ported to a wide range of processor architectures, including some that are very commonly found in SoC designs--ARM, MIPS, x86, and PowerPC.
Linux is open source, so you have the freedom to get the source code and modify it to meet your needs. You, or someone working on your behalf, can create a board support package for your particular SoC board or device. You can add protocols, features, and technologies that may be missing from the mainline source code. You can remove features that you don't need to reduce memory and storage requirements. Linux is flexible.
Linux has an active community; in the case of the Linux kernel, very active. There is a new release of the kernel every 8 to 10 weeks, and each release contains code from more than 1,000 developers. An active community means that Linux is up to date and supports current hardware, protocols, and standards.
Open source licenses guarantee that you have access to the source code. There is no vendor tie-in.
For these reasons, Linux is an ideal choice for complex devices. But there are a few caveats I should mention here. Complexity makes it harder to understand. Coupled with the fast moving development process and the decentralized structures of open source, you have to put some effort into learning how to use it and to keep on re-learning as it changes. I hope that this book will help in the process.
Selecting the right operating system
Is Linux suitable for your project? Linux works well where the problem being solved justifies the complexity. It is especially good where connectivity, robustness, and complex user interfaces are required. However, it cannot solve every problem, so here are some things to consider before you jump in:
Is your hardware up to the job? Compared to a traditional
real-time operating system
(
RTOS
) such as VxWorks, Linux requires a lot more resources. It needs at least a 32-bit processor and lots more memory. I will go into more detail in the section on typical hardware requirements.
Do you have the right skill set? The early parts of a project, board bring-up, require detailed knowledge of Linux and how it relates to your hardware. Likewise, when debugging and tuning your application, you will need to be able to interpret the results. If you don't have the skills in-house, you may want to outsource some of the work. Of course, reading this book helps!
Is your system real-time? Linux can handle many real-time activities so long as you pay attention to certain details, which I will cover in detail
in Chapter 16,
Real-Time Programming
.
Consider these points carefully. Probably the best indicator of success is to look around for similar products that run Linux and see how they have done it; follow best practice.
The players
Where does open source software come from? Who writes it? In particular, how does this relate to the key components of embedded development—the toolchain, bootloader, kernel, and basic utilities found in the root filesystem?
The main players are:
The open source community
: This, after all, is the engine that generates the software you are going to be using. The community is a loose alliance of developers, many of whom are funded in some way, perhaps by a not-for-profit organization, an academic institution, or a commercial company. They work together to further the aims of the various projects. There are many of them—some small, some large. Some that we will be making use of in the remainder of this book are Linux itself, U-Boot, BusyBox, Buildroot, the Yocto Project, and the many projects under the GNU umbrella.
CPU architects
: These are the organizations that design the CPUs we use. The important ones here are ARM/Linaro (ARM-based SoCs), Intel (x86 and x86_64), Imagination Technologies (MIPS), and IBM (PowerPC). They implement or, at the very least, influence support for the basic CPU architecture.
SoC vendors
(Atmel, Broadcom, Intel, Qualcomm, TI, and many others). They take the kernel and toolchain from the CPU architects and modify them to support their chips. They also create reference boards: designs that are used by the next level down to create development boards and working products.
Board vendors and OEMs
: These people take the reference designs from SoC vendors and build them in to specific products, for instance, set-top-boxes or cameras, or create more general purpose development boards, such as those from Avantech and Kontron. An important category are the cheap development boards such as BeagleBoard/BeagleBone and Raspberry Pi that have created their own ecosystems of software and hardware add-ons.
These form a chain, with your project usually at the end, which means that you do not have a free choice of components. You cannot simply take the latest kernel from https://www.kernel.org/, except in a few rare cases, because it does not have support for the chip or board that you are using.
This is an ongoing problem with embedded development. Ideally, the developers at each link in the chain would push their changes upstream, but they don't. It is not uncommon to find a kernel which has many thousands of patches that are not merged. In addition, SoC vendors tend to actively develop open source components only for their latest chips, meaning that support for any chip more than a couple of years old will be frozen and not receive any updates.
The consequence is that most embedded designs are based on old versions of software. They do not receive security fixes, performance enhancements, or features that are in newer versions. Problems such as Heartbleed (a bug in the OpenSSL libraries) and ShellShock (a bug in the bash shell) go unfixed. I will talk more about this later in this chapter under the topic of security.
What can you do about it? First, ask questions of your vendors: what is their update policy, how often do they revise kernel versions, what is the current kernel version, what was the one before that, and what is their policy for merging changes up-stream? Some vendors are making great strides in this way. You should prefer their chips.
Secondly, you can take steps to make yourself more self-sufficient. The chapters in section 1 explain the dependencies in more detail and show you where you can help yourself. Don't just take the package offered to you by the SoC or board vendor and use it blindly without considering the alternatives.
Project life cycle
This book is divided into four sections that reflect the phases of a project. The phases are not necessarily sequential. Usually they overlap and you will need to jump back to revisit things that were done previously. However, they are representative of a developer's preoccupations as the project progresses:
Elements of embedded Linux (Chapters 1 to 6) will help you set up the development environment and create a working platform for the later phases. It is often referred to as the
board bring-up
phase.
System architecture and design choices (Chapters 7 to 11) will help you to look at some of the design decisions you will have to make concerning the storage of programs and data, how to divide work between kernel device drivers and applications, and how to initialize the system.
Writing embedded applications (Chapters 12 and 13) shows how to make effective use of the Linux process and threads model, and how to manage memory in a resource-constrained device.
Debugging and optimizing performance (Chapters 14 and 15) describes how to trace, profile, and debug your code in both the applications and the kernel.
The fifth section on real-time (Chapter 16,Real-Time Programming) stands somewhat alone because it is a small, but important, category of embedded systems. Designing for real-time behavior has an impact on each of the four main phases.
The four elements of embedded Linux
Every project begins by obtaining, customizing, and deploying these four elements: the toolchain, the bootloader, the kernel, and the root filesystem. This is the topic of the first section of this book.
Toolchain
: The compiler and other tools needed to create code for your target device. Everything else depends on the toolchain.
Bootloader
: The program that initializes the board and loads the Linux kernel.
Kernel
: This is the heart of the system, managing system resources and interfacing with hardware.
Root filesystem
: Contains the libraries and programs that are run once the kernel has completed its initialization.
Of course, there is also a fifth element, not mentioned here. That is the collection of programs specific to your embedded application which make the device do whatever it is supposed to do, be it weigh groceries, display movies, control a robot, or fly a drone.
Typically, you will be offered some or all of these elements as a package when you buy your SoC or board. But, for the reasons mentioned in the preceding paragraph, they may not be the best choices for you. I will give you the background to make the right selections in the first six chapters and I will introduce you to two tools that automate the whole process for you: Buildroot and the Yocto Project.
Open source
The components of embedded Linux are open source, so now is a good time to consider what that means, why open sources work the way they do, and how this affects the often proprietary embedded device you will be creating from it.
Licenses
When talking about open source, the word free is often used. People new to the subject often take it to mean nothing to pay, and open source software licenses do indeed guarantee that you can use the software to develop and deploy systems for no charge. However, the more important meaning here is freedom, since you are free to obtain the source code, modify it in any way you see fit, and redeploy it in other systems. These licenses give you this right. Compare that with shareware licenses which allow you to copy the binaries for no cost but do not give you the source code, or other licenses that allow you to use the software for free under certain circumstances, for example, for personal use but not commercial. These are not open source.
I will provide the following comments in the interest of helping you understand the implications of working with open source licenses, but I would like to point out that I am an engineer and not a lawyer. What follows is my understanding of the licenses and the way they are interpreted.
Open source licenses fall broadly into two categories: the copyleft licenses such as the General Public License (GPL) and the permissive licenses such as those from the Berkeley Software Distribution (BSD), the Apache Foundation, and others.
The permissive licenses say, in essence, that you may modify the source code and use it in systems of your own choosing so long as you do not modify the terms of the license in any way. In other words, with that one restriction, you can do with it what you want, including building it into possibly proprietary systems.
The GPL licenses are similar, but have clauses which compel you to pass the rights to obtain and modify the software on to your end users. In other words, you share your source code. One option is to make it completely public by putting it onto a public server. Another is to offer it only to your end users by means of a written offer to provide the code when requested. The GPL goes further to say that you cannot incorporate GPL code into proprietary programs. Any attempt to do so would make the GPL apply to the whole. In other words, you cannot combine a GPL and proprietary code in one program.
So, what about libraries? If they are licensed with the GPL, any program linked with them becomes GPL also. However, most libraries are licensed under the Lesser General Public License (LGPL). If this is the case, you are allowed to link with them from a proprietary program.
All the preceding description relates specifically to GLP v2 and LGPL v2.1. I should mention the latest versions of GLP v3 and LGPL v3. These are controversial, and I will admit that I don't fully understand the implications. However, the intention is to ensure that the GPLv3 and LGPL v3 components in any system can be replaced by the end user, which is in the spirit of open source software for everyone. It does pose some problems though. Some Linux devices are used to gain access to information according to a subscription level or another restriction, and replacing critical parts of the software may compromise that. Set-top-boxes fit into this category. There are also issues with security. If the owner of a device has access to the system code, then so might an unwelcome intruder. Often the defense is to have kernel images that are signed by an authority, the vendor, so that unauthorized updates are not possible. Is that an infringement of my right to modify my device? Opinions differ.
Hardware for embedded Linux
If you are designing or selecting hardware for an embedded Linux project, what do you look out for?
Firstly, a CPU architecture that is supported by the kernel—unless you plan to add a new architecture yourself, of course! Looking at the source code for Linux 4.9, there are 31 architectures, each represented by a sub-directory in the arch/ directory. They are all 32- or 64-bit architectures, most with a memory management unit (MMU), but some without. The ones most often found in embedded devices are ARM, MIPS PowerPC, and X86, each in 32- and 64-bit variants, and all of which have memory management units.
Most of this book is written with this class of processor in mind. There is another group that doesn't have an MMU that runs a subset of Linux known as microcontroller Linux or uClinux. These processor architectures include ARC, Blackfin, MicroBlaze, and Nios. I will mention uClinux from time to time but I will not go into detail because it is a rather specialized topic.
Secondly, you will need a reasonable amount of RAM. 16 MiB is a good minimum, although it is quite possible to run Linux using half that. It is even possible to run Linux with 4 MiB if you are prepared to go to the trouble of optimizing every part of the system. It may even be possible to get lower, but there comes a point at which it is no longer Linux.
Thirdly, there is non-volatile storage, usually flash memory. 8 MiB is enough for a simple device such as a webcam or a simple router. As with RAM, you can create a workable Linux system with less storage if you really want to, but the lower you go, the harder it becomes. Linux has extensive support for flash storage devices, including raw NOR and NAND flash chips, and managed flash in the form of SD cards, eMMC chips, USB flash memory, and so on.
Fourthly, a debug port is very useful, most commonly an RS-232 serial port. It does not have to be fitted on production boards, but makes board bring-up, debugging, and development much easier.
Fifthly, you need some means of loading software when starting from scratch. A few years ago, boards would have been fitted with a Joint Test Action Group (JTAG) interface for this purpose, but modern SoCs have the ability to load boot code directly from removable media, especially SD and micro SD cards, or serial interfaces such as RS-232 or USB.
In addition to these basics, there are interfaces to the specific bits of hardware your device needs to get its job done. Mainline Linux comes with open source drivers for many thousands of different devices, and there are drivers (of variable quality) from the SoC manufacturer and from the OEMs of third-party chips that may be included in the design, but remember my comments on the commitment and ability of some manufacturers. As a developer of embedded devices, you will find that you spend quite a lot of time evaluating and adapting third-party code, if you have it, or liaising with the manufacturer if you don't. Finally, you will have to write the device support for interfaces that are unique to the device, or find someone to do it for you.
Hardware used in this book
The worked examples in this book are intended to be generic, but to make them relevant and easy to follow, I have had to choose specific hardware. I have chosen two exemplar devices: the BeagleBone Black and QEMU. The first is a widely-available and cheap development board which can be used in serious embedded hardware. The second is a machine emulator that can be used to create a range of systems that are typical of embedded hardware. It was tempting to use QEMU exclusively, but, like all emulations, it is not quite the same as the real thing. Using a BeagleBone Black, you have the satisfaction of interacting with real hardware and seeing real LEDs flash. I could have selected a board that is more up-to-date than the BeagleBone Black, which is several years old now, but I believe that its popularity gives it a degree of longevity and it means that it will continue to be available for some years yet.
In any case, I encourage you to try out as many of the examples as you can, using either of these two platforms, or indeed any embedded hardware you may have to hand.
The BeagleBone Black
The BeagleBone and the later BeagleBone Black are open hardware designs for a small, credit card sized development board produced by CircuitCo LLC. The main repository of information is at https://beagleboard.org/. The main points of the specifications are:
TI AM335x 1 GHz ARM® Cortex-A8 Sitara SoC
512 MiB DDR3 RAM
2 or 4 GiB 8-bit eMMC on-board flash storage
Serial port for debug and development
MicroSD connector, which can be used as the boot device
Mini USB OTG client/host port that can also be used to power the board
Full size USB 2.0 host port
10/100 Ethernet port
HDMI for video and audio output
In addition, there are two 46-pin expansion headers for which there are a great variety of daughter boards, known as capes, which allow you to adapt the board to do many different things. However, you do not need to fit any capes in the examples in this book.
In addition to the board itself, you will need:
A mini USB to full-size USB cable (supplied with the board) to provide power, unless you have the last item on this list.
An RS-232 cable that can interface with the 6-pin 3.3V TTL level signals provided by the board. The Beagleboard website has links to compatible cables.
A microSD card and a means of writing to it from your development PC or laptop, which will be needed to load software onto the board.
An Ethernet cable, as some of the examples require network connectivity.
Optional, but recommended, a 5V power supply capable of delivering 1 A or more.
QEMU
QEMU is a machine emulator. It comes in a number of different flavors, each of which can emulate a processor architecture and a number of boards built using that architecture. For example, we have the following:
qemu-system-arm
: ARM
qemu-system-mips
: MIPS
qemu-system-ppc
: PowerPC
qemu-system-x86
: x86 and x86_64
For each architecture, QEMU emulates a range of hardware, which you can see by using the option—machine help. Each machine emulates most of the hardware that would normally be found on that board. There are options to link hardware to local resources, such as using a local file for the emulated disk drive. Here is a concrete example:
$ qemu-system-arm -machine vexpress-a9 -m 256M -drive
file=rootfs.ext4,sd -net nic -net use -kernel zImage -dtb vexpress-
v2p-ca9.dtb -append "console=ttyAMA0,115200 root=/dev/mmcblk0" -
serial stdio -net nic,model=lan9118 -net tap,ifname=tap0
The options used in the preceding command line are:
-machine vexpress-a9
: Creates an emulation of an ARM Versatile Express development board with a Cortex A-9 processor
-m 256M
: Populates it with 256 MiB of RAM
-drive file=rootfs.ext4,sd
: Connects the
SD
interface to the local file
rootfs.ext4
(which contains a filesystem image)
-kernel zImage
: Loads the Linux kernel from the local file named
zImage
-dtb vexpress-v2p-ca9.dtb
: Loads the device tree from the local file
vexpress-v2p-ca9.dtb
-append "..."
: Supplies this string as the kernel command-line
-serial stdio
: Connects the serial port to the terminal that launched QEMU, usually so that you can log on to the emulated machine via the serial console
-net nic,model=lan9118
: Creates a network interface
-net tap,ifname=tap0
: Connects the network interface to the virtual network interface
tap0
To configure the host side of the network, you need the tunctl command from the User Mode Linux (UML) project; on Debian and Ubuntu, the package is named uml-utilites:
$ sudo tunctl -u $(whoami) -t tap0
This creates a network interface named tap0 which is connected to the network controller in the emulated QEMU machine. You configure tap0 in exactly the same way as any other interface.
All of these options are described in detail in the following chapters. I will be using Versatile Express for most of my examples, but it should be easy to use a different machine or architecture.
Software used in this book
I have used only open source software, both for the development tools and the target operating system and applications. I assume that you will be using Linux on your development system. I tested all the host commands using Ubuntu 14.04 and so there is a slight bias towards that particular version, but any modern Linux distribution is likely to work just fine.
Summary
Embedded hardware will continue to get more complex, following the trajectory set by Moore's Law. Linux has the power and the flexibility to make use of hardware in an efficient way.
Linux is just one component of open source software out of the many that you need to create a working product. The fact that the code is freely available means that people and organizations at many different levels can contribute. However, the sheer variety of embedded platforms and the fast pace of development lead to isolated pools of software which are not shared as efficiently as they should be. In many cases, you will become dependent on this software, especially the Linux kernel that is provided by your SoC or Board vendor, and to a lesser extent, the toolchain. Some SoC manufacturers are getting better at pushing their changes upstream and the maintenance of these changes is getting easier.
Fortunately, there are some powerful tools that can help you create and maintain the software for your device. For example, Buildroot is ideal for small systems and the Yocto Project for larger ones. Before I describe these build tools, I will describe the four elements of embedded Linux, which you can apply to all embedded Linux projects, however they are created.
The next chapter is all about the first of these, the toolchain, which you need to compile code for your target platform.
Learning About Toolchains
The toolchain is the first element of embedded Linux and the starting point of your project. You will use it to compile all the code that will run on your device. The choices you make at this early stage will have a profound impact on the final outcome. Your toolchain should be capable of making effective use of your hardware by using the optimum instruction set for your processor. It should support the languages that you require, and have a solid implementation of the Portable Operating System Interface (POSIX) and other system interfaces. Not only that, but it should be updated when security flaws are discovered or bugs are found. Finally, it should be constant throughout the project. In other words, once you have chosen your toolchain, it is important to stick with it. Changing compilers and development libraries in an inconsistent way during a project will lead to subtle bugs.
