26,39 €
The future of IoT has the potential to be limitless. Wouldn’t it be great if you could add it to your own technological stacks? But where to start? With the basics, of course.
In this book, you will start by learning about the most popular hardware and communication protocol, Raspberry Pi and MQTT. You will see how to use them together by setting up your own MQTT server on Raspberry Pi and understand how it works.
This book explores MQTT in detail, including the clients and devices that you can connect to your server. You will discover two very popular IoT development boards among project developers: the ESP8266 and ESP32 development boards. Then, you will learn how to build interactive dashboards on your Pi and monitor your client devices. The book also shows you how to build a dashboard using another popular software – Node-RED. You will be able to put your skills to the test by creating two full-scale projects. That’s not all: you will also learn how to host your own MQTT server on a virtual cloud service. Finally, you will be guided on how to move forward from here, what technologies to learn, and some project recommendations to polish or test your knowledge.
By the end of this book, you will be able to build meaningful projects using Raspberry Pi and MQTT and create dashboards for your projects on Node-RED.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 258
Veröffentlichungsjahr: 2022
A complete guide to helping you build innovative full-scale prototype projects using Raspberry Pi and MQTT protocol
Dhairya Parikh
BIRMINGHAM—MUMBAI
Copyright © 2022 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Group Product Manager: Rahul Nair
Publishing Product Manager: Meeta Rajani
Senior Content Development Editor: Sayali Pingale
Technical Editor: Rajat Sharma
Copy Editor: Safis Editing
Project Manager: Neil Dmello
Proofreader: Safis Editing
Indexer: Subalakshmi Govindhan
Production Designer: Shyam Sundar Korumilli
Marketing Coordinator: Nimisha Dua
Senior Marketing Coordinator: Sanjana Gupta
First published: August 2022
Production reference: 0240822
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
978-1-80324-448-8
www.packt.com
To my parents, Bhumika and Atit, for their sacrifices and constant encouragement.
Dhairya Parikh currently works as a data engineer at Accenture, where he builds efficient data products to help their clients get the most out of their data. He completed his undergraduate studies in electronics engineering at BVM Engineering College, Anand. He is a seasoned project developer with several award-winning projects under his belt. His most recent literary works include several articles written for the Circuit Cellar magazine. Most of his articles and projects are based on IoT and machine learning. He is an open source enthusiast with an interest in building projects that make a positive impact on people’s lives using new development hardware and writing about them in his spare time.
I would like to thank my parents and family for their constant support and encouragement throughout this project.
I would also like to thank Neil D’mello, Shazeen Iqbal, Preet Ahuja, Rafiaa Khan, and the entire Packt team for their guidance throughout this project, and Asim Zulfiqar and John Witts for their technical insights! Finally, I would like to thank Yashashree Hardikar for all the initial support.
Asim Zulfiqar is a blogger and tech content creator who has been writing tutorials on embedded systems and IoT, specifically on MQTT, on his blog and YouTube channel, High Voltages. He also provides IoT and embedded freelance services to companies through this channel.
He completed his bachelor’s degree in electronic engineering at Sir Syed University of Engineering and Technology, Pakistan. After that, he completed his Erasmus Mundus joint master’s degree program in Photonics Integrated Circuits, Sensors, and Networks at Scuola Superiore Sant’Anna (Italy), Aston University (UK), and Osaka University (Japan).
Reviewing a book is more exciting than I thought, and I have enjoyed reviewing this book. None of this would have been possible without my family, friends, teachers, and all the open-source communities that helped me gain enough knowledge to do so. I would also like to thank the author of this book, Dhairya, who helped me learn some new concepts, and Packt, for providing me this opportunity.
The future of IoT has the potential to be limitless. By 2025, it is estimated that there will be more than 21 billion IoT devices. So, wouldn’t it be great if you could add these to your known technological stacks? But where to start? Of course, with the basics.
First, we will learn about the most popular hardware used for IoT prototyping, the Raspberry Pi. Then, we will learn what MQTT, one of the most used communication protocols for communicating between devices, is. We will then explore why these are the most suitable options to get started, their advantages, and how they are currently being used in the industry. Then, you will see how to use them together by setting up your very own MQTT Server on the Raspberry Pi and understanding how it works. We will get into the details of MQTT and learn more about the clients or devices we will connect to our server. In particular, we will cover two very popular IoT development boards among project developers: ESP8266 and ESP32. You will also learn how to build interactive dashboards on your Pi and control or monitor your client devices. You will build the dashboards using another popular software – Node-RED.
You will then put your theory into practical use by creating two full-scale projects: an IoT weather station and a smart relay system. That’s not all; you will also learn how to host your very own MQTT server on a virtual cloud service. Then you will be guided on the next steps to take after reading this book, what technologies to learn along with some useful project recommendations. Finally, we will cover the popular cloud platforms (AWS and GCP) to create IoT projects and also create a project where we connect our Node MCU to AWS IoT.
This book is suitable for a wide range of audiences. Particularly, this book is targeted at students who want to start building IoT projects, educators who want to teach an introductory IT course, technology enthusiasts, and IoT and hardware developers.
Chapter 1, Introduction to Raspberry PI and MQTT, provides an introduction to the hardware we will be using, the Raspberry Pi. Moreover, it will also cover the basics of MQTT and how the communication protocol actually works. Next, we will learn to set up the Raspberry Pi. This includes installing the popular Debian-based Raspberry Pi OS on our Raspberry Pi. After that, we will install the necessary libraries and packages to make our device a local MQTT broker.
Chapter 2, MQTT in Detail, covers how exactly MQTT works. This includes a gentle introduction to MQTT brokers and clients, and different MQTT control packets will be covered in detail. Finally, we will see a demonstration of how a client connects and communicates with a broker.
Chapter 3, Introduction to ESP Development Boards, is all about implementing what we learned in the previous chapter. It will first introduce you to the popular ESP development boards – NodeMCU and ESP32. After covering the specifications of each board, we will move on to learn how to set up the boards as an MQTT client. Finally, we will create our first project wherein we will connect to our Raspberry Pi’s MQTT broker and control the onboard LED through MQTT.
Chapter 4, Node-RED on Raspberry Pi, gets you acquainted with very popular software for the Raspberry Pi – Node-RED. It is divided into four sections. First is an introduction to Node-RED, followed by a guide to installing and setting up Node-RED on Raspberry Pi. After that, we will cover the Node-RED MQTT and dashboard components, and then create a simple project to implement everything we have learned.
Chapter 5, Major Project 1: IoT Weather Station, is where, now that we have the knowledge from all the topics discussed in the previous chapters, we will be working on our first major project: making an IoT weather station. This chapter provides step-by-step instructions on how to build this.
Chapter 6, Major Project 2: Smart Home Control Relay System, helps you create a smart home device to control wall switches using the Node-RED dashboard hosted on the Raspberry Pi. The device will be based on the popular ESP32 development board. For this project, we will be preparing a PCB instead of creating the circuit on a breadboard for a more finished and professional look.
Chapter 7, Taking Your MQTT Broker Global, is where we will discuss the advantages of having an online MQTT broker further, and two major options that we currently have to achieve these advantages. We can still use the local broker on our Pi, but we can route all the data to any destination through the internet.
Chapter 8, Project Prototype to Product, How?, starts by exploring IoT services provided by some popular cloud services, such as AWS and GCP, now that the book has covered all the essentials required to get you familiar with all the concepts related to Raspberry Pi and MQTT. We will even create a project demo integrating our Node MCU board with AWS IoT.
This book has been written for beginners, so in terms of knowledge, there are no prerequisites. As for the hardware, you will need all the hardware devices listed in the following table in order to follow along and build projects with me. In terms of software requirements, you will need the Raspberry Pi Imager software (available for all three major operating systems) to create flashed SD cards for your Pi (it even supports SD card formatting), Wireshark on the Raspberry Pi OS to dissect the MQTT control packets, and Node-RED as a dashboard interface for our projects, also to be installed on Raspberry Pi OS.
If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.
You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Raspberry-Pi-and-MQTT-Essentials. If there’s an update to the code, it will be updated in the GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/860jg.
There are a number of text conventions used throughout this book.
Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “Mount the downloaded WebStorm-10*.dmg disk image file as another disk in your system.”
A block of code is set as follows:
void setup () { pinMode (LED_BUILTIN, OUTPUT); }Any command-line input or output is written as follows:
sudo apt install mosquitto mosquitto-clients
Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Select System info from the Administration panel.”
Tips or Important Notes
Appear like this.
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Once you’ve read Raspberry Pi and MQTT Essentials, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.
Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.
After completing this section, you will know all about Raspberry Pi, MQTT, NodeMCU, ESP32 development boards, and Node-RED. You will gain the knowledge required to build intermediate-complexity projects.
This part comprises of the following chapters:
Chapter 1, Introduction to Raspberry PI and MQTTChapter 2, MQTT in DetailChapter 3, Introduction to ESP Development BoardsChapter 4, Node-RED on Raspberry PiIn recent years, the Internet of Things (IoT) has been a trending field for research and development. The future of IoT has the potential to be limitless. By 2025, it is estimated that there will be more than 21 billion IoT devices. So, wouldn’t it be great if you could add these to your known technological stacks? In this book, we will start with the absolute basics.
I will walk you through two fascinating subjects throughout this book: Raspberry Pi, which is a prevalent development board for beginners, and MQTT, a very commonly used and robust communication protocol to delve into the world of IoT.
This chapter will introduce you to the basics of MQTT and Raspberry Pi. Moreover, it will help you set up your Raspberry Pi. Although simple, it is crucial to perform each step as this will help us set up our own local MQTT broker on the Raspberry Pi. This will also help you understand how to get started with your new Raspberry Pi by installing an operating system onto it.
First, we will flash the popular Debian-based Raspberry Pi OS on our Raspberry Pi. After that, we will install all the necessary libraries and packages to make our device a local MQTT broker.
We will cover the following main topics in this chapter:
What is MQTT and how does it work?A gentle introduction to Raspberry Pi Setting up your Raspberry PiSo, let’s start by knowing what MQTT is.
In this section, we will learn about the essential concepts of MQTT. First, we will look at the basic concepts of MQTT and some history, followed by the functionality and components of MQTT. Finally, we will have a brief encounter with the salient features of MQTT.
Please note that there are different versions of MQTT, and most of what we discuss is relevant to the MQTT protocol version 3.1.
MQTT stands for Message Queuing Telemetry Transport. It is a lightweight communication protocol.
According to the official MQTT v3.1 documentation:
MQTT is a Client-Server publish/subscribe messaging transport protocol. It is lightweight, open, simple, and designed to be easy to implement. These characteristics make it ideal for use in many situations, including constrained environments for communication in Machine to Machine (M2M) and Internet of Things (IoT) contexts where a small code footprint is required, and network bandwidth is at a premium.
This is a clear and clean definition of the MQTT protocol in just a few lines. It is a messaging protocol designed for easy implementation, primarily client side. It is an open and lightweight communication protocol with minimal packet overhead. It is generally used for communication between two or more devices.
Now that we know what MQTT is, we will explore the basic concepts of MQTT we came across in the previous section. More specifically, we will look at MQTT as a publish/subscribe protocol.
The publish/subscribe protocol is an alternative to the traditional client-server architecture. It means that instead of categorizing both sending and receiving machines as clients, the clients who send a message are publishers and the clients who receive the messages are the subscribers.
Another essential feature of such protocols is the decoupling between the clients. In simple words, the clients never directly communicate with each other. They are mediated by the third component of this system, known as the broker. In this book, we will be using our Raspberry Pi as an MQTT broker, which connects different client devices within a local network. The primary function of the broker is to mediate and manage all communications between the various clients (i.e., publishers and subscribers).
To better understand how the whole system works, please see Figure 1.1, which shows how the communication protocol operates with a very simplified diagram. In this example, the publishing client is a temperature monitor and the subscribing device is a mobile phone:
Figure 1.1 – Basic MQTT communication flow
Please note that this is just a simplified representation. There can be multiple publishers and subscribers connected to a single broker. As you can see, the temperature monitor sends the current temperature value of 27°C through the MQTT communication protocol, which is then received by the MQTT broker, which routes it to the subscriber, a mobile application in our case.
We will now look into some details about publishers, subscribers, and brokers with the help of an example:
Publishers: These devices or machines are responsible for sending the collected data to the brokers. For instance, if you have an air quality monitoring system that monitors the CO2 levels in the air every 30 seconds, the device will be set to publish the CO2 concentration every 30 seconds.Subscribers: These devices receive the requested sensor data from the brokers. Considering the preceding example, an air purifier can be a subscriber of our air quality monitoring system. It constantly receives the CO2 concentration values, and when it crosses a threshold value, the purifier automatically turns on.Broker: This intermediary device connects various publishers and subscribers by managing and routing the data. We will be using Raspberry Pi as a broker for the entirety of this book.Please note that both the publishers and subscribers are referred to as clients. A client can be a publisher, subscriber, or both as both these processes are entirely independent of each other, as we will see in the later chapters.
But another question arises now: how does the broker manage or route which information is sent where? The following section will answer this question, exploring MQTT functionality.
We have already seen the significant components of MQTT, but we will now explore how these components communicate with each other.
MQTT has no client device addresses or identifiers, making it easy to build an expansible, ad hoc network. The only thing all clients must know is the address of the broker. So, how do messages get routed between the clients? The solution for this is topics and messages.
This is how the whole system works:
First, the publisher sends the data collected to the broker on a particular topic, which is similar to a channel for data transmission and reception. Please note that a topic can have several subtopics too. For example, in an application where you send the temperature data from a sensor connected to your fridge, the topic will look something like this:Kitchen/Fridge/
The main topic is the kitchen, and the appliance is the subtopic. The message will be Temperature:14 on the given topic.
The subscribers listen to the topic. So, if the subscriber is listening to the Kitchen topic, it will have access to all the subtopics that are a part of this topic.The primary function of the broker is to manage all the available topics and route the information according to the type of client, namely publishers and subscribers.Now that we are aware of the details of MQTT, we will have a look at the salient features of this communication protocol.
This section will cover the main features of this communication protocol:
Lightweight and efficient:MQTT clients are tiny, and they require minimal resources to operate. So, even microcontrollers such as ESP8266 can be used as a client as long as they have an active connection to a network.
This protocol is highly efficient thanks to the small message headers that provide maximum network bandwidth efficiency.
Bidirectional communication protocol:MQTT allows to-and-fro messaging capability. This means a device can be a publisher and a subscriber simultaneously. This also allows easy broadcasting of messages to several devices at once.
Highly scalable:There is no worry about maintaining clients’ addresses or IDs; it is effortless to expand the MQTT network. Moreover, the decoupling between the publishers and subscribers makes things even more accessible. The only things required on the client side are the broker’s IP address and the topic name.
Reliability:MQTT is highly reliable when it comes to message delivery. As this is an essential aspect of any communication protocol, MQTT comes with three predefined quality of service (QoS) levels:
QoS 0: At most onceQoS 1: At least onceQoS 2: Exactly onceSupport for unreliable networks:Many IoT devices are connected over unreliable networks, and MQTT’s support for persistent sessions reduces the client’s time with the broker. For example, several monitoring devices are deployed on moving vehicles or in remote areas such as forests.
Highly secure:MQTT makes it easy to encrypt messages using TLS and authenticate clients using modern authentication protocols, such as OAuth.
This is the end of this section. We covered the basics of MQTT and the components and salient features of this popular communication protocol. This protocol will be discussed in detail in Chapter 2,MQTT in Detail, of this book.
This section will introduce you to the Raspberry Pi, the world’s most affordable credit card-sized computer.
There is a wide range of available Raspberry Pi development boards available. They are primarily available in four formats:
Model B: These are full-size boards equipped with Ethernet and USB ports.Model A: These are square-shaped boards, considered light models of Raspberry Pi. They are different from the Model B because of the absence of an Ethernet port, fewer USB ports, and a slightly less powerful processor chip. They come at a lower price due to these cuts.Zero: This is the cheapest and smallest Raspberry Pi available. It is equipped with a significantly less powerful and low-power processor, includes no USB or Ethernet port, and is equipped with a mini-HDMI port instead of a full-size HDMI. Compute: This is Raspberry Pi 4 in a compact package for embedded applications. Additional RAM and eMMC Flash customizations are available (32 different variant configurations are listed on the official Raspberry Pi website).The latest models of the Pi available are as follows:
Raspberry Pi Model 4BRaspberry Pi Model 3 (B+, B, and A)Raspberry Pi Zero WRaspberry Pi 400 (a personal computer kit)Raspberry Pi 4 Compute ModuleWe will cover the Raspberry Pi Model 4B in depth as it is the latest variant available and is the model we will be using throughout the book.
This is the latest development board from Raspberry Pi (Figure 1.2). It has several new and improved features that make it an incredible upgrade over the older models. The most significant change is the support of two 4K displays, which is an astonishing feat on hardware that costs 35 dollars.
Another distinctive feature is the upgraded CPU and RAM options. The latest board is powered by a new 1.5 GHz quad-core CPU, almost three times faster than the previous-generation processor. Moreover, the boards are available in 2 GB, 4 GB, and 8 GB LPDDR4 RAM configurations.
It also has USB C support, USB 3.0 support, and Gigabit Ethernet. The Raspberry Pi 4 is a viable dual-display desktop computer with these new hardware capabilities.
Figure 1.2 – Raspberry Pi Model 4B+: the latest Raspberry Pi development board
Now, we will dig a bit deeper and cover the hardware specifications of this development board in detail followed by a brief discussion of some popular operating systems that are available for this board.
The hardware specifications of the Raspberry Pi model 4 are as follows, as mentioned on the official Raspberry Pi 4 product page:
Broadcom BCM2711, quad-core Cortex-A72 (ARM v8) 64-bit SoC @ 1.5GHz2 GB, 4 GB, or 8 GB LPDDR4-3200 SDRAM (depending on model)2.4 GHz and 5.0 GHz IEEE 802.11ac wireless, Bluetooth 5.0, BLEGigabit EthernetTwo USB 3.0 ports; two USB 2.0 portsRaspberry Pi standard 40-pin GPIO header (fully backward compatible with previous boards)Two micro-HDMI ports (up to 4kp60 supported)Two-lane MIPI DSI display portTwo-lane MIPI CSI camera portFour-pole stereo audio and composite video portH.265 (4kp60 decode), H264 (1080p60 decode, 1080p30 encode)OpenGL ES 3.0 graphicsmicroSD card slot for loading operating system and data storage5V DC via USB-C connector (minimum 3A*)5V DC via GPIO header (minimum 3A*)Power over Ethernet (PoE) enabled (requires separate PoE HAT)Operating temperature: 0–50 degrees °C ambientThe following figure shows the available ports and some technical specifications of the Raspberry Pi 4:
Figure 1.3 – Raspberry Pi 4 ports and hardware specifications
Now that we are done with the hardware specifications, let’s move toward the available software options. As the Pi is a full-blown computer, it will run an operating system of its own. Hence, we will look at some popular operating systems available for the Raspberry Pi.
There are several operating systems available for the Raspberry Pi. We will look at some of the most popular operating systems listed on their official website:
Raspberry Pi OS (previously known as Raspbian OS).Ubuntu Core: Ubuntu operating system developed explicitly for embedded boards, with optimizations focused on security and reliability.LibreELEC: A distribution for multimedia applications based on the Kodi entertainment center.Ubuntu Desktop: This is the desktop version of Ubuntu supported on Raspberry Pi Model 3B+ and above. One of the most popular Linux operating systems used worldwide focused on daily applications for home, school, and work.Now that we have some knowledge about Raspberry Pi and MQTT, the next step is to learn how to setup our Raspberry Pi so that we can use it as a MQTT broker. That is exactly what the next section is about!
In this section, we will cover how to set up the Raspberry Pi. Although simple, it is crucial to perform each step as this will help us set up our own local MQTT broker on the Raspberry Pi. This will also help you understand how to get started with your new Raspberry Pi by installing an operating system onto it.
First, we will flash the popular Debian-based Raspberry Pi OS on our Raspberry Pi. After that, we will install all the necessary libraries and packages to make our device a local MQTT broker.
The topics we will be covering will be as follows:
Setting up an SD card for your Raspberry PiFlashing the OS image onto the SD cardSetting up your Raspberry Pi for the first timeSetting up VNC for the Raspberry PiSetting up and testing the MQTT brokerTesting the MQTT broker locallyFirst, we will discuss what will be required in terms of hardware to follow this setup process.
To follow the instructions provided in this section, you will need the following hardware:
Raspberry Pi (model 3B or higher, preferably Raspberry Pi 4)HDMI display (for first boot only)Keyboard and mouse (for first boot only)Power supply for the Pi (the official Pi power supply is recommended)MicroSD card (minimum 8 GB storage option is recommended)So, let us proceed to the next step, which is installing the official Raspberry Pi OS image and setting up the SD card.
In this step, the main aim is to get the microSD Card ready for the Raspberry Pi.
Important Note
If you have purchased a Raspberry Pi bundle with a pre-burnt SD card and some optional accessories, you can skip this step, as the SD card you have is already loaded with the required OS. But, if you would like to install a different OS, you can follow this step to do so.