31,19 €
This step-by-step guide will help you gain a deep understanding of the lightweight MQTT protocol. We’ll begin with the specific vocabulary of MQTT and its working modes, followed by installing a Mosquitto MQTT broker. Then, you will use best practices to secure the MQTT Mosquitto broker to ensure that only authorized clients are able to publish and receive messages. Once you have secured the broker with the appropriate configuration, you will develop a solution that controls a drone with Python.
Further on, you will use Python on a Raspberry Pi 3 board to process commands and Python on Intel Boards (Joule, Edison and Galileo). You will then connect to the MQTT broker, subscribe to topics, send messages, and receive messages in Python. You will also develop a solution that interacts with sensors in Java by working with MQTT messages.
Moving forward, you will work with an asynchronous API with callbacks to make the sensors interact with MQTT messages. Following the same process, you will develop an iOS app with Swift 3, build a website that uses WebSockets to connect to the MQTT broker, and control home automation devices with HTML5, JavaScript code, Node.js and MQTT messages
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 315
Veröffentlichungsjahr: 2017
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: April 2017
Production reference: 1050417
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78728-781-5
www.packtpub.com
Author
Gastón C. Hillar
Copy Editor
Muktikant Garimella
Reviewer
Ben Howes
Project Coordinator
Ulhas Kambali
Commissioning Editor
Smeet Thakkar
Proofreader
Safis Editing
Acquisition Editor
Reshma Raman
Indexer
Aishwarya Gangawane
Content Development Editor
Vikas Tiwari
Graphics
Abhinash Sahu
Technical Editor
Jijo Maliyekal
Production Coordinator
Arvindkumar Gupta
Gastón C. Hillar is Italian and has been working with computers since he was eight. He began programming with the legendary Texas TI-99/4A and Commodore 64 home computers in the early 80s. He has a bachelor's degree in computer science (graduated with honors), and an MBA (graduated with an outstanding thesis). At present, Gastón is an independent IT consultant and freelance author who is always looking for new adventures around the world.
He has been a senior contributing editor at Dr. Dobb’s and has written more than a hundred articles on software development topics. Gastón was also a former Microsoft MVP in technical computing. He has received the prestigious Intel® Black Belt Software Developer award eight times.
He is a guest blogger at Intel® Software Network (http://software.intel.com). You can reach him at [email protected] and follow him on Twitter at http://twitter.com/gastonhillar. Gastón’s blog is http://csharpmulticore.blogspot.com.
He lives with his wife, Vanesa, and his two sons, Kevin and Brandon.
At the time of writing this book, I was fortunate to work with an excellent team at Packt Publishing, whose contributions vastly improved the presentation of this book. Reshma Raman allowed me to provide her ideas for a book dedicated to MQTT, and I jumped into the exciting project of teaching the features of one of the most important protocols for IoT projects with examples in many different programming languages. Vikas Tiwari helped me realize my vision for this book and provided many sensible suggestions regarding the text, the format, and the flow. The reader will notice his great work. It’s been great working with Reshma and Vikas in another project and I can’t wait to work with them again. I would like to thank my technical reviewers and proofreaders for their thorough reviews and insightful comments. I was able to incorporate some of the knowledge and wisdom they have gained in their many years in the software development industry. This book was possible because they gave valuable feedback.
The entire process of writing a book requires a huge number of lonely hours. I wouldn’t be able to write an entire book without dedicating some time to play soccer against my sons, Kevin and Brandon, and my nephew, Nicolas. Of course, I never won a match. However, I did score a few goals.
Ben Howes is the founder and lead consultant at Zoetrope Ltd, a specialist IoT consultancy and product development firm in the UK. Ben has been creating connected hardware for over 10 years and has worked across projects spanning from start-ups to multinational deployments.
I’d like to thank Richard Webb, my co-founder, for starting Zoetrope with me.
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787287815.
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
To my sons, Kevin and Brandon, and my wife, Vanesa
MQTT is the preferred IoT publish-subscribe lightweight messaging protocol. If you develop IoT, web applications, mobile apps, or a combination of these solutions, you must learn how MQTT and its lightweight messaging system work. Of course, it is extremely important to take security into account when working with this protocol.
Most of the times, when you work with IoT solutions, you won’t be using a single programming language. Instead, you will use a combination of different programming languages and code running on different IoT boards that might use diverse operating systems. MQTT has its own specific vocabulary and different working modes. Learning MQTT is challenging because it includes too many abstract concepts that require real-life examples to be easy to understand.
This book will allow you to dive deep into the latest version of the MQTT protocol: 3.1.1. You will learn to work with Mosquitto MQTT server, command-line tools, and GUI tools to allow you to understand how everything works with MQTT and the possibilities that this protocol provides for your projects. You will learn security best practices and you will use them for a Mosquitto MQTT server.
This book will provide you with examples in the following programming languages, which are extremely popular in IoT, web applications, and mobile apps: Python, Java, HTML 5, JavaScript, Node.js, and Swift 3. You will control a drone, interact with sensors, control home automation, and interact with actuators by exchanging MQTT messages with the different programming languages and client libraries.
You will be able to run the examples on a wide range of modern IoT boards, such as Raspberry Pi 3, Raspberry Pi 2 Model B, Raspberry Pi Zero, Intel Joule, Intel Edison, and Intel Galileo Gen 2. However, any other board that supports the programming languages explained in the book will be able to run the samples.
Chapter 1, Lightweight Messaging with MQTT 3.1.1 and Mosquitto, starts our journey towards understanding the MQTT protocol. You will learn convenient scenarios for this protocol, the details of the publish-subscribe pattern, and message filtering. You will learn basic concepts related to MQTT and understand the different components: clients, brokers or servers, and connections. You will learn to install a Mosquitto broker or server on Linux, macOS, and Windows. You will learn best practices related to topics and single-level and multilevel wildcards. We will study in detail the different Quality of Service levels supported by MQTT and when it is convenient to use each of them. We will analyze their advantages and disadvantages.
Chapter 2, Securing an MQTT Mosquitto Server, covers generating a private certificate authority, a server certificate, and client certificates to enable TLS transport security and TLS client authentication with Mosquitto. Communications between the MQTT clients and the MQTT server will be encrypted. We will work with OpenSSL to generate the digital certificates. We will test the MQTT TLS configuration with MQTT.fx, MQTT-spy, and the Mosquitto command-line utilities. We will force Mosquitto to use only a specific TLS version. This way, we will understand how to encrypt communications between the MQTT clients and the Mosquitto MQTT server.
Chapter 3, Controling a Drone with Python, analyzes the requirements for controlling a drone with MQTT messages. We will define the topics that we will use and the commands that will be part of the messages’ payloads to control a drone. Then, we will work with the Paho Python Client to write Python code that connects an MQTT client to the MQTT server. We will understand the methods we need to call for the Paho Python Client and their parameters. We will analyze how callbacks work and we will be able to publish messages to topics, subscribe to topic filters, and receive and process messages. We will write code that processes commands for a drone with Python. The code will be able to run on different IoT platforms, including Intel Joule boards and Raspberry Pi 3, and also on any computer that is capable of executing Python 2.7.x or Python 3.x. We will work with the network loop for the MQTT client in Python. We will work with a GUI utility, command-line-based utilities, and, finally, with Python code to control the drone and display the feedback provided by the drone.
Chapter 4, Interacting with Sensors in Java, analyzes the requirements for controlling sensors wired to different IoT boards with MQTT messages and Java code. We will define the topics that we will use and the commands that will be part of the messages’ payloads to control sensors. We will take advantage of the last will and testament feature of MQTT with Java. We will work with the Eclipse Paho Java Client to write Java code that will connect a non-blocking (asynchronous) MQTT client to the MQTT server. We will create a security helper class to establish a TLS secured connection, and we will work with asynchronous notifications and callbacks in Java. We will subscribe to topics, understand how notification events work, and we will write code that processes commands that interact with sensors. Finally, we will work with a GUI utility to interact with our Java application through MQTT messages. We will be able to turn on and turn off the sensors and retrieve data from them.
Chapter 5, Controlling Home Automation in JavaScript, analyzes the requirements to control LEDs wired to different IoT boards with MQTT messages over WebSockets. We will define the topics that we will use and the commands that will be part of the message’s payloads to set the colors for LEDs. Then, we will create an HTML 5 web page combined with JavaScript and the Eclipse Paho JavaScript asynchronous client to control LEDs. We will write code to publish messages to target remote devices in JavaScript and we will be able to process received messages to provide feedback to the user. We will work with MQTT.js to write a Node.js script to process commands that simulate the interaction with LEDs on different IoT boards. We will be able to run the home automation web application and understand how all the pieces work together by exchanging MQTT over WebSockets messages. Finally, we will understand the necessary steps to work with MQTT over WebSockets with TLS.
Chapter 6, Interacting with Actuators in Swift, analyzes the requirements for controlling actuators wired to different IoT boards with MQTT messages. We will define the topics that we will use and the commands that will be part of the message’s payloads to turn on and turn off motors. Then, we will create an iOS app with Xcode and Swift 3, combined with the CocoaMQTT client, to control a motor. We will write code to publish messages to target remote devices in Swift 3 and we will be able to process received messages to provide feedback to the user. We will work with MQTT.js to write a Node.js script to process commands that simulate the interaction with motors on the different IoT boards. We will be able to run the iOS app and the Node.js script and understand how all the pieces work together by exchanging MQTT messages.
You will need any computer or device capable of running Python 3.x or greater, Chrome 56.x or greater, JDK 8 or greater, and Node.js 6.7.x or greater.
In order to work with Mosquitto 1.4.11 or greater, you will need a computer with any of the operating systems supported by this version of Mosquitto. The list is available at https://mosquitto.org/download/.
In order to work with Xcode 8.x, you will need a Mac computer capable of running OS X 10.11.5 or later, with 8 GB RAM.
This book is a great resource for developers who want to learn more about the MQTT protocol to apply it to their individual IoT projects. Prior knowledge of working with IoT devices is essential.
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/MQTT-Essentials-A-Lightweight-IoT-Protocol. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/MQTTEssentialsALightweightIoTProtocol_ColorImages.pdf.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
In this chapter, we will start our journey towards the usage of the preferred IoT publish-subscribe lightweight messaging protocol in diverse IoT solutions combined with mobile apps and web applications. We will learn how MQTT and its lightweight messaging system work. We will learn MQTT basics, the specific vocabulary for MQTT, and its working modes. We will use different utilities and diagrams to understand the most important concepts related to MQTT. We will:
Imagine that we have dozens of different devices that must exchange data between them. These devices have to request data to other devices and the devices that receive the requests must respond with the demanded data. The devices that requested the data must process the data received from the devices that responded with the demanded data.
The devices are Internet of Things (IoT) boards that have dozens of sensors wired to them. We have the following IoT boards with different processing powers: Raspberry Pi 3, Raspberry Pi Model B, Intel Edison, and Intel Joule 570x. Each of these boards has to be able to send and receive data. In addition, we want many mobile devices to be able to send and receive data, some of them running iOS and others Android. We have to work with many programming languages.
We want to send and receive data in near real time through the Internet and we might face some network problems, that is, our wireless networks are somewhat unreliable and we have some high-latency environments. Some devices have low power, many of them are powered by batteries and their resources are scarce. In addition, we must be careful with the network bandwidth usage because some of the devices use metered connections.
A metered connection is a network connection in which we have a limited amount of data usage per month. If we go over this amount of data, we get billed extra charges.
We can use HTTP requests and build a publish-subscribe model to exchange data between different devices. However, there is a protocol that has been specifically designed to be lighter than the HTTP 1.1 protocol and work better when unreliable networks are involved and connectivity is intermittent. The MQ Telemetry Transport (MQTT) is better suited for this scenario in which many devices have to exchange data between themselves in near real time through the Internet and we need to consume the least possible network bandwidth.
The MQTT protocol is a Machine-to-Machine (M2M) and IoT connectivity protocol. MQTT is a lightweight messaging protocol that works with a broker-based publish-subscribe mechanism and runs on top of Transmission Control Protocol / Internet Protocol (TCP/IP).
The following diagram shows the MQTT protocol on top of the TCP/IP stack:
The most popular versions of MQTT are 3.1 and 3.1.1. In this book, we will work with MQTT 3.1.1. Whenever we reference MQTT, we are talking about MQTT 3.1.1, that is, the newest version of the protocol. The MQTT 3.1.1 specification has been standardised by the OASIS consortium. In addition, MQTT 3.1.1 became an ISO standard (ISO/IEC 20922) in 2016.
MQTT is lighter than the HTTP 1.1 protocol, and therefore, it is a very interesting option whenever we have to send and receive data in near real time with a publish-subscribe model and we require the lowest possible footprint. MQTT is very popular in IoT, M2M, and embedded projects, but it is also gaining presence in web applications and mobile apps that require assured messaging and an efficient message distribution. As a summary, MQTT is suitable for the following application domains in which data exchange is required:
As a summary, MQTT was designed to be suitable to support the following typical challenges in IoT, M2M, embedded, and mobile applications:
Before we dive deep into MQTT, we must understand the publish-subscribe pattern, also known as the pub-sub pattern. In the publish-subscribe pattern, a client that publishes a message is decoupled from the other client or clients that receive the message. The clients don't know about the existence of the other clients. A client can publish messages of a specific type and only the clients that are interested in specific types of messages will receive the published messages.
The publish-subscribe pattern requires a broker, also known as server. All the clients establish a connection with the broker. The client that sends a message through the broker is known as the publisher. The broker filters the incoming messages and distributes them to the clients that are interested in the type of received messages. The clients that register to the broker as interested in specific types of messages are known as subscribers. Hence, both publishers and subscribers establish a connection with the broker.
It is easy to understand how things work with a simple diagram. The following diagram shows one publisher and two subscribers connected to a broker:
A Raspberry Pi 3 board with an altitude sensor wired to it is a publisher that establishes a connection with the broker. An iOS smartphone and an Android tablet are two subscribers that establish a connection with the broker.
The iOS smartphone indicates to the broker that it wants to subscribe to all the messages that belong to the sensor1/altitude topic. The Android tablet indicates the same to the broker. Hence, both the iOS smartphone and the Android tablet are subscribed to the sensor1/altitude topic.
A topic is a named logical channel and it is also referred to as a channel or subject. The broker will send publishers only the messages published to topics to which they are subscribed.
The Raspberry Pi 3 board publishes a message with 100 feet as the payload and sensor1/altitude as the topic. The board, that is, the publisher, sends the publish request to the broker.
The data for a message is known as payload. A message includes the topic to which it belongs and the payload.
The broker distributes the message to the two clients that are subscribed to the sensor1/altitude topic: the iOS smartphone and the Android tablet.
Publishers and subscribers are decoupled in space because they don't know each other. Publishers and subscribers don't have to run at the same time. The publisher can publish a message and the subscriber can receive it later. In addition, the publish operation isn't synchronized with the receive operation. A publisher requests the broker to publish a message and the different clients that have subscribed to the appropriate topic can receive the message at different times. The publisher can send messages as an asynchronous operation to avoid being blocked until the clients receive the messages. However, it is also possible to send a message to the broker as a synchronous operation with the broker and to continue the execution only after the operation was successful. In most cases, we will want to take advantage of asynchronous operations.
A publisher that requires sending a message to hundreds of clients can do it with a single publish operation to a broker. The broker is responsible for sending the published message to all the clients that have subscribed to the appropriate topic. Because publishers and subscribers are decoupled, the publisher doesn't know whether there is any subscriber that is going to listen to the messages it is going to send. Hence, sometimes it is necessary to make the subscriber become a publisher too and to publish a message indicating that it has received and processed a message. The specific requirements depend on the kind of solution we are building. MQTT offers many features that make our lives easier in many of the scenarios we have been analyzing.
The broker has to make sure that subscribers only receive the messages they are interested in. It is possible to filter messages based on different criteria in a publish-subscribe pattern. We will focus on analyzing topic-based filtering, also known as subject-based filtering.
Consider that each message belongs to a topic. When a publisher requests the broker to publish a message, it must specify both the topic and the message. The broker receives the message and delivers it to all the subscribers that have subscribed to the topic to which the message belongs.
The broker doesn't need to check the payload for the message to deliver it to the corresponding subscribers; it just needs to check the topic for each message that has arrived and needs to be filtered before publishing it to the corresponding subscribers.
A subscriber can subscribe to more than one topic. In this case, the broker has to make sure that the subscriber receives the messages that belong to all the topics to which it has subscribed. It is easy to understand how things work with another simple diagram. The following diagram shows two future publishers that haven't published any messages yet, a broker and two subscribers connected to the broker:
A Raspberry Pi 3 board with an altitude sensor wired to it and an Intel Edison board with a temperature sensor wired to it will be two publishers. An iOS smartphone and an Android tablet are two subscribers that establish a connection to the broker.
The iOS smartphone indicates to the broker that it wants to subscribe to all the messages that belong to the sensor1/altitude topic. The Android tablet indicates to the broker that it wants to subscribe to all the messages that belong to any of the following two topics: sensor1/altitude and sensor42/temperature. Hence, the Android tablet is subscribed to two topics while the iOS smartphone is subscribed to just one topic.
The following diagram shows what happens after the two publishers connect and publish messages to different topics through the broker:
The Raspberry Pi 3 board publishes a message with 120 feet as the payload and sensor1/altitude as the topic. The board, that is, the publisher, sends the publish request to the broker. The broker distributes the message to the two clients that are subscribed to the sensor1/altitude topic: the iOS smartphone and the Android tablet.
The Intel Edison board publishes a message with 75 F as the payload and sensor42/temperature as the topic. The board, that is, the publisher, sends the publish request to the broker. The broker distributes the message to the only client that is subscribed to the sensor42/temperature topic: the Android Tablet. Thus, the Android tablet receives two messages.
The MQTT broker is known as MQTT server in MQTT 3.1.1, and therefore, we will refer to it as the server. However, we must take into account that the documentation for MQTT servers, tools and client libraries can use the old MQTT broker name to refer to the server.
The MQTT server uses the previously explained topic-based filtering to filter and distribute messages to the appropriate subscribers. There are many MQTT server implementations that provide additional message filtering features by providing custom plug-ins. However, we will focus on the features that are part of the MQTT requirements.
As previously explained, in MQTT the publishers and the subscribers are completely decoupled. Publishers and subscribers are MQTT clients that only establish a connection with the MQTT server. An MQTT client can be both a publisher and a subscriber at the same time, that is, the client can publish messages to specific topics and also receive messages that belong to the topics to which the client has subscribed to.
There are MQTT client libraries available for the most popular programming languages and platforms. One of the most important things that we must consider when we select the MQTT client library is the list of MQTT features that they support and the ones that we need for our solution. Sometimes, we can choose between many libraries for a specific programming language and platform and some of them might not implement all the features. We will use many libraries for diverse programming languages and platforms throughout the book and we will learn how to evaluate whether a library is suitable or not for our needs. Specifically, we will work with Python, Java, JavaScript and Swift.
Any device that has a TCP/IP stack and is capable of using an MQTT library can become an MQTT client, that is, a publisher, a subscriber or both a publisher and a subscriber. The MQTT library makes it possible for the device to talk MQTT on top of TCP/IP and to interact with specific types of MQTT servers. For example, any of the following can become an MQTT client: an Arduino board, a Raspberry Pi 3 board, an Intel Joule board, an iPhone, an iPad, a laptop running Windows, a server running Linux, a Macbook running macOS, and Android tablet, among other devices.
There are many MQTT servers available for the most popular platforms, including Linux, Windows and macOS. Many of them are servers that can work as MQTT servers among other features. An MQTT server might implement only a subset of the MQTT capabilities and might have specific limitations. Hence, it is very important to check all the capabilities we will require in our solution before selecting an MQTT server. As happens with other middleware, we have open source versions, free versions and paid versions. Thus, we also have to make sure we select the appropriate MQTT server based on our budget and our specific needs.
Throughout this book, we will work with the Eclipse Mosquitto MQTT server (http://www.mosquitto.org
