Architectural Patterns and Techniques for Developing IoT Solutions E-Book

Jasbir Singh Dhaliwal

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 438

Veröffentlichungsjahr: 2023

Beschreibung

As the Internet of Things (IoT) expands and moves to new domains, architectural patterns need to enable faster digital transformation and more uniform development. Through numerous use cases and examples, this book helps you conceptualize and implement IoT architectural patterns and use them in diverse contexts in real-world scenarios.
The book begins by introducing you to a variety of IoT architectural patterns and then helps you understand how they are used in domains such as retail, smart manufacturing, consumer, smart cities, and smart agriculture. You’ll also find out how cross-cutting concerns such as security require special considerations in the IoT context. As you advance, you’ll discover all the nuances that are inherent in each layer of IoT reference architecture, including considerations related to analytics for edge/constrained devices, data visualizations, and so on. In the concluding chapters, you’ll explore emerging technologies such as blockchain, 3D printing, 5G, generative AI, quantum computing, and large language models (LLMs) that enhance IoT capabilities to realize broader applications.
By the end of this book, you’ll have learned to architect scalable, secure, and unique IoT solutions in any domain using the power of IoT architectural patterns, and you will be able to avoid the pitfalls that typically derail IoT projects.

Leseprobe

Architectural Patterns and Techniques for Developing IoT Solutions

Build IoT applications using digital twins, gateways, rule engines, AI/ML integration, and related patterns

Jasbir Singh Dhaliwal

BIRMINGHAM—MUMBAI

Architectural Patterns and Techniques for Developing 
IoT Solutions

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Preet Ahuja

Publishing Product Manager: Surbhi Suman

Senior Editor: Shruti Menon

Technical Editor: Nithik Cheruvakodan

Copy Editor: Safis Editing

Project Manager: Neil D'mello

Proofreader: Safis Editing

Indexer: Rekha Nair

Production Designer: Nilesh Mohite

Marketing Coordinator: Rohan Dobhal

Senior Marketing Coordinator: Linda Pearlson

First published: August 2023

Production reference: 1310823

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB.

ISBN 978-1-80324-549-2

www.packtpub.com

To my mother, Joginder Kaur, and to the memory of my father, Surjit Singh, for their immense sacrifices and teaching me the value of honest and sincere hard work. To my wife, Ramandeep Kaur, for being my loving partner and encouraging me to pursue my dreams, irrespective of the prevailing circumstances. To my lovely daughters, Kiranpreet Kaur and Manpreet Kaur, for ensuring that there is never a dull moment in our lives.

– Jasbir Singh Dhaliwal

Foreword

Computing has evolved from its humble roots in the 1940s, with an initial focus on the fundamentals of mathematical problems; to the 60s and 70s, where it was centered on symbolic systems, wherein the field first began to confront issues of complexity; to the 80s, with the rise of personal computing and the problems of human/computer interaction; then on to the 90s and our present century, addressing distributed and connected systems at scale. While these kinds of systems tend to dominate the narrative of computing, not every system is a Google, a Facebook, or an X, representing systems with soft edges and demanding elastic computing infrastructures at a global scale. There is another class of systems important to humanity; systems that touch and interact with the real world. Mind you, these kinds of systems have been present since the 40s with the advent of computers such as Whirlwind, but what is different is that now we see the intersection of cloud computing and the Internet of Things (IoT), with millions upon millions of sensors and actuators interfacing with the physical world.

This book is a comprehensive guide to building, deploying, and evolving software-intensive systems for IoT. You will find here solid, pragmatic advice on how to design such systems, how to evaluate them, and how to deliver them. Three things in particular delight me about this book: there is a clear emphasis on design patterns; broad coverage across problem domains, from manufacturing and agriculture to cities and beyond; and coverage of the connection of IoT systems to contemporary developments in artificial intelligence.

I found this to be a compelling, well-written, and very approachable book from which I learned some new things, and I hope you will too.

Grady Booch

ACM Fellow, IBM Fellow, IEEE Fellow, and IEEE Computing Pioneer

Contributors

About the author

Jasbir Singh Dhaliwal has over 26 years of software development and management experience, including 10 years in delivering complex IoT projects. Currently employed with IBM as a Principal Architect (IoT and cloud) and considered a thought leader with over 31 IoT patents, Jasbir has a deep understanding of IoT concepts/architectures, which is refined by delivering IoT projects in diverse domains such as consumer goods, smart buildings, healthcare, precision agriculture, automobile, and manufacturing. His extensive experience in both the public cloud and embedded domains gives him a unique edge in conceiving innovative end-to-end IoT solutions and applications. He holds a bachelor's degree in computer science and engineering from Punjab Engineering College, India.

About the reviewers

Amit Bahuguna has worked for over two decades in various software engineering roles, across different geographies, covering the entire software development life cycle, for both government and private enterprises. Amit has specialized in IT architecture over the last 15 years and has worked on the architectures of several operational technologies and IoT projects during this period. He received a B.E. in electronic and communications engineering from Manipal Institute of Technology, and a master’s in software systems engineering from the University of Melbourne. He is currently employed as a Solution Architect with the ICT department of the University of Sydney. He is also a member and certified professional of the Australian Computer Society.

I am thankful to my family, friends, and colleagues for sharing their valuable time and experience in acquainting me with, teaching me about, and enriching my knowledge of ICT. I am grateful to the open communities where access to information and learning are unrestricted and people are interested in helping one another, and I am continually humbled by how much I don’t know.

Atif Ali is a mechatronics engineer specializing in automation, robotics, and IIoT, with a focus on electro-mechanical design and software integration for complex physical systems and processes. He is a passionate open source advocate, focused on the intersection of hardware and software for perceptive physical world observability. Atif has been heavily involved in both industry and academia for the past several years, in sectors ranging from clean energy and web3 to cybersecurity and mining, holding positions at ETH Zürich, UBC Vancouver, and Grafana, among others. He also served as a technical advisor for various start-ups, and was on the faculty board for the Global Masters in IoT at the Zigurat Global Institute of Technology.

Bart Szczepanski, a co-founder of a Polish IoT firm named Rebels Software, assists customers with attentive care and guides them through the world of IoT. At Rebels Software, Bart ensures that customers receive the necessary support and expertise in navigating the complexities of IoT. He holds a master’s degree in electronics and telecommunication from Wrocław University of Technology and takes pride in his roles as a dedicated husband and a father to three children.

Philipp Staeheli is an experienced IoT Solution Architect working for an international and innovation-driven cleantech company. He has a deep understanding of IoT from both technical and organizational/business aspects. Prior to his current role, he worked for several years as a hardware and software developer as well as a project leader across different industries. Philipp is passionate about creating customer value and sharing his knowledge with others. He holds a degree in electrical engineering as well as business administration and has obtained several certifications in software architecture, cybersecurity, and Agile methods.

Acknowledgments

I would like to thank the following people:

Akhilesh Gupta, Principal Architect, FEV India Pvt. Ltd. – automotive connectivity and connected vehicles, for helping me to differentiate between IoT and general cybersecurity. His guidance helped me understand the challenges in securing IoT devices, especially connected vehicles.

Amar Mundi, Principal Architect, HCL Technologies, is one of the best technical minds and is always a phone call away for any kind of support. His guidance was instrumental in shaping a few of the initial chapters.

Amarjit Singh Dhaliwal, Sant Longowal Institute of Engineering and Technology (SLIET), for introducing me to the relevant people in academia. His support has ensured that this book has a good balance of theoretical concepts as well as practical guidance.

Glen D. Singh from the Packt team, for helping a budding author like me to understand the nuances of content preparation and presentation and guiding me to take the book to the right audience.

Grady Booch, IBM Fellow, for providing consistent mentoring and encouragement. He is one of the most generous souls that I have encountered in my professional life. His encouragement is the sole reason that this book was able to see the light of day.

Harsha Vacher, founder of KTech Labs Inc, for providing insights into IoT security.

Jerry Cuomo, IBM Fellow, for providing guidance on how to accelerate the pace of content preparation while not compromising on overall quality.

Karnail Singh Sidhu and Narinder Sidhu, the owners of Kalala Wines, Canada, for sharing their perspectives on the nuances involved in wine/grape farming. Their knowledge of the agriculture domain is truly impressive.

Kavita Mittal, Application Security Specialist at Thoughtworks, for enlightening me on the differences between IoT and IT security. I am always impressed by her technical know-how and supportive nature.

Kim Bartkowski, IBM Distinguished Designer, for sparing time from a very busy schedule to give me tips on how to enhance the visual appeal of the diagrams.

Neetika Jain, IBM Delivery Manager, for being a constant guide and helping in fine-tuning my understanding of concepts related to analytics, security, and a host of other topics.

Nihar Kapadia from the Packt team, for reviewing the content in detail and providing constructive input.

Professor Kalsi, SLIET, Longowal, for guiding me on the nuances involved in smart agriculture and its significance in an Indian context.

Raghu Ramaswamy, IBM Executive IT Architect, for your constructive feedback and encouraging words.

Rohan Dobhal from the Packt team, for helping me understand the mechanics of efficient marketing and promotion.

Sameep Mehta, Distinguished Engineer at IBM, for sharing personal experiences of writing a book. His ability to churn out content in a short time inspired me to recalibrate my writing speed, ensuring that the manuscript was completed on time.

Sean Lobo and Neil D'mello from the Packt team, for keeping me on my toes and ensuring that all chapter submissions were done on time.

Shivani Aggarwal, Solution Architect, HCL Technologies, for sharing insights regarding how to provide security for constrained devices.

Shruti Menon from the Packt team, for the diligent editorial reviews and always going above and beyond to ensure that the content was crisp yet complete. Her attention to detail is exemplary.

Sonia Mezzetta, Program Director, Product Management, Data Fabric Solution Architecture at IBM, for sharing her experiences of publishing a book and practical tips for ensuring the book’s wider circulation.

Surbhi Suman from the Packt team, for always being patient even when answering a list of long-winded queries. Her ability to get things done is amazing.

Ulrike Vauth, Distinguished Engineer at IBM, for regular mentoring sessions. Your guidance on how to balance professional and personal aspirations helped me to effectively juggle both worlds.

Last but not the least, special thanks to those who intended to help but were genuinely hard-pressed for time.

Table of Contents

Preface

Part 1: Understanding IoT Patterns

1

Introduction to IoT Patterns

An overview of IoT

IoT reference architecture

Unique requirements of IoT use cases

Recommended architecture principles and considerations

Summary

2

IoT Patterns for Field Devices

Device gateway

Pattern summary

Digital twin

Pattern summary

Device management

Pattern summary

Summary

3

IoT Patterns for the Central Server

AI/ML integration

Pattern summary

The rule engine

Pattern summary

File upload

Pattern summary

Enterprise system integration

Pattern summary

Summary

Part 2: IoT Patterns in Action

4

Pattern Implementation in the Consumer Domain

Use case – deploying home automation

A detailed description of the use case

Use case – a smart egg boiler

A detailed description of the use case

Realizing the use case

Summary

5

Pattern Implementation in the Smart City Domain

A smart speaker for modernizing education

Monitoring the condition of perishable goods

Driver behavior monitoring

Automatic replenishment of consumables and raw materials

Additional use cases

Summary

6

Pattern Implementation in the Retail Domain

An overview of the retail domain

Using real-time IoT data

Implementing sensors and actuators in retail stores

Use case – real-time tracking in retail outlets

Privacy concerns

Leveraging IoT patterns

Summary

7

Pattern Implementation in the Manufacturing Domain

An overview of smart manufacturing

Key terms/definitions

Exploring the evolution of the manufacturing domain

The benefits of smart manufacturing

Challenges in transitioning from traditional to smart manufacturing

Automatic inspection of finished goods or parts

Summary

8

Pattern Implementation in the Agriculture Domain

An overview of smart agriculture

Key terms/definitions

Factors influencing greater adoption of smart agriculture technologies

Use cases of IoT in smart agriculture

Resolving agricultural challenges with a land consolidation platform

Mitigating agricultural challenges

Summary

Part 3: Implementation Considerations

9

Sensor and Actuator Selection Guidelines

Key terms/definitions

Usage scenarios of sensors

Operation and usage scenarios of actuators

Key characteristics of actuator types

Use cases for actuators

Use case – connected coffee vending machine

Factors to be considered while selecting a sensor or actuator

Introducing wireless sensor and actuator networks

Summary

10

Analytics in the IoT Context

Key terms/definitions

Implementing IoT analytics

Stages of implementing IoT analytics

Integrating ML capabilities into IoT analytics

Understanding the importance of data quality

Relevance of edge analytics

Considerations for IoT visualization

Summary

11

Security in the IoT Context

Key terms/definitions

Comparing IoT security and IT security

Challenges in securing IoT solutions

IoT security vulnerabilities

Major IoT security breaches

Mitigating IoT security vulnerabilities

Domain-specific security considerations

Applicable security standards and best practices

Summary

Part 4: Extending IoT Solutions

12

Exploring Synergies with Emerging Technologies

Blockchain

Benefits of combining IoT and blockchain

Possible use cases

Generative AI

Benefits of combining IoT and generative AI

Possible use cases

Large language models

Benefits of combining IoT and LLM

Possible use cases

AI/ML

Benefits of combining IoT and AI/ML

Possible use cases

Immersive technologies

Benefits of combining IoT with immersive technologies

Possible use cases

3D and 4D printing

Benefits of combining IoT with 3D and 4D printing

Possible use cases

5G and 6G technology

Benefits of combining IoT with 5G and 6G technologies

Possible use cases

Drones

Possible use cases

Social media

Benefits of combining IoT and social media

Possible use cases

Cloud computing

Benefits of combining IoT and cloud computing

Possible use cases

Energy harvesting

Benefits of combining IoT and energy harvesting

Possible use cases

Quantum computing

Benefits of combining IoT and quantum computing

Possible use cases

Web 3.0

Benefits of combining IoT and Web 3.0

Possible use cases

Edge computing

Benefits of combining IoT and edge computing

Possible use cases

Summary

13

Epilogue

Project implementation considerations

Lessons learned from IoT projects

NFR considerations

Battery or power optimization

Cost optimization

IoT connectivity protocols

Summary

Index

Other Books You May Enjoy

Preface

The book helps you to apply modern architectural patterns and techniques to achieve scalability, resilience, and security in intelligent IoT solutions built for diverse domains such as manufacturing and industry, consumer goods, agriculture, and smart city applications.

Who this book is for

This book is for IoT systems and solutions architects, as well as other IoT practitioners such as developers, technical program and pre-sales managers, and so on, who are interested in understanding how various IoT architectural patterns and techniques can be applied for developing unique and diverse IoT applications.

What this book covers

Chapter 1, Introduction to IoT Patterns, provides basic knowledge about IoT concepts that will help in understanding the architectural patterns and use cases detailed in subsequent chapters.

Chapter 2, IoT Patterns for Field Devices, lists the architectural patterns that are relevant to field devices, including device gateways, digital twins, and device management.

Chapter 3, IoT Patterns for the Central Server, discusses the architectural patterns that are relevant to a central server, such as AI/ML integration, rule engines, file upload, and enterprise system integration.

Chapter 4, Pattern Implementation in the Consumer Domain, explores how the patterns covered in the previous chapters can be combined to realize use cases (home automation and smart egg boilers) in the consumer domain.

Chapter 5, Pattern Implementation in the Smart City Domain, offers insights into how architectural patterns can help in realizing use cases in the smart city domain, including smart speakers, condition monitoring for perishable goods, driver behavior monitoring, and the automatic replenishment of consumables.

Chapter 6, Pattern Implementation in the Retail Domain, explains how the patterns learned in the previous chapters can help in realizing use cases (real-time tracking in retail outlets) that are relevant to the retail domain. Also, the chapter lists the retail domain-specific concepts that are related to IoT solutions.

Chapter 7, Pattern Implementation in the Manufacturing Domain, starts with the required know-how about smart manufacturing and then details the implementation of a use case (the automatic inspection of finished goods) using IoT architectural patterns.

Chapter 8, Pattern Implementation in the Agriculture Domain, describes the benefits of integrating IoT with the agricultural domain and also provides details about the implementation of a specific use case – a land consolidation platform.

Chapter 9, Sensor and Actuator Selection Guidelines, provides details about key concepts related to sensors and actuators and outlines the guidelines for selecting the most appropriate sensor or actuator depending on the use case requirements and related constraints.

Chapter 10, Analytics in the IoT Context, presents details about how the ingested IoT data can be used to generate insights. The chapter focuses on analytics as it relates to IoT implementations.

Chapter 11, Security in the IoT Context, discusses the specific considerations that need to be taken to ensure that IoT solutions are completely secure.

Chapter 12, Exploring Synergies with Emerging Technologies, explores the potential of combining IoT with other emerging technologies (such as blockchain, generative AI, 3D printing, and AR/VR) to create more powerful applications/use cases.

Chapter 13, Epilogue, identifies the practical challenges that are typically encountered while implementing IoT solutions as well as specific tips for mitigating those challenges. It also lists the key learnings that the author had while working on IoT projects.

To get the most out of this book

Before reading this book, the reader should acquire basic know-how about IoT. Prior knowledge of IoT’s fundamental concepts and its application areas is good to have before reading this book but is not mandatory.

Image credits

Several images in Chapters 4, 6 to 10, 12, and 13 have been created using assets from freepik.comand flaticon.com.

Conventions used

There are a number of text conventions used throughout this book.

Bold: Indicates a new term, an important word, or words that you see onscreen. Here is an example: “Devices such as video cameras send the data to a Device Gateway (DG) over protocols such as Wi-Fi.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message. You can also contact the author on LinkedIn (https://www.linkedin.com/in/jasbir-singh-dhaliwal-617a193) or via email ([email protected]).

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Architectural Patterns and Techniques to Develop IoT Solutions, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily.

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below:

https://packt.link/free-ebook/9781803245492

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Understanding IoT Patterns

Readers will be made aware of the fundamental IoT patterns that they can use when architecting IoT solutions. Each chapter will start with details about a pattern, its significance, and the type of scenarios under which the pattern would be usable.

This part comprises the following chapters:

Chapter 1, Introduction to IoT PatternsChapter 2, IoT Patterns for Field DevicesChapter 3, IoT Patterns for the Central Server

1

Introduction to IoT Patterns

The Internet of Things (IoT) has gained significant traction in the recent past and this field is poised for exponential growth in the coming years. This growth will span all the major domains/industry verticals, including consumer, home, manufacturing, health, travel, and transportation. This book will provide a novel perspective to those who want to understand the fundamental IoT patterns and how these patterns can be mixed and matched to implement unique and diverse IoT applications.

This introductory chapter details the architectural considerations that you must bear in mind while designing IoT solutions. Architecting IoT solutions is challenging as there are additional complexities due to the physical hardware selection, complex integrations, and connectivity requirements involved. This chapter also serves as a foundation for the patterns that will be introduced in the subsequent chapters.

In this chapter, we will cover the following topics:

An overview of IoTIoT reference architectureUnique requirements of IoT use casesRecommended architecture principles and considerations

An overview of IoT

IoT has generated a lot of interest recently and has moved from a purely academic pursuit to the point where real use cases are being realized. IoT implementation is inherently complex due to multiple and diverse technologies (embedded, cloud, edge, big data, artificial intelligence (AI), machine learning (ML), and so on) being involved and due to the range of deployment options that are available (constraint devices in the field to the almost unlimited availablity of compute and other resources in the cloud). IoT enables diverse use cases and spans multiple domains (home automation, healthcare, track-and-trace, connected vehicles, autonomous driving, and more).

The relevance of IoT is only going to increase in the coming years because of the following reasons:

IoT use cases encompass physical and virtual worlds and as a result, interesting and rich use cases can be developed (compared to purely virtual/software systems such as word processors, ERP systems, and more). It can be said that the scope and variety of IoT use cases is only limited by a person’s imagination.The immense potential of IoT has been validated by both academics and implementors. This can be attributed to the following reasons:The increased capability and efficiency of hardware components with a continous decrease in cost (and size) in line with Moore’s law. Efficient battery utilization by the current generation of hardware components has also reduced the hassles of frequent battery replacement.The rise of commercial cloud providers (hyperscalers), which enables unlimited scalability in terms of compute, storage, complex analytics, high-speed data ingestion, and more. These characteristics are very well suited to the needs of IoT applications. The following are some of the services that are provided by public cloud providers/hyperscalers and that can be leveraged while developing IoT use cases:Device managementFirmware updatesEdge management/analyticsDevice/data securityDigital twinsIoT analyticsData ingestionData visualizationsData storageVideo analyticsNotificationsUbiquitous and low-cost connectivity in the form of traditional connectivity options (for example, Wi-Fi, 3G, and 4G) as well as connectivity options such as LoRaWAN and NB-IoT. Technologies such as NB-IoT are especially useful for IoT as they support long-range connectivity and provide long battery life. The advent of 5G has further enlarged the scope of IoT use cases by providing high bandwidth and minimal latency.The increased maturity of related technologies such as blockchain, robotics, AI/ML, energy harvesting, AR/VR, drones, social media, and more. These technologies enable IoT practitioners to augment IoT capabilities with the capabilities provided by these technologies to push the boundaries of innovation further and envisage non-conventional ideas.The increased adoption of mobile and wearable devices. These devices enable anytime access to IoT data and help control and monitor IoT devices in real time.Increased market competition, which forces enterprises to treat data as the fulcrum for decision-making as well as monetization opportunities. IoT also acts as the foundation for operationalizing additional revenue models such as service revenue over and above product sale revenue.

It is important to understand how IoT systems are different from non-IoT systems. A few of the key differentiators are as follows:

Humans play a vital role in operating and managing most non-IoT (traditional IT/OT systems) systems, whereas IoT systems are designed to operate on their own or with minimal human intervention.IoT devices are constrained in terms of the compute, storage, or both, whereas most non-IoT applications are deployed on standard workstations where an ample amount of storage and compute is available.IoT applications, once deployed, are expected to last for years (10 to 15 years is the norm in the manufacturing industry) compared to non-IoT applications, where the shelf life is less (typical refresh cycles range between 3 to 5 years). Accordingly, IoT systems must be architected by balancing both current and long term needs.Considerable heterogeneity is observed in the selection of the hardware/software components as well as connectivity protocols. This is because there are different technologies to choose from, and for each technology choice, there are multiple implementations and offerings available from vendors. In comparison, there are fewer technology options possible in non-IoT systems.There are differences in the characteristics of the data that is generated in IoT and non-IoT systems. All the seven V’s of big data (velocity, variety, variability, volume, veracity, visualization, and value) are high in IoT systems compared to non-IoT systems.Very few IoT systems operate in isolation and are generally integrated with other enterprise systems. In IoT systems, there is a need to integrate Information Technology (IT) and Operational Technology (OT), as well as hardware devices. This presents an entirely new level of integration complexity that is rarely seen in traditional systems.Security is important in any connected system, but it becomes much more important in IoT as the attacks can result in physical harm (industrial robots gone rogue) in addition to reputation/financial loss. Additionally, most IoT field devices are installed in vulnerable environments where they can easily be tampered with. Therefore, the attack surface in an IoT use case is much larger than that of a non-IoT use case.

These unique characteristics of IoT systems can be visualized in the following diagram:

Figure 1.1 – Unique characteristics of an IoT system

This complexity can be quite daunting to anyone who has just ventured into the IoT domain. Although a rich variety of IoT use cases (or solution domains) is possible, there is also a certain degree of commonality that is present in most of the IoT use cases and related architectures. We have mentioned these similarities so that anyone who is new to this domain can understand the existing architectures and use cases.

IoT reference architecture

The IoT reference architecture follows a layered model, as shown in the following diagram:

Figure 1.2 – Layered IoT reference architecture

Let’s look at these layers in more detail:

Perception/actuation layer: This layer indicates the physical layer where sensors (pressure, temperature, and so on) gather information about the environment. In turn, the environment is affected by the actuators (electric motor, thermostat control, and so on).Connectivity layer: This layer provides the connectivity required to send data (perception data from sensors and control commands to actuators, and so on) to/from the aggregation/processing layer. This layer is realized by leveraging connectivity options (5G, Wi-Fi, NB-IoT, LoRA, and so on). The decision to choose a specific connectivity option depends on various factors such as range and bandwidth.Processing layer: The processing layer ingests, analyzes, and stores data received from the connectivity layer. The processing can be performed either near the data source (edge computing) or in a private/public cloud. Data processing and storage elements, such as databases, data streaming engines, and AI/ML algorithms, form part of this layer.Services layer: This layer connects the processing layer to the application layer. Another way of looking at this layer is considering it as a set of APIs that can be consumed by the application layer to develop IoT applications such as smart homes, precision agriculture, smart manufacturing, and more.Application layer: This layer represents the applications that are to be used by end users. These applications are typically hosted at the edge or in the cloud (central server) and are consumed using mobile devices as mobile apps. Alternatively, they can be deployed on a web server and accessed using browsers.

The IoT patterns listed in subsequent chapters will align with the IoT reference architecture that we just discussed. Additionally, other important IoT topics listed in the latter part of the book (such as data analytics and IoT security) will also build upon the understanding of this concept.

The layered reference architecture provides various benefits, such as independent scalability of different layers and enhanced maintainability as change is restricted to specific layers. The IoT patterns will help you develop the required functionality at the specific layer in less time and in a reproducible fashion. The architectural patterns (detailed in subsequent chapters) that are relevant to the different layers of the reference architecture are shown in the following diagram:

Figure 1.3 – IoT patterns realized at different layers of the reference architecture

Next, we will look at the unique requirements that we should be aware of while implementing IoT use cases.

Unique requirements of IoT use cases

IoT use cases tend to have very unique requirements concerning power consumption, bandwidth, analytics, and more. Additionally, the inherent complexity of IoT implementations (computationally challenged field devices on one end of the spectrum vis-à-vis almost infinite capacity of the cloud on the other) forces architects to make difficult architectural decisions and implementation choices. The diversity of the available implementation technologies and the absence of well-established standards are additional challenges that makes architecture decisions difficult.

This book attempts to alleviate some of the challenges associated with architecting IoT use cases by identifying the commonalities between the architectures that can support these use cases. It is important not to get blindsided by the diversity of the use cases and recognize the fact that diversity exists at the superficial level and under the hood. This book intends to bridge this gap in the current understanding by demonstrating how the implementation of diverse IoT use cases can be traced back to a handful of architectural patterns.

Before presenting the various IoT patterns, it is worth mentioning the unique expectations from IoT architectures that are different from non-IoT architectures:

Sensing events and actuation commands have a wide range of latency expectations – from real-time to fire and forget.Data analysis results need to be reported/visualized/consumed on a variety of consumer devices – mobiles, desktops, tablets, and more. Similarly, data consumers have diverse backgrounds, data needs, and application roles (personas).One is often forced to integrate with legacy as well as cutting-edge devices and/or external systems – very few trivial use cases have isolated/standalone architectures. There is a considerable difference in the way the data is extracted from legacy versus non-legacy systems – legacy systems may internally collate the data and then push it to the external port (file transfer), whereas newer systems may push the data in a continuous stream (time-series data). This variability is one of the critical considerations when choosing a particular IoT architectural pattern.Varied deployment requirements – edge, on-premise, hybrid, the cloud, and more.Adherence to strict regulatory compliances, especially in medical and aeronautical domains.There are expectations considering immediate payback, return on investment (ROI), business outcomes, and new service business models.Continuous innovation, which results in new services or offerings (especially by cloud vendors), forcing IoT architectures to be in continuous sync mode with these new offerings or services.The scarcity of skilled architects who can formulate end-to-end IoT solutions – although people with specific skills sets might be available (device architects, connectivity architects, and cloud architects); however, there are very few end-to-end IoT architects.No common standard for devices, device connectivity, IoT protocols, or the message transport layer leads to complex device management.Typically, IoT stacks don’t operate in isolation and any non-trivial deployed IoT solution would need to integrate with other external systems (ERPs, AMDBs, MESs, and so on). Even here, there is no standard for how to integrate these systems seamlessly. The external systems typically predate IoT deployments by decades and are heavily customized with no consideration for integration needs.From one perspective, IoT implementation is a process automation initiative. In general, the process exists but is performed manually and IoT is expected to automate the process either partially or fully. Generally, these existing workflows are not documented and exist as part of tribal knowledge of the process practitioners, which poses challenges for IoT architects as they don’t have clarity regarding the processes and workflows. Hence, they face a dilemma regarding which subprocesses should be automated to maximize their ROI – they have to decide if they are content with minor improvements (local optimization) and forgo benefits that can be accrued by considering global optimizations.Device life cycle management is a challenge in domains such as cardio medical devices as they can’t afford downtime but still need a timely firmware update (especially patches related to security fixes, which can’t be deferred beyond a certain point).The need to calibrate field sensors at regular intervals poses a challenge. The rate of drift varies from sensor to sensor and from one environment to another. There is a tendency to compensate for this drift by applying AI/ML models at the edge or in the cloud, but these steps are far from ideal as they lack accuracy and may not fully consider local or ambient conditions.Use cases that rely on positional information tend to have limited acceptance as all the locational sensors (indoor or outdoor) have limited accuracy.The migration of massive amounts of edge-processed historical data (accumulated over decades) to the cloud is another key architectural challenge that is observed in many Machine-to-Machine (M2M) to IoT transformation initiatives.The desired non-functional requirement (NFR) (scalability, availability, security, data residency/privacy, and so on) values vary from use case to use case and add another layer of complexity.Consumers of IoT data have diverse backgrounds (for example, the information needs of a home automation user would differ widely from an industrial user who wants to monitor plant uptime, which, in turn, would be different from the needs of paramedical staff using IoT for automated clinical trials), so they have different ways of operating and leveraging IoT systems. Although this may seem to have more bearing on device UI design, it can impact the solution architecture in subtle ways as well.

In the next section, we will list the architectural principles or considerations that will help you address the unique requirements of implementing IoT solutions.

Recommended architecture principles and considerations

Certain principles, which ensure that architectures, once realized, are scalable, modifiable, robust, and fault-tolerant are especially relevant for IoT architectures. Let’s take a look at some of these:

Built on open communication protocols to support diverse device communication needs: As IoT is an amalgamation of real (hardware) and virtual (software) realms, each of which evolves at its own independent pace. Robust IoT architectures should be flexible enough to support current and possible future enhancements in both these realms – for example, on the one hand, continual advancements are made for connectivity/power capabilities on the device/hardware side, while on the other hand, there are central server side advances regarding analytics and AI/ML capabilities. Hence, there is an inherent impedance mismatch between real and virtual worlds (concerning the rate as well as nature of these enhancements). IoT architects should not only be aware of this mismatch but should also incorporate the required considerations to support the use case requirements for a longer time frame. These requirements are partially handled by adhering to a layered architecture whereby the components in a specific layer can be plugged in or plugged out with minimal impact on the overall architecture.Designed for “end-to-end” security: Security is an important consideration for any software system, especially in cases where data or commands are communicated over public communication channels. However, in terms of IoT, security requires deeper consideration, primarily due to two reasons:Actions initiated in the real/physical world can’t be rescinded, unlike the actions in the virtual/software world: An irrigation pump that is instructed (maliciously) to start pumping water in an agriculture field would have pumped considerable water before someone detects the anomaly and initiates corrective action. This contrasts with the scenario in the software world, where a simple update instruction is sufficient to undo/roll back database changes. Scenarios can be even more disastrous in domains such as healthcare, where IoT systems often control human life (for example, an oxygen ventilator controlled by an IoT system).The attack vector is considerably broader compared to pure software systems: This is because the complete data pipeline (end device > gateway > communication channel > central server > application) needs to be secured and each entity in the data pipeline has diverse applicable security requirements – end devices (with their inherently constrained compute/storage capabilities) can’t support the security rigor that the central server can support, so each component’s security vulnerabilities and the relevant security guardrails need to be independently analyzed. Similarly, data should be protected in transit as well as at rest at all times.Enterprise integration enabled by the “API-first” approach: Any production-grade IoT system will typically be integrated with other external systems to deliver full value. Real-world data collated by IoT systems is fed (data push) into external systems to enable richer use cases. Similarly, data from the external systems (data pull) is used to enrich the collated data. This type of integration is not possible unless the IoT system has been architected with API-first as one of the core architectural tenants whereby IoT data can be consumed by enterprise applications. These APIs also enable workflows that span both IoT and non-IoT (that is, external systems).Satisfy diverse data needs: IoT systems are leveraged by a diverse set of users, each with different backgrounds and information needs. Accordingly, it is important to capture the raw data needs of all the (current and future) stakeholders and to present the data in a way that is easily assimilable by a diverse set of stakeholders (personas). Role-based access control (RBAC) is one mechanism that shows the required information to stakeholders while at the same time obscuring non-relevant information. Also, some of the stakeholders will have real-time data needs (operators who want real-time notifications for emergency alarms), whereas others will want insights from consolidated data (batch processing). Decoupling data ingestion from data processing is one such principle that enables us to accomplish this need. Some of the other data collation/manipulation requirements are listed as follows:Diverse (structured, semi-structured, and unstructured) operational data from sources such as Manufacturing Execution Systems (MESs) and Laboratory Information Management Systems (LIMSs) should be consolidated in a common data store (data lake) either at the edge, the cloud, or both.Separating streaming, batch, and right-time data pipelines for scalability, efficiency, and cost optimization considerations. De-coupling data producers from consumers ensures a robust architecture as well as the flexibility of technology and implementation choices.Technology-neutral architecture providing deployment flexibility: IoT systems can be deployed in different configurations, such as on-premise, public cloud, private cloud, and/or hybrid multi-cloud configurations, depending on the customer’s sensitivity to security as well as governance and regulatory needs. Considering this, the architecture should be generic enough that it can cater to diverse deployment needs and can be supported by multiple technology stacks. This is generally achieved by creating an IoT reference architecture (devoid of specific technology choices) and then transitioning to a technical architecture (where generic architectural components are replaced by specific technology components).Design for high availability: Although the need for high availability varies widely from one IoT use case to another, some use cases are categorized as mission-critical with almost zero downtime expectations, whereas others can accommodate a considerable downtime period. The central server architecture should mimic the uptime expectations as typically, less downtime translates into higher costs. In the context of IoT, high availability must be considered from an overall system perspective. For example, in scenarios where longer central server downtime is acceptable, end devices need to have higher data buffering capabilities (that is, greater storage space) to minimize data loss.Support for “unlimited scalability”: IoT deployments start small with a few end devices but tend to scale to a large number in a short duration. As a result, generally, in IoT solutions, horizontal scalability is preferred over vertical scalability.Device communication considerations: Data is communicated over a bi-directional communication channel between the gateway and central server. This channel can be supported by multiple communication technologies (with some of the common ones being cellular, Wi-Fi, LoRa, and SigFox). Considerations such as range (physical distance from the central server), payload size, battery life, and ambient noise play a role in finalizing the ideal communication technology for a particular IoT implementation. Some of the other considerations from the device side include the ability to store/buffer data in case of connectivity loss to central server, sleep/wakeup logic for conserving battery power, and data aggregation/filtering needs.

The following diagram summarizes the key architectural principles/considerations discussed in this section:

Figure 1.4 – Architectural considerations for developing IoT solutions

Summary

This introductory chapter helped you understand the architectural considerations need to be considered while developing or deploying IoT solutions. Additionally, the chapter provided contextual knowledge that will help you understand the patterns listed in this book. The characteristics that make IoT solutions different from other traditional software systems or IT solutions were discussed, along with information about the different layers of the IoT reference architecture. In the next two chapters, we will dive deep into the IoT architectural patterns.

2

IoT Patterns for Field Devices

This chapter lists the key patterns that are relevant to field devices or Things. After reading this chapter, you will be able to identify the existence of these patterns in IoT architectures. It provides details regarding the scenarios in which the patterns are suitable or applicable, along with the constraints that need to be considered. This will help you understand existing IoT architectures with relative ease.

This chapter covers the following three key patterns:

Device gateway (DG): A DG serves as a bridge between field devices (sensors, actuators, and so on) and the central server. In a standalone deployment (without the central server), DG coordinates actions between local devices (sensors and actuators).Digital twin (DT): DT is used to maintain a virtual state of the field devices on the central server, thereby allowing for remote monitoring and control operations. By performing the required processing on the accumulated data, DT makes it possible to predict the future state of the field devices. In addition, DT helps to overcome intermittent connectivity isues.Device management: Device management helps configure, update, and manage the field devices and is hosted on the central server.

Let’s look at these patterns in further detail.

Device gateway

DG is an important pattern as it helps link physical and virtual worlds. The physical world is monitored by sensors and actions are initiated by actuators, as per the commands that are sent by a DG. The notation that is used for the DG in this book is shown in the following diagram:

Figure 2.1 – Notation for the DG pattern

Important Note

The DG is also referred to as the Field Gateway in the IoT literature.

In addition to its role in enabling edge/local intelligence by hosting a Local Rule Engine (LRE) and performing latency-sensitive decisions, DG enables data communication with the central server, where more complex decisions (those requiring a global context) must be made. The need for DG arises because most sensors/actuators are constrained in terms of compute, memory, storage, or power, so they can’t establish connectivity with the central server. A good practical example of DG is a smartphone as it connects to multiple devices (for example, headphones, lights over BLE, and so on) and sends the data over HTTP/MQTT to the central server. DG is functionally superior to routers as it can execute business logic at the edge rather than just routing traffic.

Another perspective is that the DG pattern encapsulates the different protocols or data formats in which sensors/actuators typically communicate. As there is no standardization of either the communication protocol (BLE, Wi-Fi, ZigBee, OPC UA, and so on) or data format across different sensors/actuators, DG fills the role of the protocol translator; it communicates on different communication protocols with sensors/actuators on one side and communicates over uniform data or communication protocols with the central server on the other, as shown in the following diagram:

Figure 2.2 – A DG is capable of supporting multiple protocols; a comparison with a smart device

Important Note

DG acts as a connectivity enabler and protocol translator, and provides data-buffering capabilities. However, it may not be needed in scenarios where devices are smart enough to provide these capabilities.

At the top-right of the preceding diagram, the key functionality of the DG pattern is highlighted. As you can see, sensors, actuators, and other devices can interact with DG using a myriad of communication technologies/protocols (both wired and wireless, such as Wi-Fi, ZigBee, BACnet, Modbus, Bluetooth/BLE, RFID, NFC, and more; these are marked as 1 in the diagram). However, the interface between DG and the central server is of a single type (the diagram shows the interface as JSON over HTTPS, though other interfaces, such as AMQP and MQTT, are also used in the IoT ecosystem; they are marked as 2). Another key distinguishing factor between 1 and 2 is that 1 is non-IP-based communication, whereas 2 is IP-based communication.

If the devices are smart (they have an adequate amount of compute, memory, and storage and can establish connectivity with the central server), DG is not required as the devices themselves are capable of connectivity and managing how data is transferred.

In addition to allowing dumb devices to send/receive data to/from the central server and acting as a protocol translator, DG provides additional functionalities as described in the following points:

Data aggregation/filtering: In some scenarios, it is not required to send all the data that’s been captured by the sensors to the central server (due to bandwidth constraints or the application not requiring data to be pushed at a high frequency). In this case, DG will accumulate data and send summarized data to the central server (from the past hour, sending data only if it is different from when it was read prior, and so on).Data security at rest and in motion: DG not only ensures that data that is stored locally is secured (that is, encrypted) but also that the data sent to the central server is secured by leveraging all the best practices related to Authentication, Authorization, and Accounting (AAA).Support local data access requirements: DG allows you to access data locally (to remove dependencies from the central server for critical data access requirements) via APIs. In some scenarios, DG also hosts Human Machine Interfaces (HMIs) for visualization and reporting purposes.LRE Support: DG can enable LRE (the LRE pattern will be covered in Chapter 3, IoT Patterns for the Central Server), where generated events (for example, from sensors) are observed and suitable actions are triggered.Firmware/configuration upgrades of connected devices: Firmware upgrades for connected devices (or for DG itself) are pushed by the central server as needed. Similarly, configuration settings (for example, changes in the data capture frequency) are sent by the central server to DG. Commands may also to sent to DG for troubleshooting/diagnostics purposes.Data buffering: In the case of intermittent connectivity with the central server, DG can buffer data (depending upon the limit imposed by the available local storage) and send it once connectivity has been established, thus avoiding loss of data.Application middleware for DG-hosted applications: DG exposes APIs to report local analytics results (based on historical data), as well as current data that’s been captured by sensors. Also, in certain critical scenarios, commands for actuators may be issued locally rather than you having to wait for the central server to make decisions. Again, this is made possible by the exposed APIs that are leveraged by applications hosted on DG.

Pattern summary

Let's take a look at the pattern summary for a DG:

Problem solved:Business:Interoperability of diverse sensors/actuators that have different connectivity protocols with the central serverEnsuring the security of data at rest as well as during transmissionEnables real-time decision-making at the edge while avoiding a long (and non-deterministic) round trip to the central serverPrevents the loss of data in case of intermittent connectivity with the backend server by buffering data at the edgeProvides support for local/edge applicationsReduces code duplicity as common functionalities (such as connectivity, data buffering, data encryption, and so on) are handled by one entity, such as DG, rather than by all the end devices (that is, sensors and actuators)Facilitates remote firmware upgrades as well as remote configuration updates of end devices, thus providing cost and effort efficienciesEnables connectivity for legacy end devicesEnables efficient utilization of bandwidth by mechanisms such as data aggregation anddata filteringTechnical:Provides separation of concerns as common functionalities (such as connectivity, data buffering, data encryption, and so on) are handled by the DG; the sensors and actuators handle interaction with the physical environmentDiscourages tight coupling between DG and the central server, ensuring parallel development/enhancement of DG and the central serverUsage context:To provide connectivity to constrained/legacy devicesExample/usage scenarios:A smartphone acts as a DG to send data from end devices (for example, smartwatches and home automation controls such as thermostats, blinds, and so on) to the central server for complex analyticsIn the manufacturing or industrial context, DG acts as a connectivity enabler to connect legacy machines to the central server for complex analytics (predictive maintenance, OEE calculations, and more)Pattern rationale:To implement a functionality (such as central server connectivity, data buffering, data security, and so on) that is common for all the end devices, thus eliminating redundancy and complexity of the code/logicTo provide a uniform communication channel or protocol between diverse end devices (that have a variety of communication patterns and protocols) and the central serverTo ease the development of local or edge