40,81 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
A comprehensive guide to help you understand the principles of Reactive and asynchronous programming and its benefits
Key Features
- Explore the advantages of Reactive programming
- Use concurrency and parallelism in RxPY to build powerful reactive applications
- Deploy and scale your reactive applications using Docker
Book Description
Reactive programming is central to many concurrent systems, but it's famous for its steep learning curve, which makes most developers feel like they're hitting a wall. With this book, you will get to grips with reactive programming by steadily exploring various concepts
This hands-on guide gets you started with Reactive Programming (RP) in Python. You will learn abouta the principles and benefits of using RP, which can be leveraged to build powerful concurrent applications. As you progress through the chapters, you will be introduced to the paradigm of Functional and Reactive Programming (FaRP), observables and observers, and concurrency and parallelism. The book will then take you through the implementation of an audio transcoding server and introduce you to a library that helps in the writing of FaRP code. You will understand how to use third-party services and dynamically reconfigure an application.
By the end of the book, you will also have learned how to deploy and scale your applications with Docker and Traefik and explore the significant potential behind the reactive streams concept, and you'll have got to grips with a comprehensive set of best practices.
What you will learn
- Structure Python code for better readability, testing, and performance
- Explore the world of event-based programming
- Grasp the use of the most common operators in Rx
- Understand reactive extensions beyond simple examples
- Master the art of writing reusable components
- Deploy an application on a cloud platform with Docker and Traefik
Who this book is for
If you are a Python developer who wants to learn Reactive programming to build powerful concurrent and asynchronous applications, this book is for you. Basic understanding of the Python language is all you need to understand the concepts covered in this book.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 466
Veröffentlichungsjahr: 2018
Ähnliche
Hands-On Reactive Programming with Python
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Richa TripathiAcquisition Editor: Shriram ShekharContent Development Editor: Zeeyan PinheiroTechnical Editor: Romy DiasCopy Editor: Safis EditingProject Coordinator: Vaidehi SawantProofreader: Safis EditingIndexer: Rekha NairGraphics: Alishon MendonsaProduction Coordinator: Deepika Naik
First published: October 2018
Production reference: 1241018
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78913-872-6
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Romain Picard is currently a data science engineer. He has been working in the digital TV and telecommunications industry for 20 years. His daily work consists of data manipulation, machine learning model training, and model deployment. Almost all of these tasks are based on Python code, and he uses reactive programming whenever it's applicable. He was previously a media software architect and a software developer. In these previous positions, he designed and developed TV and OTT players that have been used in tens of millions of set top boxes. Romain is especially interested in algorithms, looking for the most suitable algorithm for each use case.
About the reviewer
Ricardo Bánffy is a hardware engineer by training and software engineer at heart. He has been writing executable poetry since the late 1980s in all types of languages for all kinds of computers, from GUI tools for Apple II+ computers and business software for mainframes to hardware provisioning and monitoring tools for massive Linux clusters. A passionate Python advocate since the late 1990s, he is often busy attending or speaking at conferences and local meetups. His most recent work involves orchestration tools for transforming large datasets using Python and lots of clever tricks.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Hands-On Reactive Programming with Python
Dedication
Packt Upsell
Why subscribe?
Packt.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Get in touch
Reviews
An Introduction to Reactive Programming
What is reactive programming?
Event-driven programming
Reactive versus proactive
Reactor and proactor
Reactive systems
Introduction to ReactiveX and RxPY
ReactiveX principles
Operators
Installating RxPY
A reactive echo application
Marble diagrams
The map operator
The from_ operator
Flow diagrams
Reactivity diagrams
Reactivity diagram elements
Reactivity diagrams of an echo example
Summary
Questions
Further reading
Asynchronous Programming in Python
What is asynchronous programming?
The history of asynchronous programming in Python
Asynchronous handlers with callbacks
Asynchronous handlers with generators
Understanding generators
Using generators for asynchronous handlers
Introduction to AsyncIO
Futures
Coroutines
Event loop
An HTTP echo server
aiohttp
An AsyncIO HTTP server
Summary
Questions
Further reading
Functional Programming with ReactiveX
What is functional programming?
Some base elements of functional programming
Lambdas
Closures
Side effects and pure functions
Higher-order functions
Functional reactive programming
Observable cycles
The observable cycle issue
ReactiveX Subject
Solving the cycle issue with Subject
Structuring functional and reactive code
The HTTP echo server with an HTTP driver
ReactiveX operators
The empty operator
The merge operator
Implementation of the HTTP server driver
Bootstrapping the event loop
Adding the program logic
Summary
Questions
Further reading
Exploring Observables and Observers
Creating observables
Creating observables from values
The of operator
The just operator
The range operator
The repeat operator
The never and throw operators
Observables driven by time
The timer operator
The interval operator
Custom observables
The from_callback operator
The create operator
Hot and cold observables
Operators – publish and connect
Operators – ref_count and share
Subscription and disposal
Subscribing to an observable
Disposing a subscription
Custom disposal
Error handling
The catch_exception operator
The retry operator
Observables and AsyncIO
The start operator
The from_future operator
Summary
Questions
Concurrency and Parallelism in RxPY
Concurrency and schedulers
Available schedulers
The NewThread scheduler
The ThreadPool scheduler
The AsyncIO scheduler
Summary
Questions
Implementation of an Audio Transcoding Server
Technical requirements
Structuring the project
The base structure
Maximizing code readability
Using named tuples
Using dataclasses
Avoiding backslashes
Introduction to Cyclotron
The command-line echo example
Operators used in this application
The skip operator
The filter operator
The flat_map operator
The let operator
Implementation of the encoding server
Parsing the configuration file
Implementation of the encoder driver
Using the encoder driver
Exposing the REST APIs
Putting it all together
Using the encoding server
Summary
Questions
Further reading
Using Third-Party Services
Technical requirements
An introduction to S3 storage
Installing Docker and Minio
An introduction to Docker
Docker installation on Linux
Getting started with Docker
Using Minio as S3 storage
Using S3 as storage
Implementation of the S3 driver
Returning data in the encoding driver
Using the S3 driver
Dealing with blocking API and CPU-bound operations
Using thread pools for CPU-bound operations
Moving the blocking I/O to a dedicated thread
Summary
Questions
Further reading
Dynamic Reconfiguration and Error Management
Monitoring file changes
Understanding inotify
Monitoring file changes with inotify
Operators used for dynamic configuration
The debounce operator
The take operator
The distinct_until_changed operator
The start_with operator
The combine_latest operator
Encode driver reconfiguration
Configuring the audio encoding
Monitoring the configuration file
Implementing an inotify driver
Monitoring changes in the configuration file
Error management
Summary
Questions
Operators in RxPY
Transforming observables
The buffer operator
The window operator
The group_by operator
Filtering observables
The first operator
The last operator
The skip_last operator
The take_last operator
The ignore_elements operator
The sample operator
The distinct operator
The element_at operator
Combining observables
The join operator
The switch_latest operator
The zip operator
The zip_list operator
Utility operators
The delay operator
The do_action operator
The materialize/dematerialize operators
The time_interval operator
The timeout operator
The timestamp operator
The using operator
The to_list operator
Conditional operators
The all operator
The amb operator
The contains operator
The default_if_empty operator
The sequence_equal operator
The skip_until operator
The skip_while operator
The take_until operator
The take_while operator
Mathematical operators
The average operator
The concat operator
The count operator
The max operator
The min operator
The reduce operator
The sum operator
Summary
Questions
Further reading
Testing and Debugging
Testing
Introduction to Python unit testing
Dependency injection versus mock
Testing a custom operator
Injecting asynchronous obervables
Testing AsyncIO code
Logging
Debugging
Adding traces
Debugging AsyncIO
Summary
Questions
Further reading
Deploying and Scaling Your Application
Technical requirements
Introduction to Traefik
Reverse proxies and load balancers
What is a reverse proxy?
What is a load balancer?
Traefik principles
Packaging a service with Docker Compose
Introduction to Docker Compose
Installing Docker Compose
Starting Traefik
Packaging the audio transcoder
How images are composed together
Composing all services
Using containers for horizontal scaling
Adding TLS support with Let's Encrypt
Introduction to Let's Encrypt
Deploying on a public server
Enabling the ACME protocol
Summary
Questions
Further reading
Reactive Streams for Remote Communication
Communication patterns and observables
Publish/subscribe
Channel
Request/response
Observable multiplexing
Framing
An implementation of line-based framing
Serialization
An implementation of JSON serialization
Routing
Implementing publish/subscribe
The publisher
Implementation of a TCP server
Implementation of the publisher
The subscriber
Implementation of a TCP client
Implementation of the subscriber
Summary
Questions
Further reading
A Checklist of Best Practices
The observable creation decision tree
Keeping side-effects as small as possible
What to do when nothing happens
Miscellaneous recommendations
Favor composition of existing operators
Always specify a scheduler
Subscriptions should not throw exceptions
Disposal should not throw exceptions
Be clear on hot and cold observables
Summary
Questions
Further reading
Assessments
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Welcome to Hands-On Reactive Programming with Python! This book is a step-by-step journey into event-driven programming and, more specifically, reactive programming. The resources available on this topic are still rather sparse, especially for Python. I hope that this book will help to fill a part of that gap, and that it will give you the knowledge and will to write code in a functional and reactive way.
Magic is rarely something good in computer science. In my career, I have met quite a number of developers who were happy when a development started to work magically or when a bug was fixed magically (usually with some help of a delay). Any programmer can be lazy, but certainly not in this way! Fortunately, I've also had the chance to meet a lot of developers with the good laziness skill: understanding everything you do to ensure that you will not have to do it twice. I firmly believe that to use any framework correctly, you have to understand how it works under the hood. This is why this book is structured this way, with explanations of the principles that make reactive and asynchronous programming work the way they do.
Writing this book has been incredibly rewarding, and worth the effort. As I write this preface, there is no online documentation for the Python implementation of ReactiveX, and it is still difficult to find code examples of the operators for the other programming languages' implementations. This is why I have taken care to provide working examples for all operators that are documented in this book. I have carefully read the implementation of each of them to ensure that they are documented exactly as they behave.
However, documenting the RxPY operators is not the main aim of this book. The practical examples provided chapter after chapter should gradually help you to become confident with ReactiveX and asynchronous programming, including rather advanced topics such as observable multiplexing.
I have worked hard to make this book as didactic as possible and as accurate as possible. However, if you find any mistakes or if you feel that some parts are missing explanations, feel free to tell me about it.
Who this book is for
This book is intended for software developers who are already at ease with Python and have heard about reactive programming but have not had a chance to work with it yet. However, no expertise is needed beyond that. Each chapter of the book describes new notions step by step so that the reader can gradually gain the necessary knowledge to understand the chapters that follow.
Hopefully, this will not be a read once and put on the shelf book. The operators documentation and descriptions of ways to tackle common problems should make this book regularly handy, even after being fully read.
What this book covers
Chapter 1, An Introduction to Reactive Programming, gives an overview of event-driven programming and reactive programming.
Chapter 2, Asynchronous Programming in Python, presents the foundations of asynchronous programming and the different ways to deal with input/output. It provides an initial introduction to AsyncIO.
Chapter 3, Functional Programming with ReactiveX, provides insights on how to structure the code of an application with ReactiveX, and how to use functional programming in particular.
Chapter 4, Exploring Observables and Observers, goes into the details of observables and observers, and all the possible ways to create observables.
Chapter 5, Concurrency and Parallelism in RxPY, explains how RxPY manages CPU concurrency via schedulers.
Chapter 6, Implementation of an Audio Transcoding Server, goes through the initial implementation of a realistic reactive and functional application.
Chapter 7, Using Third-Party Services, is a continuation of the previous chapter, explaining how to integrate an existing SDK with AsyncIO and RxPY. It also introduces Docker, a container-management platform.
Chapter 8, Dynamic Reconfiguration and Error Management, goes into functionalities and robustness in more depth, explaining how ReactiveX makes it simple to manage dynamic changes.
Chapter 9, Operators in RxPY, runs through a detailed documentation of 40 widely used operators that have not been covered in the book so far.
Chapter 10, Testing and Debugging, explains how to test reactive code, and how to debug it.
Chapter 11, Deploying and Scaling Your Application, shows how to deploy a Python application on the cloud and how to scale it, thanks to Docker, Docker Compose, and Traefik.
Chapter 12, Reactive Streams for Remote Communication, covers a rather advanced topic. It paves the way to reactive systems, and the global use of observables to communicate between applications.
Chapter 13, A Checklist On Best Practices, contains some final notes on things that are easy to forget when starting to code reactive applications.
To get the most out of this book
This book is about Python programming, but it is not an introduction to Python programming. It supposes that the reader is already familiar with the Python programming language. Moreover, asynchronous programming is not easy to comprehend at first. For this, some knowledge on computer science can help but is not mandatory. This book contains a lot of code samples and diagrams. In order to test these code samples, a computer with the following software is needed:
A personal computer with either a Linux distribution, macOS X, or Windows
Python already installed, at least Python 3.6
A code editor or an IDE to view and modify the code provided
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Hands-On-Reactive-Programming-with-Python. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "In the RxPY implementation, operators are methods of the Observable class."
A block of code is set as follows:
Observable.from_(...) .filter() .distinct() .take(20) .map(...)
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
argv.subscribe( on_next=lambda i: print("on_next: {}".format(i)), on_error=
lambda
e: print("on_error: {}".format(e)), on_completed=lambda: print("on_completed"))
Any command-line input or output is written as follows:
$ source venv-rx/bin/activate
(venv-rx)$ pip install rx
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "By definition, any program has to deal with external events through inputs/outputs (I/O)."
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
An Introduction to Reactive Programming
This first chapter covers the principles of reactive programming and ReactiveX. It is composed of three parts. The first part explains what reactive programming is and how it compares to other concepts and paradigms that are often used in event-driven programming. The second part explains the foundations of ReactiveX and RxPY, its Python implementation. This exploration of RxPY is explained with a simple example that allows us to understand the basics of ReactiveX. Finally, the last part is dedicated to the documentation of the ReactiveX project and the documentation of your projects. By the end of this chapter, you will be able to write some ReactiveX code and understand existing code.
The following topics will be covered in this chapter:
What is reactive programming?
An introduction to ReactiveX and RxPY
A reactive echo application
Marble diagrams
Flow diagrams
What is reactive programming?
Reactive programming has gained a lot of popularity since 2010. Although its concepts and usage date from the early days of computing, this recent popularity is mainly due to the publication of the ReactiveX project. This might seem surprising for developers who had rarely used event-driven programming before. However, for people who faced tremendous state machines or callback hell, this seems more of an inevitable fact.
Before playing with ReactiveX and RxPY, this first part describes the principles being used in reactive programming and how they are used in asynchronous frameworks. This initial study of low-level features is not strictly necessary when using ReactiveX and asynchronous frameworks, but it helps a lot to understand how they work, which thus helps us to use them correctly.
Event-driven programming
What is the common connection between state machines, Petri net, Kahn Process Networks (KPN), the observer design pattern, callbacks, pipes, publish/subscribe, futures, promises and streams? Event-driven programming!
By definition, any program has to deal with external events through inputs/outputs (I/O). I/O and event management are the foundations of any computer system: reading or writing from storage, handling touch events, drawing on a screen, sending or receiving information on a network link, and so on. Nothing useful can be done without interacting with I/O, and I/O are almost always managed through events. However, 50 years after the creation of the first microprocessor, event-driven programming is still a very active topic with new technologies appearing almost every year.
The main purpose of this important activity is that, despite the fact that event-driven programming has existed since the beginning of computer science, it is still hard to use correctly. More than writing event-driven code, the real challenge lies in writing readable, maintainable, reusable, and testable code. Event-driven programming is more difficult to implement and read than sequential programming because it often means writing code that is not natural to read for human beings—instead of a sequence of actions that execute one after the other until the task to execute is completed, the beginning of an action starts when an event occurs, and then the actions that are triggered are often dispersed within the program. When such a code flow becomes complex—and this starts only after few indirect paths in a code—then it becomes more and more difficult to understand what is happening. This is what is often called the callback hell. One has to follow callbacks calling callbacks, which call further callbacks, and so on.
During the late nineties, event-driven programming became quite popular with the advent of graphical user interfaces (GUIs). At that time, developers had the following options to write GUI applications:
Objective-C on NextStep and macOS
C++ on Windows
C or eventually C++ on Unix (with X11)
Java, with the hope of writing the same application for all these systems
All these environments were based on callbacks and it stayed that way for a very long time, until programming languages included features to improve the readability of event-driven code.
So event-driven code is often more difficult to read than sequential code because the code logic can be difficult to follow, depending on the programming language and the frameworks being used. Reactive programming, and more specifically ReactiveX, aims at solving some of these challenges. Python and its relatively recent support of async/await syntax also aims to make event-driven programming easier.
It is important to understand that reactive programming and event-driven programming are not programming paradigms, such as imperative or object-oriented programming, but are orthogonal to them. Event-driven programming is implemented within an existing paradigm. So, one can use event-driven programming with an object-oriented language or a functional language. Reactive programming is a specific case of event-driven programming. This can be seen in the following figure:
Reactive versus proactive
So what is reactive programming?
An easy way to get the idea behind it is to use an analogy with people's behavior: someone who is proactive is somebody who takes initiatives. A proactive person will propose ideas or test things before somebody asks him to do so. On the other hand, a reactive person is somebody who waits for information before doing something. So, a proactive person acts on his own while a reactive person reacts to external changes. There are pros and cons to each behavior: a proactive person proposing solutions ahead of time is great, but may have difficulties dealing with unexpected changes. On the other hand, a reactive person may be very efficient in dealing with very dynamic environments.
Reactive programming can be considered as implementing a behavior in a similar way to a reactive person. A reactive system reacts on external events and provides a result that depends only on the event it has received. So why would reactive programming be better than sequential programming? Better is always a matter of preferences and context, so reactive programming may not be the solution most adapted to all the problems you will encounter. However, as you will see through this book, reactive programming shines at implementing event-driven code.
Reactive programming is inherently asynchronous. So it makes it easier to deal efficiently with inputs and outputs than with synchronous paradigms. Reactive programming favors composition. Each component is completely independent from another and can be plugged in with other components. This also makes testing quite easy and, as a consequence, it also helps to refactor existing code. Moreover, it is quite engaging, and with experience you will see that almost everything is a flow of events.
Reactor and proactor
When looking for information on event-driven programming concepts, two other similar terms are often mentioned: the reactor and the proactor. These are notions that are not really important when using high-level libraries such as ReactiveX and AsyncIO. Still, it is interesting to know what they are so that you can better understand what is going on under the hood. They are two kinds of low-level APIs that allow us to implement an event-driven library. For example, AsyncIO, which is the Python asynchronous API which can use a reactor or a proactor to expose the same APIs. Using a reactor or a proactor as the foundation of a framework is driven by the support, or not, of the proactor on the operating system. All operating systems support a reactor via the POSIX select system call or one of its derivatives. Some operating systems such as Windows implement proactor system calls. The difference between these two design patterns is the way I/O are managed.
Figure 1.2 shows a sequence diagram of how a reactor works. The three main components involved in this pattern are as follows:
Reactor
Event handler
Event demultiplexer
When the Main Program needs to execute an asynchronous operation, it starts by telling it to the Reactor, with the identification of the Concrete Event Handler that will be notified when an event occurs. This is the call to the register_handler. Then the Reactor creates a new instance of this Concrete Event Handler and adds it to its handler list. After that, the Main Program calls handle_events, which is a function that blocks until an event is received. The Reactor then calls the Event Demultiplexer to wait until an event happens. The Event Demultiplexer is usually implemented through the select system call or one of its alternatives, such as poll, epoll, or kqueue. select is called with the list of handles to monitor. These handles are associated with the handlers that were registered before. When an event that corresponds to one of the handles occurs, the Event Demultiplexer returns the list of handles that correspond to the event that happened. The Reactor then calls the associated event handlers, and the event handlers implement the actual service logic. The following diagram demonstrates this:
Figure 1.3 shows a sequence diagram of how a Proactor
Reactive systems
This book will cover many aspects on reactive programming. But an important thing to be aware of is that using reactive programming does not mean implementing a reactive system. A reactive system is much more than implementing a component with asynchronous and reactive programming. These are two notions that may be easily mingled due to the similarities in the way they are named, but they are completely different. As already explained, reactive programming is a way to code. On the other hand, a reactive system is an architecture pattern that allows us to write robust systems; that is, applications that are made of many components communicating via network channels, and with instances running on several (many?) servers, virtual machines, or containers. This architecture pattern is described in the reactive manifesto (https://www.reactivemanifesto.org/).
The four pillars of the reactive system are shown in the following figure:
These four pillars are interdependent. The value of a reactive system is being reactive thanks to an elastic and resilient design. A reactive system relies on a message-driven communication between the components of the system. More specifically:
A reactive system is responsive
: It responds to events and user interactions rapidly and consistently. Responsiveness ensures that the application stays usable, and that, in case of a problem, these problems can be detected very quickly and thus handled correctly. Responsiveness is achieved thanks to the three other pillars of a reactive system.
A reactive system is resilient
: The system stays responsive even in the event of failure. Resilience can be achieved in several ways, such as replication and isolation. Failures are handled and contained in each component. Other components are dedicated to recover the components that failed and replication allows it to provide high-availability.
A reactive system is elastic
: The system stays responsive when the workload varies. The system can adapt to workloads that increase or decrease so that the allocated resources of the system are not oversized or undersized. In order to provide elasticity, the design must be vertically and horizontally scalable, with no performance bottleneck.
A reactive system is message-driven
: The different components of the system communicate via asynchronous message channels. Communication via messages allows us to isolate components. Saturation is controlled via back-pressure.
As you can see, reactive programming does not provide these four pillars in itself. Reactive programming is one of the tools that can be used to implement a reactive system, but it is not sufficient. Many other tools, such as message brokers, containers, orchestrations, or monitoring tools are needed to build a reactive system.
Introduction to ReactiveX and RxPY
ReactiveX is a library which aims to make asynchronous programming easy. As the header of the project's website says, it is: The Observer pattern done right. ReactiveX is a library based on the idea of observable streams. A stream is an entity that emits zero, one, or several items, over a period of time. This stream of items can be observed by other entities that are interested in receiving these items and manipulated by them. This simple idea is the basis of what has become an incredibly successful way of doing asynchronous programming.
As said in the very first paragraph of this book, asynchronous programming is a very active field. ReactiveX is a typical example of technologies that did not exist a few years ago but that are now heavily used. It was originally one of the components of the Volta project at Microsoft. This project consisted of a set of developer tools to help with developing client and server parts of web applications. The Volta project was suspended in 2008 but ReactiveX continued to be developed, up to the point when it was publicly released for the .NET platform in 2010. The library was very successful, with a community starting to grow up and big companies such as Netflix and GitHub using it. In 2012, implementations for .NET, JavaScript, and C++ were published as open source projects. Since that time, ReactiveX has even impacted the standardization of some programming languages. ReactiveX now has official implementations for almost 20 programming languages and was the foundation of the Java reactive streams (http://www.reactive-streams.org) standard and the EcmaScript observable (https://github.com/tc39/proposal-observable) API. Nowadays, many other libraries, heavily inspired by ReactiveX, are available for virtually any programming language.
All of this is based on concepts that have already existed for many years, such as the observer design pattern, the iterator design pattern, and some principles from functional programming. The ingenuity came from combining them in such a way that it avoids the callback hell. Even better, it is equally suited for frontend applications that deal with user events and GUI widgets, and backend applications that work with network and database requests.
ReactiveX principles
ReactiveX is based on two entities: observables and observers. These are the only things that one needs to understand to be able to start writing code. Everything else is based on the behavior of one of these two entities.
Observables represent a source of events. An observable is an entity that can emit zero or one of several items. An observable has an explicit lifetime with a start and an end. When an observable completes or faces an error, it cannot send items anymore; its lifetime has ended. An observable may never end. In this case, it is an infinite source of events. Observables are a way to manage sequences of items in an asynchronous way. Table 1.1, which follows, shows a comparison between how to access items in a synchronous or asynchronous way. As you can see, observables fill a gap and are allowed to operate on multiple items in an asynchronous way.
Single item
Multiple items
Synchronous
Getter
Iterable
Asynchronous
Future
Observable
Observables work in push mode, as opposed to the pull mode of an iterable. Each time a new item is available, the observable pushes it to its observer. Table 1.2 shows the difference between the pull mode of an iterator and the push mode of an observable. This is what makes the behavior reactive and easy to handle with asynchronous code: whether items are emitted immediately or later is not important to the observer receiving it, and the code semantic is very similar to the one used in synchronous code:
Event
Iterable (pull)
Observable (push)
Retrieve data
For i in collection
on_next
Error
Exception is raised
on_error
Complete
End of loop
on_completed
Observers are the receiving part of the items. An observer subscribes to an observable so that it can receive items emitted by this observable. Just as the observable emits items one after another, an observer receives them one after another. The observable informs the observer of the end of the sequence, either by indicating that the observable has completed (successfully) or by indicating that an error has occurred. These two kinds of completion are notified in a similar way, and so can be handled in a similar way. With ReactiveX, the error management is not a special case, but on a par with the items and completion management. In contrast to iterables that use exceptions, there is no radically different way of handling success from failure.
The implementation of RxPY (as well as all other implementations of ReactiveX) involves two other entities: a subscription function and a disposable object. Figure 1.5 shows a simplified representation of these entities. The AnonymousObservable class is the class that is almost always used to create an observable (directly or via another subclass). This class contains two methods to manage the lifetime of the observable and its observer. The first one, init, is not even a method but the constructor of the class. It takes a subscription function as an input argument. This subscription function will be called when an observer subscribes to this observable. The observable constructor returns a disposable function that can be called to free all resources used by the observable and observer. The second method of the AnonymousObservable class is subscribe. This is the method that is used to attach an observer to an observable and start the observable; that is, to make it start emitting items. The AnonymousObservable class can be used directly, but there are many cases where using an existing RxPY AnonymousObservable subclass is easier. This is typically the case when you need to create an observable from an iterable object or a single object.
The Observer class is a base class that contains three methods. They correspond to the behavior explained previously. This class must be subclassed to implement these three methods. The method on_next is called each time an item is emitted by the observable. The method on_completed is called when the observable completes successfully. Finally, the method on_error is called when the observable completes because of an error. The on_item method will never be called after the on_completed or the on_error methods.
The subscription entity is a function that takes an observer as input parameter. This function is called when the subscribe method of the observable is called. This is where the emission of items is implemented. The emission of these items can be either synchronous or asynchronous. Items are emitted in a synchronous way if the subscription function directly calls the on_items methods of the observable. But items can also be emitted asynchronously if the observer instance is saved and used later (after the subscription function returns). The subscription function can return a Disposable object or function. This Disposable object will be called when the observable is being disposed.
Finally, the Disposable class and its associated dispose function are used to clean up any resources used by an Observer or a subscription. In the case of an asynchronous observable, this is how the subscription function is notified that it must stop emitting items, because Observer is no more valid after that. The following figure shows these components:
Let's try to make more sense of all these definitions. The following figure shows a sequence diagram of how these calls are organized when an observable is created, subscribed, and finally disposed:
First the application creates an AnonymousObservable and provides the subscription function associated to this observable. It then creates an observer object (actually a subclass of Observer). After that, the observable is subscribed, with reference to the observer object provided as an input parameter. During the call to subscribe, the subscription function is called. In this example, the subscription function is synchronous: it emits three items (the integers 1, 2, and 3), and completes the observable. When the subscription function returns, the observable is already completed. At that point, the subscribe method of AnonymousObservable returns the Disposable function to the application. The application finally calls this dispose function to clean up any resource still used by subscription and observer.
