35,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
RabbitMQ is Open Source Message Queuing software based on the Advanced Message Queue Protocol Standard written in the Erlang Language. RabbitMQ is an ideal candidate for large-scale projects ranging from e-commerce and finance to Big Data and social networking because of its ease of use and high performance. Managing RabbitMQ in such a dynamic environment can be a challenging task that requires a good understanding not only of how to work properly with the message broker but also of its best practices and pitfalls.
Learning RabbitMQ starts with a concise description of messaging solutions and patterns, then moves on to concrete practical scenarios for publishing and subscribing to the broker along with basic administration. This knowledge is further expanded by exploring how to establish clustering and high availability at the level of the message broker and how to integrate RabbitMQ with a number of technologies such as Spring, and enterprise service bus solutions such as MuleESB and WSO2. We will look at advanced topics such as performance tuning, secure messaging, and the internals of RabbitMQ. Finally we will work through case-studies so that we can see RabbitMQ in action and, if something goes wrong, we'll learn to resolve it in the Troubleshooting section.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 285
Veröffentlichungsjahr: 2015
Ähnliche
Table of Contents
Learning RabbitMQ
Learning RabbitMQ
Copyright © 2015 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: December 2015
Production reference: 1171215
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78398-456-5
www.packtpub.com
Credits
Author
Martin Toshev
Reviewers
Van Thoai Nguyen
Héctor Veiga
Commissioning Editor
Ashwin Nair
Acquisition Editor
Vinay Argekar
Content Development Editor
Kirti Patil
Technical Editor
Danish Shaikh
Copy Editor
Vibha Shukla
Project Coordinator
Nidhi Joshi
Proofreader
Safis Editing
Indexer
Hemangini Bari
Graphics
Disha Haria
Production Coordinator
Arvindkumar Gupta
Cover Work
Arvindkumar Gupta
About the Author
Martin Toshev is a software developer and Java enthusiast with more than eight years of experience and vast expertise originating from projects in areas such as enterprise Java, social networking, source code analysis, Internet of Things, and investment banking in companies such as Cisco and Deutsche Telekom. He is a graduate of computer science from the University of Sofia. He is also a certified Java professional (SCJP6) and a certified IBM cloud computing solution advisor. His areas of interest include a wide range of Java-related technologies (Servlets, JSP, JAXB, JAXP, JMS, JMX, JAX-RS, JAX-WS, Hibernate, Spring Framework, Liferay Portal, and Eclipse RCP), cloud computing technologies, cloud-based software architectures, enterprise application integration, and relational and NoSQL databases. Martin is one of the leaders of the Bulgarian Java Users group (BGJUG), a regular speaker at Java conferences, and one of the organizers behind the jPrime conference in Bulgaria (http://jprime.io/).
About the Reviewers
Van Thoai Nguyen has worked in the software industry for a decade in various domains. In 2012, he joined BuzzNumbers as one of the core senior software engineers, where he had opportunities to design, implement, and apply many cool technologies, tools, and frameworks. A RabbitMQ cluster was employed as the backbone of the real-time data processing platform, which includes various data collectors, data filtering, enrichment, and storage using a sharded cluster of MongoDB and SOLR. He is still maintaining the open source .NET RabbitMQ client library, Burrow.NET (https://github.com/vanthoainguyen/Burrow.NET), which he built during the time he worked for BuzzNumbers. This library is still being used in many different applications in that company. Van is interested in clean code and design, SOLID principle, and BIG data. You can read his blog at http://thoai-nguyen.blogspot.com.au/.
Héctor Veiga is a software engineer specializing in real-time data integration and processing. Recently, he has focused his work on different cloud technologies, such as AWS, to develop scalable, resilient, and high-performing applications with the latest open source technologies, such as Scala, Akka, or Apache Spark. Additionally, he has a strong foundation in messaging systems, such as RabbitMQ and AMQP. He also has a master's degree in telecommunications engineering from the Universidad Politécnica de Madrid and a master's degree in information technology and management from the Illinois Institute of Technology.
He currently works as part of the Connected Driving real-time data collection team and is actively developing scalable applications to ingest and process data from several different sources. He utilizes RabbitMQ heavily to address their messaging requirements. In the past, he worked at Xaptum Technologies, a company dedicated to M2M technologies.
Héctor also helped with the reviewing process of RabbitMQ Cookbook and RabbitMQ Essentials, both from Packt Publishing.
I would like to thank my parents, Pilar and Jose Carlos, as well as my sister, Paula, for always supporting me and motivating me to keep pushing on. Without them, all this would not have been possible.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and readPackt's entire library of books.
Why subscribe?
Free access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.
I would like to thank all of the people that supported me during the process of writing this book and especially my mother Milena, my beloved Tsveti and my grandmother Maria. Without them this would not have been possible.
Preface
Learning RabbitMQ provides you with a practical guide for the notorious message broker and covers the essentials required to start using it. The reader is able to build up knowledge along the way—starting from the very basics (such as what is RabbitMQ and what features does it provide) and reaching the point where more advanced topics, such as RabbitMQ troubleshooting and internals, are discussed. Best practices and important tips are provided in a variety of scenarios; some of them are related to external systems that provide integration with the message broker or that are integrated as part of the message broker in the form of a RabbitMQ plugin. Practical examples are also provided for most of these scenarios that can be applied in a broader context and used as a good starting point.
An example system called CSN (Corporate Social Network) is used to illustrate the various concepts provided throughout the chapters.
Each chapter ends with an Exercises section that allows the reader to test his understanding on the presented topic.
What this book covers
Chapter 1, Introducing RabbitMQ, provides you with a brief recap on enterprise messaging and a short overview of RabbitMQ along with its features. Other similar technologies are mentioned and an installation guide for the message broker is provided at the end of the chapter. The basic terminology behind RabbitMQ such as exchanges, queues, and bindings is introduced.
Chapter 2, Design Patterns with RabbitMQ, discusses what messaging patterns can be implemented using RabbitMQ, including point-to-point, publish-subscribe, request-reply, and message router types of communication. The patterns are implemented using the building blocks provided by the message broker and using the Java client API.
Chapter 3, Administration, Configuration and Management, reveals how to administer and configure RabbitMQ instances, how to install and manage RabbitMQ plugins, and how to use the various utilities provided as part of the RabbitMQ installation in order to accomplish a number of administrative tasks. A brief overview of the RabbitMQ management HTTP API is provided.
Chapter 4, Clustering, discusses what built-in clustering support is provided in the message broker and how it can be used to enable scalability in terms of message queues. A sample RabbitMQ cluster is created in order to demonstrate how nodes can be added/removed from a cluster and how RabbitMQ clients can connect to the cluster.
Chapter 5, High Availability, extends on the concepts of clustering by providing an overview of how a RabbitMQ cluster can be made more reliable in terms of mirrored queues and how messages can be replicated between remote instances using the Federation and Shovel plugins. High availability in terms of client connections and reliable delivery is also discussed with AMQP transactions, publisher confirms, and client reconnections.
Chapter 6, Integrations, provides you with a number of practical scenarios for integration of the message broker with the Spring framework, with ESB (enterprise services bus) systems such as MuleESB and WS02, and with database management systems (RDBMS and NoSQL). Deployment options for RabbitMQ using systems such as Puppet, Docker, and Vagrant are discussed in the chapter. A brief overview of how RabbitMQ applications can be tested using third-party frameworks is provided at the end of the chapter.
Chapter 7, Performance Monitoring and Tuning, gives a detailed list of factors that must be considered in terms of performance tuning of the message broker. The PerfTest tool is used to demonstrate how the RabbitMQ performance can be tested. At the end of the chapter, several monitoring solutions that provide support for RabbitMQ such as Nagios, Munin, and Monit are used to demonstrate how the message broker can be monitored in terms of stability and performance.
Chapter 8, Troubleshooting, illustrates a number of problems that can occur during the startup of the message broker and normal operation along with the various causes and resolutions in such cases. A brief primer on the Erlang programming language is provided for the purpose of understanding and analyzing the RabbitMQ crash dump—either directly or using the Crashdump Viewer for convenience.
Chapter 9, Security, provides a high-level overview of the vulnerability landscape related to the message broker along with a number of techniques to secure a RabbitMQ setup. Authentication, authorization, and secure communication are among the most important concepts covered in the chapter.
Chapter 10, Internals, discusses the internal architecture of the message broker and provides a detailed overview on the most important components that RabbitMQ comprises of.
Appendix A, Contributing to RabbitMQ, provides a short guide on how to get the RabbitMQ sources, how to set up a development environment, and how to build the message broker. A short discussion on how to contribute to the RabbitMQ ecosystem is provided as part of the appendix.
What you need for this book
In order to get the most out of this book, the reader is expected to have at least a basic understanding of what messaging is all about and a good understanding in at least one object-oriented programming language. As the book features the RabbitMQ Java client API in order to demonstrate how to use the message broker, it is good to have at least a basic understanding of the Java programming language. Most of the examples are not specific to a particular operating system; if they are, it is explicitly mentioned whether this is, for example, a Windows- or Unix-based distribution such as Ubuntu. For this reason, there is no particular requirement for an operating system in order to run the examples.
Who this book is for
If you are a developer or system administrator with basic knowledge in messaging who wants to learn RabbitMQ or further enhance your knowledge in working with the message broker, then this book is ideal for you. For a full understanding of some the examples in the book, basic knowledge of the Java programming language is required. Feeling comfortable with RabbitMQ is a great way to leverage your expertise in the world of messaging systems.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, names of third-party applications, utilities, folder names, filenames, file extensions andpathnames are shown in bold as follows: "We already saw how easy it is to start/stop/restart instances using the rabbitmqctl and rabbitmq-server utilities that are part of the standard RabbitMQ installation."
A block of code displayed in a box with console font:
A block of configuration or output is also displayed in a box as follows:
New terms and important words are also shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Clicking the Next button moves you to the next screen."
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
Chapter 1. Introducing RabbitMQ
In the world of enterprise messaging systems there are a number of patterns and practices that are already successfully applied in order improve to scalability and interoperability between different components in a system or between varying in size and complexity systems. RabbitMQ is one such messaging solution, which combines powerful messaging capabilities with easy use and availability on a number of target platforms.
The following topics will be covered in this chapter:
Enterprise messaging
A typical enterprise will have a number of systems that must typically communicate with each other in order to implement a well-defined business process. A question that is frequently tackled for this reason is how to implement the communication channel between these types of systems? For example, consider the following diagram:
The question mark in the preceding picture denotes the communication media for the six systems that are illustrated. In the diagram, we can think of these separate systems as the components of one large system and the problem stays the same. Before discussing the various alternatives for integration, a number of key factors are considered, as follows:
Let's assume that the systems communicate directly via some kind of remote procedure calls as shown in the following diagram:
This implies that separate communication links must be established between each pair of systems, which leads to tight coupling, a lot of effort to maintain all of the links, reduced scalability, and reduced extensibility (for every new system that is added, a few more communication links with other systems must be created). However, real-time communication requirements might be met with some additional effort to design the communication links.
A second approach is to use a shared file system in order to exchange files between the systems that are being integrated, as illustrated in the following diagram:
A shared file system is used to provide the communication medium. Each system may export data to a file that can be imported and used by other systems. The fact that each system may support its own data format leads to the fact that each system must have a particular mechanism to import data from every other system that it needs to communicate with. This, on other hand, leads to the same problems that are described in the case of direct communication. Moreover, real-time communication requirements might be more difficult to establish and reading or writing data from disk is also an expensive operation.
A third option is to use a shared database as shown in the following diagram:
Here, all the systems should depend on the same database schema. Although this reduces coupling between systems and improves extensibility (new systems must conform to a single database schema in order to integrate with other systems), real-time workload processing is still an issue. Scalability and maintainability depend directly on the type of database that is being used and they could turn out to be weak factors especially if it is a relational database (this may not be the case if NoSQL solutions are applicable for the particular use case).
Messaging comes to the rescue when considering the problems that arise when applying the previous approaches. Consider the following diagram for the Enterprise Messaging System:
A message is the central unit of communication used in enterprise messaging systems. A message typically consists of the following:
The messaging system itself provides mechanisms to validate, store, route, and transform messages that are being sent between the different systems. Each system is responsible for crafting its own message that is transferred via the messaging system (also called the messaging broker) to other systems that are connected to the broker and configured to receive that particular type of message. Each system may create a message in a proper format that is specified by the protocol of the message broker—meaning that the system is only coupled with that particular protocol. If the broker implements a protocol that is based on a well-recognized standard, then this would further decouple the systems from that particular message broker implementation.
Real-time workload processing is typically quite fast as the particular protocol that is implemented by different messaging brokers is optimized to process message data in a reliable and secure manner with minimal overhead. Most messaging solutions provide a number of facilities that allow easy configuration, management, and monitoring; thus, simplifying maintainability. Clustering support is also considered by most implementations due to scalability and reliability requirements and increasing workload demands. Integrating new systems is a matter of implementing a mechanism for direct communication with the message broker.
In case the different systems provide different implementations of messaging protocols (such as REST, SOAP, JSON-RPC, JMX, AMQP, and many others), a messaging system could further provide various adapters for the different protocols as well as extended mechanisms for routing and transformation of different types of messages—this extended functionality also categorizes the message brokers as Enterprise Service Bus (ESB) solutions. One major drawback of an ESB is that it must implement all the communication requirements of all systems that are being integrated by the ESB, otherwise workarounds must be used in order to implement direct communication between the integrated systems (thus, neglecting the usage of an ESB).
Use cases
There are a variety of scenarios where messaging systems may be applied, such as the following:
As you can see, messaging solutions can be applied to a variety of scenarios that typically involve a number of systems that must communicate in a timely manner or perform a large number of time-consuming tasks. Messaging solutions are also extensively being deployed by Cloud providers in order to provide messaging as a service for Cloud-based applications.
Solutions
A wide variety of open source and property messaging solutions are available for use, which are based on the multitude of use cases. The choice of a messaging broker depends on a number of factors, as shown in the following:
Patterns
A messaging system provides patterns for communication between system endpoints.
Point-to-point
In a point-to-point communication model, there is exactly one sender and one receiver of a message. In case there are multiple senders that are applicable for the purpose of receiving the message, only one of them succeeds. Such receivers are also referred to as competingconsumers, indicating that any of them are eligible to receive the message. The sender does not receive a response in a point-to-point model.
Publish-subscribe
In a Publish-subscribe communication model, there is one sender and multiple receivers (subscribers) of the message. It is a form of fire-and-forget, where the sender does not await for a response once the message is sent to the broker.
Request-response
In a request-response communication model, there is one sender and one receiver that sends a response to the sender of the message.
Understanding RabbitMQ
The RabbitMQ messaging server is an open source implementation of the AMQP 0-9-1 protocol (Advanced Message Queuing Protocol) that defines how messages should be queued, routed, and delivered in a reliable and secure manner. AMQP 1.0, which is an OASIS (Organization for the Advancement of Structured Information Standards), is not directly supported in the message broker; however, RabbitMQ provides a plugin for AMQP 1.0 (as it is not backward-compatible with AMQP 0-9-1). OASIS is a non-profit organization that works for the development of a number of technology standards. As an open standard, AMQP promotes interoperability among the messaging brokers that implement the protocol. It also defines the delivery semantics for a message, which dictates how many times that message can be sent from one endpoint to another—zero or once, exactly once or multiple times. As a wire protocol, AMQP provides better performance in regard to other messaging protocols such as XMPP (Extensible Messaging and Presence Protocol).
Before we discuss more about RabbitMQ as a message broker, we will introduce some terminologies from the RabbitMQ world that we will use frequently throughout the book:
The AMQP protocol allows a client to establish a one-way logical link to send messages for exchange. Each logical link is a separate AMQP channel that may provide additional options for the reliable transfer of messages. In this regard, a single-client TCP connection to the RabbitMQ server allows multiple AMQP channels of communication. Since AMQP does not provide the capability to retrieve the list of queues, exchanges, or bindings that are defined in the RabbitMQ message broker, client applications must specify the exchange name, queue names, and, optionally, routing information by means of routing keys for particular bindings. AMQP is a programmatic protocol that allows its clients to create and delete exchanges, queues, and bindings when necessary. RabbitMQ addresses some limitations of AMQP by providing custom extensions apart from the fact that the AMQP protocol itself is extensible. In order to simply application development, RabbitMQ provides several exchange types out of the box, as follows:
Receivers can either subscribe to a queue in order to receive messages (also called push-style communication) or request messages on demand from a queue (also called pull-style communication).
Features
The RabbitMQ message broker provides a number of features and tools that support production-level deployment, management, and configuration of the RabbitMQ server instances as shown in the following:
All of these features will be covered in detail in the next chapters.
Comparison with other technologies
As
