47,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Master the art of developing message-based applications with RabbitMQ
About This Book
- Learn how to administer, manage, and extend your own message broker, RabbitMQ
- Develop clients to make a message bridge between your software systems using RabbitMQ
- Discover how to achieve proficiency with RabbitMQ with the well-defined descriptions of the topics
Who This Book Is For
If you are an intermediate-level RabbitMQ developer, who wants to achieve professional-level expertise in the subject, this book is for you. You'll also need to have a decent understanding of message queuing.
What You Will Learn
- Administer RabbitMQ using different tools
- Understand the roots and details of messaging, message brokers, and AMQP protocol
- Scale the RabbitMQ server using the clusters and high availability techniques
- Extend RabbitMQ by developing the Erlang OTP-based applications that use the RabbitMQ API
- Manage the RabbitMQ server using its powerful tools
- Monitor the RabbitMQ Server using different open source tools such as Nagios, Munin, and Zabbix
- Ensure your RabbitMQ's security using SSL, SASL, and access control
- Develop RabbitMQ clients using Java, Python, and C# with an industry example
In Detail
RabbitMQ is one of the most powerful Open Source message broker software, which is widely used in tech companies such as Mozilla, VMware, Google, AT&T, and so on. RabbitMQ gives you lots of fantastic and easy-to-manage functionalities to control and manage the messaging facility with lots of community support. As scalability is one of our major modern problems, messaging with RabbitMQ is the main part of the solution to this problem.
This book explains and demonstrates the RabbitMQ server in a detailed way. It provides you with lots of real-world examples and advanced solutions to tackle the scalability issues.
You'll begin your journey with the installation and configuration of the RabbitMQ server, while also being given specific details pertaining to the subject. Next, you'll study the major problems that our server faces, including scalability and high availability, and try to get the solutions for both of these issues by using the RabbitMQ mechanisms. Following on from this, you'll get to design and develop your own plugins using the Erlang language and RabbitMQ's internal API. This knowledge will help you to start with the management and monitoring of the messages, tools, and applications. You'll also gain an understanding of the security and integrity of the messaging facilities that RabbitMQ provides. In the last few chapters, you will build and keep track of your clients (senders and receivers) using Java, Python, and C#.
Style and approach
An easy-to-follow guide, full of hands-on examples based around managing, monitoring, extending, and securing RabbitMQ and its internal tools. You will learn how to develop your own clients using Java, Python, and C#.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 262
Veröffentlichungsjahr: 2016
Ähnliche
Table of Contents
Mastering RabbitMQ
Mastering RabbitMQ
Copyright © 2015 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: December 2015
Production reference: 1211215
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78398-152-6
www.packtpub.com
Credits
Authors
Emrah Ayanoglu
Yusuf Aytaş
Dotan Nahum
Reviewers
Steve Fosdal
Van Thoai Nguyen
Jorge Puente-Sarrín
Ken Taylor
Héctor Veiga
Commissioning Editor
Ashwin Nair
Acquisition Editor
Reshma Raman
Content Development Editor
Anish Sukumaran
Technical Editor
Abhishek R. Kotian
Copy Editor
Pranjali Chury
Project Coordinator
Mary Alex
Proofreader
Safis Editing
Indexer
Monica Ajmera Mehta
Graphics
Disha Haria
Production Coordinator
Conidon Miranda
Cover Work
Conidon Miranda
About the Authors
Emrah Ayanoglu has been into technology since a young age, when he was programming with his Tandy 1000 using Basic language. His deep interest and passion for programming lead to him pursue computer engineering at Bilkent University, Ankara. He now works as a software engineer and heavily works on integrating software systems using RabbitMQ.
He frequently speaks at different tech conferences about scalability and real-time web applications where RabbitMQ has a major role. Additionally, he participates in different open source projects.
For the future, he hopes to participate more in open source projects and work on the real-time scalable applications.
Yusuf Aytaş is a software engineer. He completed his B.S. and M.S. in computer science from Bilkent University, Ankara. He has worked in both early stage startups and multinational companies. He is proficient in agile methodologies, continuous delivery, and software development best practices.
About the Reviewers
Steve Fosdal has been writing software for over 10 years. He joined Slalom Consulting as a Solution Architect in late 2015.
His work has included building scalable, distributed applications for traffic prediction and real-time data integration using Akka, Scala, RabbitMQ, and Apache Spark.
He is also the primary contributor to camel-metrics, an open source Apache Camel component for route metrics.
I would like to thank my wife, Hilary, for her support and encouragement in everything that I do. Without her, I would not be able to be who I am today.
Van Thoai Nguyen has worked in the software industry for a decade in various domains. In 2012, he joined BuzzNumbers as one of the core senior software engineers, where he had opportunities to design, implement, and apply many cool technologies, tools, and frameworks. A RabbitMQ cluster was employed as the backbone of the real-time data processing platform, which includes various data collectors, data filtering, enrichment, and storage using a sharded cluster of MongoDB and SOLR. He is still maintaining the open source .NET RabbitMQ client library—Burrow.NET (https://github.com/vanthoainguyen/Burrow.NET)—which he built during the time he worked for BuzzNumbers. This library is still being used in many different applications in that company. He is interested in clean code and design, SOLID principle, and big data. You can find his blog at http://thoai-nguyen.blogspot.com.au/.
He is currently reviewing the book Learning RabbitMQ by Packt Publishing.
Jorge Puente-Sarrín is from Peru and is a software developer at RebelMouse. Prior to RebelMouse, he worked at Red Científica Peruana (RCP) and El Comercio, where he lead the adoption and integration of MongoDB into the company's IT infrastructure. He is a passionate developer focused on building distributed systems solutions using asynchronous programming with Python and .NET. Also, he has been contributing toward the translation of documentation projects and online courses into Spanish. He is a proud member of Masters of MongoDB, a group of persons promoting MongoDB around the world. He has also technically reviewed RabbitMQ Cookbook, by Packt Publishing.
Ken Taylor has worked in software development and technology for over 15 years. During the course of his career, he has worked as a systems analyst on multiple software projects for several industries as well as U.S. government agencies. He has successfully used RabbitMQ for messaging on multiple projects. He previously reviewed the books RabbitMQ Cookbook and RabbitMQ Essentials by Packt Publishing. He is a member and speaker of the 757 Ruby user's group and the Hampton Roads .NET user's group (HRNUG). He holds an A.S. in computer science from Paul D. Camp Community College and was awarded a U.S. Patent for a real estate financial software product. He is currently working at Outsite Networks Inc. in Norfolk, Virginia. He lives in Virginia Beach with his lovely wife, Lucia, and his two sons, Kaide and Wyatt.
I would like to thank my family for being a constant support in all of my endeavors.
Héctor Veiga Ortiz is a Software Engineer specializing in real-time data integration. Recently, he has focused his work on different cloud technologies (AWS, Heroku, OpenShift, etc.) to develop scalable, resilient and high-performing applications able to handle high-volume real-time data in diverse protocols and formats. Additionally, he has a strong foundation in messaging systems knowledge, such as RabbitMQ and AMQP. Lately, he has been focusing his work on the Akka, Apache Spark and Apache Flink. Also, Héctor has a master's degree in Telecommunication Engineering from the Universidad Politécnica de Madrid and a master's degree in Information Technology and Management from the Illinois Institute of Technology.
Héctor currently works at HERE as part of Global Data Integrations and is actively developing scalable applications to consume and preprocess data from several different sources. HERE heavily utilizes RabbitMQ to address their messaging requirements. In the past, Héctor worked at Xaptum Technologies, a company dedicated to M2M technologies.
Héctor has also worked on reviewing RabbitMQ Cookbook, Learning RabbitMQ, and RabbitMQ Essentials all by Packt Publishing.
I would like to thank Laura for her support. She keeps inspiring me and supporting me with everything I do. Without her, this would not have been possible.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.
Why subscribe?
Free access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.
Preface
RabbitMQ is an open source messaging broker. It's often referred to as a message-oriented middleware that implements the Advanced Message Queuing Protocol (AMQP). Fundamentally, RabbitMQ provides a common platform for sending and receiving messages, where it guarantee the safety of messages until they are received. By playing an intermediary role between message consumers and producers, AMQP makes it easy to decouple applications.
Out of the box, RabbitMQ provides support for many messaging patterns. RabbitMQ guarantees data delivery, provides non-blocking operations, and sends push notifications. Moreover, it provides infrastructure for publish/subscribe, asynchronous processing, and work queues.
RabbitMQ provides a variety of features, including the tuning of application performance, clustering, flexible routing, federation, and so on. If you need specific features, RabbitMQ has several plugins that cater different needs. The RabbitMQ plugins extend its features in different ways, and you can also write your own plugin.
Through this book, we aim to give you a deep understanding of RabbitMQ and its use cases by providing multiple opportunities to learn about the message-oriented middleware, messaging architecture, messaging patterns, and solutions to real-life scenarios using RabbitMQ.
What this book covers
This book covers many aspects of software development with RabbitMQ. It provides thorough understanding of messaging, RabbitMQ, message-oriented software development, and so on.
Chapter 1, Getting Started, introduces you to message queues, message brokers, AMQP, and RabbitMQ.
Chapter 2, Configuring RabbitMQ, covers the configuration opportunities in RabbitMQ in detail.
Chapter 3, Architecture and Messaging, goes over RabbitMQ components—Producer, Message Broker, Consumer and the Message. This chapter provides learning opportunities for interoperability, heterogeneous integration, scalability, and so on.
Chapter 4, Clustering and High Availability, provides opportunities to tune RabbitMQ for high availability, federation, and much more.
Chapter 5, Plugins and Plugin Development, highlights several important features of RabbitMQ and gives an insight into creating your own plugin.
Chapter 6, Managing Your RabbitMQ Server, covers in detail the management of RabbitMQ using the command-line tools, management plugin, and rest API.
Chapter 7, Monitoring, discusses the methods to monitor RabbitMQ instances through a command line, management plugin, and well-known monitoring software.
Chapter 8, Security in RabbitMQ, covers the details about potential security vulnerabilities and securing RabbitMQ.
Chapter 9, Java RabbitMQ Client Programming, talks about developing RabbitMQ client using the Java platform.
Chapter 10, Ruby Client Programming, talks about developing a RabbitMQ client using Ruby.
Chapter 11, Python Client Programming, demonstrates developing a RabbitMQ client using Python.
What you need for this book
For this book, you need an understanding of software development—how to write functions, classes, and debugging skills. Moreover, you also need hands-on experience in developing applications.
Who this book is for
If you are an intermediate-level RabbitMQ developer and want to achieve professional-level expertise in the subject, this book is for you. You'll also need to have a decent understanding of message queuing.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "We have the dpkg dependency management tool for installing RabbitMQ."
A block of code is set as follows:
Any command-line input or output is written as follows:
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "In Windows, we should use the environment variables of the System Properties for modifying the environment variables of RabbitMQ."
Note
Warnings or important notes appear in a box like this.
Tip
Tips and tricks appear like this.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files from your account at http://www.packtpub.com for all the Packt Publishing books you have purchased. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the ErrataSubmissionForm link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
Chapter 1. Getting Started
Scalability is one of the major problems of our time, and messaging is an integral part of the solution. It finally comes down to the message broker software to manage and control messaging between applications, processes, and threads. Message brokers can help to solve scalability issues and architectural issues, such as coupling.
RabbitMQ is one of the most powerful open source message broker software that is widely used in the tech companies such as Mozilla, VMware, Google, AT&T, and so on. RabbitMQ is a highly configurable messaging platform developed and supported by a knowledgeable and committed community.
Before diving into the details and technologies behind the RabbitMQ, let's introduce you to the topics that we will cover in this chapter:
Message brokers and message queue
Recently, software systems evolved dramatically. Applications have to communicate with other applications, these applications can be internal and external to the application itself. For the same application, we may have different type of clients, such as browsers, mobile clients, and so on. Hence, we absolutely need a communication layer between internal applications and between applications and clients. We need to deliver different messages to different applications or clients. Delivering messages can be a bottleneck if the communication layer isn't scalable. Pursuing scalable systems for communication layer leads us to Message Brokers and Message Queues. Let's now discuss what Message Brokers and Message Queues are.
Message brokers
A Message Broker is an architectural pattern that can receive messages from multiple destinations, determine the correct destination, and route the message along the correct route, as stated in the book Enterprise Integration Patterns by Hohpe and Woolf. Message brokers enable systems to deal with messaging and routing by mediating communication among components. Once applications implement a message broker pattern, it decreases the coupling between application components.
Message Brokers are centralized, in the architectural sense, to control and manage all messages. Therefore, all of the incoming and outgoing messages are sent through Message Brokers, which analyze and deliver the messages to their correct destination. This procedural step can be understood with the following diagram:
Message Broker
Message Brokers address the following concerns in the communication layer:
Many tasks of the Message Broker need a Message Queue for exchanging or passing data to the destination. The next section covers Message Queues. We will talk about the mechanism behind Message Brokers in Chapter 3, Architecture and Messaging.
Message Queues
A Message Queue is, briefly, a queue for messaging. Queue is the basic data structure behind the functioning of a Message Queue. Message Queue operations are similar to Queue data structure operations, such as the enqueue and dequeu operations. An enqueue operation leads to adding an element to the back of the queue. A dequeue operation leads to the deletion of an element from the front of the queue.
Message Queues provide concurrent and asynchronous operations to scale applications. In a message queue, messages wait up until a message is retrieved by an application. Let's take a look at the following diagram:
Message Queue
Different types of standards and protocols define the Message Queuing specifications. Some protocols are open to everyone; however, some protocols are closed. Let's come back to our topic. RabbitMQ uses Advanced Message Queuing Protocol (AMQP) that determines the policies of the Message Queues. The next topic will cover detailed information on AMQP. Chapter 3, Architecture and Messaging, covers the detailed explanation of Message Queues.
An introduction to the advanced message queue protocol
John O'Hara from J. P. Morgan started AMQP in 2003. He put incredible amount of work into it. Then, J. P. Morgan approached other firms to establish an organization for creating open standards in messaging. According to AMOP's official website (http://www.amqp.org), AMQP is an open standard for passing messages between applications or organizations. So, AMQP just defines the messaging properties, queue properties, how messages are routed between applications and clients, how Message Brokers ensure that the message is received or sent, and other concerns such as reliability and security.
According to the AMQP website (http://www.amqp.org), AMQP has lots of capabilities to accomplish goals:
Interoperability and reliability are very important for today's software engineering problems. The power of AMQP comes from its features like interoperability, reliability and so forth. Especially, with interoperability, we can use different types of technologies in sender and receiver. The main problem for most of the Internet giants is scalability. Scalability has direct relationship to reliability. Chapter 3, Architecture and Messaging, covers the details and specifications of AMQP.
An overview of RabbitMQ
As mentioned earlier, RabbitMQ is an open source Message Broker software that tries to solve messaging problems by implementing the AMQP. RabbitMQ is licensed with Mozilla Public License. RabbitMQ became part of GoPivotal in May 2013 and the community has helped in the development of RabbitMQ. Since then, the community has been trying to improve RabbitMQ.
RabbitMQ logo
As we explained, RabbitMQ solves messaging problems; however, you may ask yourself what kind of messaging problems RabbitMQ solves. In summary, RabbitMQ has the following functionalities to solve messaging problems:
Now that we have discussed the concepts behind the RabbitMQ, let's try to install and use RabbitMQ on your computer.
Installation of RabbitMQ
Installation of RabbitMQ is not distinctly different from other software in different operation systems. Unix-based operating systems can build RabbitMQ from source code and Microsoft Windows can run the standard MSI installers. RabbitMQ installation files can be found in the download webpage of the RabbitMQ website, as shown in the following image:
Download Webpage of RabbitMQ
The only prerequisite for the RabbitMQ installation is the Erlang runtime environment because RabbitMQ runs on the Erlang VM. Therefore, we have to install Erlang before installing the RabbitMQ. Erlang can be downloaded from the Erlang download webpage, as shown in the following image, and installation instructions will be covered in the topics that follow:
Download Webpage of Erlang
Now, let's to talk about the installation of both RabbitMQ and Erlang on Windows, Mac OS X, Ubuntu, Fedora, and Amazon Web Services.
Windows
RabbitMQ runs on both 32-bit and 64-bit machines from the same package. Erlang is installed either as 32-bit and 64-bit. So, RabbitMQ can be easily installed on the Windows operating system. Let's install these stuff for running RabbitMQ.
Firstly, we should install the Erlang runtime environment on Windows. Erlang has Windows installers for 32-bit and 64-bit as shown in the previous image. We can easily download the related binary file to our computer and install Erlang using it:
Installation of Erlang in Windows
After installing the Erlang runtime environment, we've completed the requirements of RabbitMQ installation. The next step is to download and install the RabbitMQ binary file with related the Windows version:
Installation of Erlang in Windows
We can find the related Windows installer for RabbitMQ with the help of RabbitMQ download webpage as shown in the screenshot showing the download webpage of RabbitMQ. Then, we just need to click and install the RabbitMQ on our Windows computer. Besides installing using the installer, we can install using the Windows binary file that is served within the RabbitMQ download webpage. The following instructions will be enough for installing RabbitMQ without the installer:
It is possible to install the RabbitMQ on your Windows computer in both ways. Note that you may add the RabbitMQ binaries directory to the windows system path in the system/environment variable settings.
Mac OS X
As we specified, we only have one requirement to install RabbitMQ on our computers. In Mac OS X, we have package managers and we have the opportunity to compile from the source for both Erlang runtime environment and RabbitMQ.
Firstly, both Erlang and RabbitMQ can be easily installed on Mac OS X using package managers. Although we have lots of package managers on Mac OS X, Homebrew and MacPort are the ones that are mostly used in Mac OS X. So, we'll talk about the installation using Homebrew and MacPorts.
Homebrew has both RabbitMQ and Erlang on its repository. As RabbitMQ has a dependency with Erlang, Homebrew finds its dependent software and installs them together.
Note
Homebrew is just another package manager for Mac OS X. Homebrew is quite easy to install on Mac OS X and has lots of packages. So, you would find your application in its repository. Check it out at http://brew.sh
So, we just need to install RabbitMQ in Homebrew using the brew install rabbitmq command on our terminal as shown in the following image:
Homebrew Installation of RabbitMQ
MacPorts has the similar method of operation with Homebrew. MacPorts also installs the software with its dependencies. Therefore, we just need to install RabbitMQ in MacPorts using the port install rabbitmq-server command on our terminal, as shown in the following image:
MacPorts Installation of RabbitMQ
Another way to install RabbitMQ and Erlang is by compiling the source codes in Mac OS X. Before compiling, we need the following to compile Erlang source code:
After downloading and unzipping the source codes of the Erlang, we just need common commands on the Erlang folder for compiling from source code, as follows:
Finally, we just need to download and unzip the RabbitMQ binary files.
Ubuntu
Ubuntu is just another Linux distribution based on Debian. Similar instructions as the ones we discussed for the installation on the Mac OS X would be applied for Ubuntu.
Ubuntu has a package manager called Advanced Packaging Tool (apt-get) and has a Debian package manager called dpkg. So, we are able to install RabbitMQ and Erlang runtime environment using apt-get. Moreover, similar to Mac OS X, we can compile from source codes of Erlang.
Firstly, as we said in the previous paragraph, we can install RabbitMQ using apt-get and dpkg. Before installing RabbitMQ, we should add the RabbitMQ repository to the APT repository using the following line (add the following line to /etc/apt/sources.list):
Now, we are ready to install RabbitMQ and its dependency Erlang runtime environment, as shown in the following image:
Ubuntu Installation of RabbitMQ
Also, we have the dpkg dependency management tool for installing RabbitMQ. RabbitMQ has packages for dpkg in its download webpage. We can download it from its website then run the following command:
Secondly, we have another option, which was explained in the Mac OS X topic. That is, compiling from source codes. We just need to compile the downloaded Erlang source code, and we are ready to run the downloaded binary files of RabbitMQ. You can look at the details in the Mac OS X section.
Fedora
Fedora is yet another Linux distribution based on Red Hat. Installation instructions of Fedora are similar to Ubuntu's installation. Fedora has package managers called rpm and yum.
Firstly, we are able to install RabbitMQ with its dependency Erlang using package managers. Before using yum, we should run the following command to add RabbitMQ repository:
Then, we can install RabbitMQ using the following command as shown in the following screenshot:
Fedora Installation of RabbitMQ
Fedora has another package manager, which comes from Red Hat, called rpm. As RabbitMQ publishes package as rpm, we can easily install using the rpm package manager. After downloading the rpm package from RabbitMQ webpage, we can install RabbitMQ with Erlang, using following command:
Secondly, we have another option, which was explained in the Mac OS X topic. That is compiling from source code. We just need to compile the downloaded Erlang source code and we are ready to run the downloaded binary files of RabbitMQ. You can look at the details in the Mac OS X section.
Amazon elastic compute cloud (EC2)
Amazon Web Services (AWS
