39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Microservices have become a popular choice for building distributed systems that power modern web and mobile apps. They enable you to deploy apps as a suite of independently deployable, modular, and scalable services. With over 70 practical, self-contained tutorials, the book examines common pain points during development and best practices for creating distributed microservices. Each recipe addresses a specific problem and offers a proven, best-practice solution with insights into how it works, so you can copy the code and configuration files and modify them for your own needs.
You’ll start by understanding microservice architecture. Next, you'll learn to transition from a traditional monolithic app to a suite of small services that interact to ensure your client apps are running seamlessly. The book will then guide you through the patterns you can use to organize services, so you can optimize request handling and processing. In addition this, you’ll understand how to handle service-to-service interactions. As you progress, you’ll get up to speed with securing microservices and adding monitoring to debug problems. Finally, you’ll cover fault-tolerance and reliability patterns that help you use microservices to isolate failures in your apps.
By the end of this book, you’ll have the skills you need to work with a team to break a large, monolithic codebase into independently deployable and scalable microservices.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 251
Veröffentlichungsjahr: 2018
Ähnliche
Microservices Development Cookbook
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Technical Reviewer:Matt McLarty Commissioning Editor: Merint MathewAcquisition Editor: Alok DhuriContent Development Editor: Rohit Kumar SinghTechnical Editor: Ruvika RaoCopy Editor: Safis EditingProject Coordinator: Vaidehi SawantProofreader: Safis EditingIndexer: Mariammal ChettiyarGraphics: Jason MonteiroProduction Coordinator: Shantanu Zagade
First published: August 2018
Production reference: 1300818
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78847-950-9
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Paul Osman has been building external and internal platforms for over 10 years. From public APIs targeted at third parties to internal platform teams, he has helped build distributed systems that power large-scale consumer applications. He has managed teams of engineers to rapidly deliver service-based software systems with confidence.Paul has published articles and given multiple conference talks on microservices and DevOps. He is a passionate advocate of open technology platforms and tools.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Microservices Development Cookbook
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the author
Packt is searching for authors like you
Preface
Why microservices?
Microservice prerequisites
Microservices benefits
Scaling
Team organization
Reliability
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Conventions used
Sections
Getting ready
How to do it…
Get in touch
Reviews
Breaking the Monolith
Introduction
Organizing your team
How to do it…
Discussion
Decomposing by business capability
How to do it...
Identifying bounded contexts
How to do it...
Migrating data in production
How to do it...
Refactoring your monolith
How to do it...
Evolving your monolith into services
How to do it...
Evolving your test suite
Getting ready
How to do it...
Using Docker for local development
Getting ready
How to do it...
Routing requests to services
How to do it...
Edge Services
Introduction
Controlling access to your service with an edge proxy server
Operational notes
How to do it...
Extending your services with sidecars
How to do it...
Using API Gateways for routing requests to services
Design considerations
How to do it...
Stopping cascading failures with Hystrix
How to do it...
Rate limiting
How to do it...
Using service mesh for shared concerns
How to do it...
Inter-service Communication
Introduction
Service-to-service communication
How to do it...
Asynchronous requests
How to do it...
Service discovery
How to do it...
Registering with the service registry
Finding services
Server-side load balancing
How to do it...
Client-side load balancing
How to do it...
Building event-driven microservices
How to do it...
Message producer
Message consumer
Evolving APIs
How to do it...
Client Patterns
Introduction
Modeling concurrency with dependent futures
How to do it...
Backend for frontend
How to do it...
Consistent RPC with HTTP and JSON
How to do it...
Using Thrift
How to do it...
Using gRPC
How to do it...
Reliability Patterns
Introduction
Using circuit breakers
How to do it...
Retrying requests with exponential backoff
How to do it...
Improving performance with caching
How to do it...
Fronting your services with a CDN
How to do it...
Gracefully degrading the user experience
Verifying fault tolerance with Gameday exercises
Prerequisites
How to do it...
A template for Gameday exercises
Introducing automated chaos
How to do it...
Security
Introduction
Authenticating your microservices
How to do it...
Securing containers
How to do it...
Secure configuration
How to do it...
Secure logging
Infrastructure as Code
How to do it...
Monitoring and Observability
Introduction
Structured JSON logging
How to do it...
Collecting metrics with StatsD and Graphite
How to do it...
Collecting metrics with Prometheus
How to do it...
Making debugging easier with tracing
How to do it...
Alerting us when something goes wrong
How to do it...
Scaling
Introduction
Load testing microservices with Vegeta
How to do it...
Load testing microservices with Gatling
How to do it...
Building auto-scaling clusters
How to do it...
Deploying Microservices
Introduction
Configuring your service to run in a container
How to do it…
Running multi-container applications with Docker Compose
How to do it…
Deploying your service on Kubernetes
How to do it…
Test releases with canary deployments
How to do it…
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Why microservices?
Microservices has become an increasingly popular subject over the last few years. As with any new architectural concept, there is plenty of room for misunderstanding. Even the term microservices is confusing. Newcomers are often unsure about the appropriate size of a microservice(hint: it's not actually about the size of the code base) and can get stuck on how to get started with this architectural style.
Service-oriented architectures are nothing new. Web services were being promoted by various companies in the 1990s as a solution to large, inflexible code bases. The promise was that web services would provide reusable capabilities that could be easily consumed by your code bases. Technologies such as SOAP and WSDL started gaining adoption, but never seemed to deliver on the ease of use promise. Meanwhile, open source languages such as PHP, Ruby, and Python with frameworks such as Symfony, Rails, and Django made developing monolithic web-centric code bases easier.
Fast forward a couple of decades and we started seeing a renewed interest in services. So, what's changed? For one, with the advent of rich web and mobile applications, every system is now a distributed system. Thanks to the advent of cloud computing, compute and storage resources are cheaper than they've ever been. Containers are changing the way we think about deploying and operating our services. Many consumer services are outgrowing their monolithic code bases, and teams are finding them hard to scale. Microservices can help with many of these challenges.
Microservice prerequisites
Microservices aren't a panacea. While they have many benefits(which we'll discuss later), they also introduce some specific challenges. Before deciding to make the move to microservices, it's important to have certain infrastructure and tooling in place. Martin Fowler has written about Microservices Prerequisites (https://martinfowler.com/bliki/MicroservicePrerequisites.html), as has Phil Calcado (http://philcalcado.com/2017/06/11/calcados_microservices_prerequisites.html). I won't repeat what others have written; instead, I will just say that it pays to have a certain amount of automation and monitoring in place before you start developing microservices. Your teams should be comfortable sharing on-call duties and you should have a system for managing alerts and escalations, such as PagerDuty (http://pagerduty.com/).
Microservices benefits
The various benefits of microservices are as discussed in the next sections.
Scaling
In a monolithic code base, scaling is anall-or-nothing approach. Microservices make it easier to scale separate parts of your application based on their own needs. For example, you might have a particular part of your application that is in the critical path of every user request(that is, authentication/authorization services), whereas other parts are only used by a subset of your users(that is, search or messaging). Different traffic patterns will translate to different scaling needs and different techniques that should be used to scale a service. A service that requires a read for every request from a user should use a data store that makes reads cheap. Services that do not need to provide strongly consistent results can take advantage of extensive caching.
Team organization
When teams of engineers are working on separate code bases with separate deployments, they are able to make a lot of decisions independently, without the need to coordinate with other teams in the organization. This means that engineers are free to commit code, design their own code review processes, and deploy to production without always needing to coordinate. In a monolith, it's not uncommon for engineers to have to get their changes into a queue that is then deployed at a set time with changes from other teams. If something goes wrong(poison deploys are one of the most common causes of outages), then the whole change set gets rolled back, delaying work by multiple teams. Microservices help you avoid this by allowing teams to move with more autonomy.
Reliability
When a monolith fails, it tends to fail completely. A database is unavailable, and then the application tries to use stale connections in a connection pool, eventually the threads or processes serving requests lock up, and users are left with a white screen of death or a inoperable mobile application. Microservices allow you to decide on a case-by-case basis how a failure in a particular part of your application should be treated. If your service cannot reach a database, perhaps it's better to return a stale cache, or an empty response. If your service has to throw up its hands and start returning HTTP 503 responses, upstream services can respond by applying back pressure, allowing the service to catch up. Microservices give you much more freedom to isolate failures in your system, resulting in a happier experience for your users.
This book will serve as a handy reference for many of the subjects that will come up as you develop microservices. We'll start with recipes that will help you make the transition from a monolith to a suite of microservices. Subsequent chapters will address specific areas or challenges that come up when choosing how best to architect and manage your microservices. Recipes that cover code will include working, simple, tested examples that you can use in your own applications. My hope is that this book will help you think about, plan, and execute the development of microservice-based applications. Enjoy!
Who this book is for
If you are a developer who would like to build effective and scalable microservices, then this book is for you. Basic knowledge of the microservices architecture is assumed.
What this book covers
Chapter 1, Breaking the Monolith, shows how to make the transition from monolith to microservices, with the recipes focused on architectural design. You'll learn how to manage some of the initial challenges when you begin to develop features using this new architectural style.
Chapter 2, Edge Services, teaches you how to use open source software to expose your services to the public internet, control routing, extend your service's functionality, and handle a number of common challenges when deploying and scaling microservices.
Chapter 3, Inter-Service Communication, discusses recipes that will enable you to confidently handle the various kinds of interactions we're bound to require in a microservice architecture.
Chapter 4, Client Patterns, discusses techniques for modeling dependent service calls and aggregating responses from various services to create client-specific APIs. We'll also discuss managing different microservices environments and making RPC consistent with JSON and HTTP, as well as the gRPC and Thrift binary protocols.
Chapter 5, Reliability Patterns, discusses a number of useful reliability patterns that can be used when designing and building microservices to prepare for and reduce the impact of system failures, both expected and unexpected.
Chapter 6, Security, includes recipes that will help you learn a number of good practices to consider when building, deploying, and operating microservices.
Chapter 7, Monitoring and Observability, introduces several tenants of monitoring and observability. We'll demonstrate how to modify our services to emit structured logs. We'll also take a look at metrics, using a number of different systems for collecting, aggregating, and visualizing metrics.
Chapter 8, Scaling, discusses load testing using different tools. We will also set up auto-scaling groups in AWS, making them scalable on demand. This will be followed by strategies for capacity planning.
Chapter 9, Deploying Microservices, discusses containers, orchestration, and scheduling, and various methods for safely shipping changes to users. The recipes in this chapter should serve as a good starting point, especially if you're accustomed to deploying monoliths on virtual machines or bare metal servers.
To get the most out of this book
This books assumes basic knowledge of microservices architectures. Other instructions are mentioned in the respective recipes as needed.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packtpub.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub athttps://github.com/PacktPublishing/Microservices-Development-Cookbook. We also have other code bundles from our rich catalog of books and videos available athttps://github.com/PacktPublishing/. Check them out!
Sections
In this book, you will find several headings that appear frequently (Getting ready, How to do it...).
To give clear instructions on how to complete a recipe, use these sections as follows:
Getting ready
This section tells you what to expect in the recipe and describes how to set up any software or any preliminary settings required for the recipe.
How to do it…
This section contains the steps required to follow the recipe.
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Breaking the Monolith
In this chapter, we will cover the following recipes:
Organizing your team to embrace microservices
Decomposing by business capability
Identifying bounded contexts
Migrating data in production
Refactoring your monolith
Evolving your monolith into services
Evolving your test suite
Using Docker for local development
Routing requests to services
Introduction
One of the hardest things about microservices is getting started. Many teams have found themselves building features into an ever-growing, hard-to-manage monolithic code base and don't know how to start breaking it apart into more manageable, separately deployable services. The recipes in this chapter will explain how to make the transition from monolith to microservices. Many of the recipes will involve no code whatsoever; instead, they will be focused on architectural design and how best to structure teams to work on microservices.
You'll learn how to begin moving from a single monolithic code base to suites of microservices. You'll also learn how to manage some of the initial challenges when you begin to develop features using this new architectural style.
Organizing your team
Conway's law tells us that organizations will produce designs whose structure is a copy of their communication structure. This often means that the organizational chart of an engineering team will have a profound impact on the structure of the designs of the software it produces. When a new startup begins building software, the team is small—sometimes it is comprised of just one or two engineers. In this setup, engineers work on everything, including frontend and backend systems, as well as operations. Monoliths suit this organizational structure very well, allowing engineers to work on any part of the system at any given time without moving between code bases.
As a team grows, and you start to consider the benefits of microservices, you can consider employing a technique commonly referred to as an theInverse Conway Maneuver. This technique recommends evolving your team and organizational structure to encourage the kind of architectural style you want to see emerge. With regard to microservices, this will usually involve organizing engineers into small teams that you will eventually want to be responsible for a handful of related services. Setting your team up for this structure ahead of time can motivate engineers to build services by limiting communication and decision-making overhead within the team. Simply put, monoliths continue to exist when the cost of adding features as services is greater than the cost of adding a feature to the monolith. Organizing your teams in this way reduces the cost of developing services.
This recipe is aimed at managers and other leaders in companies who have the influence to implement changes to the structure of the organization.
How to do it…
Re-organizing a team is never a simple task, and there are many non-obvious factors to consider. Factors such as personality, individual strengths and weaknesses, and past histories are outside the scope of this recipe, but they should be considered carefully when making any changes. The steps in this recipe provide one possible way to move a team from being organized around a monolithic code base to being optimized for microservices, but there is no one-size-fits-all recipe for every organization.
Use the following steps as a guide if you think they apply, but otherwise use them for inspiration and to encourage thought and discussion:
Working with other stakeholders in your organization, build out a product roadmap. You may have limited information about the challenges your organization will face in the short term, but do the best you can. It's perfectly natural to be very detailed for short-term items on a roadmap and very general for the longer term.
Using the product roadmap, try to identify technical capabilities that will be required to help you deliver value to your users. For example, you may be planning to work on a feature that relies heavily on search. You may also have a number of features that rely on content uploading and management. This means that search and uploading are two technical capabilities you know you will need to invest in.
As you see patterns emerge, try to identify the main functional areas of your application, paying attention to how much work you anticipate will go into each area. Assign higher priorities to the functional areas you anticipate will need a lot of investment in the short to medium term.
Create new teams, ideally consisting of four to six engineers, who are responsible for one of the functional areas within your application. Start with the functional areas that you anticipate will require the most work over the next quarter or so. These teams can be focused on the backend services or they can be cross-functional teams that include the mobile and web engineers. The benefit of having cross-functional teams is that the team can then deliver the entire vertical component of the application autonomously. The combination of service engineers with engineers consuming their services will also enable more information sharing, and hopefully, empathy.
Discussion
Using this approach, you should end up with small, cohesive, and focused teams responsible for core areas of your application. The nature of teams is that individuals within the team should start to see the benefit of creating separately managed and deployed code bases that they can work in autonomously without the costly overhead of coordinating changes and deployments with other teams.
To help illustrate these steps, imagine your organization builds an image-messaging application. The application allows users to take a photo with their smart phone and send it, along with a message, to a friend in their contacts list. Their friends can also send them photos with messages. A fictional roadmap for this fictional product could involve the need to add support for short videos, photo filters, and support for emojis. You now know that the ability to record, upload, and play videos, the ability to apply photo filters, and the ability to send rich text will be important to your organization. Additionally, you know from experience that users need to register, log in, and maintain a friends list.
Using the preceding example, you may decide to organize engineers into a media team, responsible for uploading, processing and playing, filters, and storage and delivery, a messaging team, responsible for the sending of photo or video messages with associated text, and a users team, responsible for providing reliable authentication, registration, on-boarding, and social features.
Decomposing by business capability
In the early stages of product development, monoliths are the best suited to delivering features to users as quickly and simply as possible. This is appropriate, as at this point in a products development you do not have luxury problems of having to scale your teams, code bases or ability to serve customer traffic. Following good design practices, you separate your applications concerns into easy-to-read, modular code patterns. Doing so allows engineers to work on different sections of the code autonomously and limits the possibility of having to untangle complicated merge conflicts when it comes time to merge your branch into the master and deploy your code.
Microservices require you to go a step further than the good design practices you've already been following in your monolith. To organize your small, autonomous teams around microservices, you should consider first identifying the core business capabilities that your application provides. Business capability is a business school term that describes the various ways your organization produces value. For example, your internal order management is responsible for processing customer orders. If you have a social application that allows users to submit user-generated content such as photos, your photo upload system provides a business capability.
When thinking about system design, business capabilities are closely related to the Single Responsibility Principle (SRP) from object-oriented design (OOD). Microservices are essentially SRP extended to code bases. Thinking about this will help you design appropriately sized microservices. Services should have one primary job and they should do it well. This could be storing and serving images, delivering messages, or creating and authenticating user accounts.
How to do it...
Decomposing your monolith by business capability is a process. These steps can be carried out in parallel for each new service you identify a need for, but you may want to start with one service and apply the lessons you learn to subsequent efforts:
Identify a business capability that is currently provided by your monolith. This will be the target for our first service. Ideally this business capability is something that has some focus on the roadmap you worked on in the previous recipe and ownership can be given to one of your newly created teams. Let's use our fictional photo messaging service as an example and assume we'll start with the ability to upload and display media as our first identified business capability. This functionality is currently implemented as a single model and controller in your
Ruby on Rails
monolith:
In the preceding screenshot,
AttachmentsController
has four methods (called
actions
in Ruby on Rails lingo), which roughly correspond to the
create, retrieve, update, delete
(
CRUD
) operations you want to perform on an
Attachment
resource. We don't strictly need it, and so will omit the update action. This maps very nicely to a RESTful service, so you can design, implement, and deploy a microservice with the following API:
POST /attachments
GET /attachments/:id
DELETE /attachments/:id
With the new microservice deployed (migrating data is discussed in a later recipe), you can now begin modifying client code paths to use the new service. You can begin by replacing the code in the
Attachm
entsController
action's methods to make an HTTP request to our new microservice. Techniques for doing this are covered in the
Evolving your monolith into services
recipe later in this chapter.
