34,79 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
- Veröffentlichungsjahr: 2016
Learn to develop efficient and scalable microservices for server-side programming in Node.js using this hands-on guide
About This Book
- Real world example explained chapter after chapter with code examples.
- Useful concepts for other languages like Java or PHP
- Easy to follow by people with little to none experience in Node.js
- Node.js Version 0.12.2 and the latest compatible versions of Seneca and PM2
Who This Book Is For
Node.js developer with basic server-side development knowledge - but no knowledge of implementing microservices in Node.js applications. Also useful for developers in other languages like Java or C#.
What You Will Learn
- Identify where the microservice oriented architectures can tackle the most common problems in the software used by the big organisations.
- Re-architecture an existing monolithic system into a microservices oriented software.
- Build robust and scalable microservices using Seneca and Node.js.
- Testing of the microservices in insolation in order to create a solid system.
- Deploy and manage microservices using PM2
- Monitoring the health of a microservice (CPU, memory, I/O...) and how the degradation of the performance in one microservice could degrade the performance of full system.
In Detail
Microservices architecture is a style of software architecture. As the name suggests, microservicess refers to small services. For a large implementation, this means breaking the system into really small, independent services. Alternative to monolithic architecture (where the entire system is considered as a single big, interwoven segment), microservices approach is getting more and more popular with large, complex applications that have a very long lifecycle, which require changes at regular intervals. Microservices approach allows this type of changes with ease as only a part of the system undergoes changes and change control is easy.
An example of such large system can be an online store—includes user interface, managing product catalog, processing orders, managing customer's account. In a microservices architecture each of these tasks will be divided and into smaller services. Also, these services will be further broken down into independent services—for user interface, there will be separate services for input, output, search bar management, and so on. Similarly, all other tasks can be divided in very small and simple services.
Style and approach
This book is a comprehensive guide to developing production-ready implementation of the microservices architecture with Node.js. Each chapter builds up on the earlier one creating a strong foundation on which you create your own microservice in a distributed environment.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 319
Ähnliche
Table of Contents
Developing Microservices with Node.js
Developing Microservices with Node.js
Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: April 2016
Production reference: 1220416
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78588-740-6
www.packtpub.com
Credits
Author
David Gonzalez
Reviewer
Kishore Kumar Yekkanti
Commissioning Editor
Veena Pagare
Acquisition Editor
Divya Poojari
Content Development Editor
Abhishek Jadhav
Technical Editor
Pranil Pathare
Copy Editor
Vibha Shukla
Project Coordinator
Judie Jose
Proofreader
Safis Editing
Indexer
Monica Ajmera Mehta
Production Coordinator
Arvindkumar Gupta
Cover Work
Arvindkumar Gupta
About the Author
David Gonzalez is a language-agnostic software engineer working in financial services for a number of years, trying to find solutions for the right level of abstraction and learning how to get the right balance between too concrete and too abstract.
He studied in Spain, but soon moved to the wider and more interesting market of Dublin, where he has been living since 2011. David is currently working as an independent consultant in the FinTech sector. The URL to his Linkedin account is https://ie.linkedin.com/in/david-gonzalez-737b7383.
He loves experimenting with new technologies and paradigms in order to get the broader picture of the complex world of software development.
To my wife, Ester, thanks for your unconditional support in every single aspect of my life.
To my unborn daughter, Elena, may the life give you all the happiness that you are bringing to your parents.
About the Reviewer
Kishore Kumar Yekkanti is an experienced professional who has worked across various domains and technologies over the past ten years. He is passionate about eliminating the waste during the software development. Kishore is a huge contributor to and follower of agile principles. He is a full-stack developer who wants to build the end-to-end systems, and a polyglot programmer. His current focus is on scaling microservices in highly distributed applications that are deployed using container-based systems (Docker) on cloud. He has worked as the lead/principal engineer for many well-known companies such as Thoughtworks, CurrencyFair, and others, where he lead the teams to attain nirvana through microservices.
For my partner and best friend Jyothsna, and my daughter, Dhruti, who continues to humour me despite of my crazy schedules.
www.PacktPub.com
eBooks, discount offers, and more
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.
Why subscribe?
Preface
This book is a hands-on guide to start writing microservices using Node.js and the most modern frameworks, especially Seneca and PM2. In the chapters, we will discuss how to design, build, test, and deploy microservices using the best practices. We will also discuss a valuable lesson: how to make the right level of compromise in order to avoid over-designing and get the business requirements aligned with the technical solutions.
What this book covers
Chapter 1, Microservices Architecture, discusses the pros and cons of microservices-oriented architectures. It will be the foundation for the rest of the book.
Chapter 2, Microservices in Node.js – Seneca and PM2 Alternatives, introduces Node.js, Seneca, and PM2. In it, we will discuss the structure of a Node.js application and how to run it using PM2. We will also dive into a few alternatives to Seneca and PM2.
Chapter 3, From the Monolith to Microservices, covers how to tackle the organic growth (unplanned software changes) using microservices. We will also talk about how to split a monolithic application into microservices.
Chapter 4, Writing Your First Microservice in Node.js, explains how to write our first microservice.
Chapter 5, Security and Traceability, covers how security and traceability are two important characteristics of modern systems. We need to keep the information secure and the actions traceable. In this chapter, we will discuss how to do it using Seneca.
Chapter 6, Testing and Documenting Node.js Microservices, introduces using the main frameworks for testing on Node.js: Mocha and Chai. We will also make use of Sinon (another framework) to mock services. Swagger will be our choice for documenting microservices.
Chapter 7, Monitoring Microservices, covers using PM2 to monitor our microservices. We will use it along with Keymetrics to get the maximum benefit of such an advanced tool.
Chapter 8, Deploying Microservices, explains how, using PM2, we are going to learn to deploy microservices in our different environments, managing our ecosystem of applications with a single command and reducing the overhead introduced by the microservices architecture. We will also discuss Docker, one of the most advanced systems to deploy applications not only in Node.js, but also in general applications.
What you need for this book
In order to follow the book, we are going to need to install Node.js, PM2 (it is a package that is installed through npm), and MongoDB.
We will also need an editor. Personally, I use Atom, but any general purpose editor should be enough.
Who this book is for
This book is for developers that have some experience with Node.js but want to learn about Seneca and microservices. It is 70% practice-oriented (as we will be writing a lot of code), but it also is 30% theory, based on the code created that will help the reader to apply the exposed patterns to new software.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "We know that the input parameter is a PaymentRequest instance."
A block of code is set as follows:
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
Any command-line input or output is written as follows:
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "In this case, I have used Chrome and the console shows an Uncaught TypeError: Cannot set property 'innerText' of null error in line 7."
Note
Warnings or important notes appear in a box like this.
Tip
Tips and tricks appear like this.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
Chapter 1. Microservices Architecture
Microservices are becoming more and more popular. Nowadays, pretty much every engineer on a green field project should be considering using microservices in order to improve the quality of the systems they build. They should know the architectural principles involving such systems. We will expose the difference between microservices and Service-Oriented Architecture (SOA). We will also introduce a great platform to write microservices, Node.js, which will allow us to create high-performing microservices with very little effort.
In this chapter, you will learn about microservices from the architectural point of view:
Need for microservices
The world of software development has evolved quickly in the past 40 years. One of the key points of this evolution has been the size of these systems. From the days of MS-DOS, we have taken a hundred-fold leap into our present systems. This growth in size creates a need for better ways of organizing code and software components. Usually, when a company grows due to business needs, known as organic growth, the software is organized on a monolithic architecture as it is the easiest and quickest way of building software. After few years (or even months), adding new features becomes harder due to the coupled nature of the created software.
Monolithic software
The natural trend for new high-tech companies such as Amazon or Netflix is building their new software using microservices, which is the ideal scenario: they get a huge advantage of microservices-oriented software (through out this book, you will learn how) in order to scale up their new products without a big effort. The problem is that not all companies can plan their software upfront. Instead of planning, these companies build software based on the organic growth experienced: few software components group business flows by affinity. It is not rare to see companies with two big software components: the user-facing website and the internal administration tools. This is usually known as a monolithic software architecture.
Some of these companies face big problems when trying to scale the engineering teams. It is hard to coordinate teams that build, deploy, and maintain a single software component. Clashes on releases and reintroduction of bugs are a common problem that drains a large chunk of energy from the teams. One of the solution to this problem (it comes with benefits) is to split the monolithic software into microservices so that the teams are able to specialize in a few smaller modules and autonomous and isolated software components that can be versioned, updated, and deployed without interfering with the rest of the systems of the company.
Splitting the monolith into microservices enables the engineering team to create isolated and autonomous units of work that are highly specialized in a given task such as sending e-mails, processing card payments, and so on.
Microservices in the real world
Microservices are small software components that are specialized in one task and work together to achieve a higher-level task. Forget about software for a second and think about how a company works. When someone applies for a job in a company, he applies for a given position: software engineer, system administrator, office manager. The reason for this can be summarized in one word: specialization. If you are used to work as a software engineer, you will get better with the experience and add more value to the company. The fact that you don't know how to deal with a customer, won't affect your performance as that is not your area of expertise and will hardly add any value to your day-to-day work.
Tip
Specialization is often the key to improve the efficiency. Doing one thing and doing it right is one of the mantras of software development.
A microservice is an autonomous unit of work that can execute one task without interfering with other parts of the system, similar to what a job position is to a company. This has a number of benefits that can be used in favor of the engineering team in order to help scale the systems of a company.
Nowadays, hundreds of systems are built using microservices-oriented architectures, as follows:
Microservice-oriented architectures
Microservices-oriented architectures have some particularities that makes them desirable for any mid/large-sized company that wants to keep their IT systems resilient and in scale up/down-ready status.
How is it better?
They are not the holy grail of software engineering, but, when handled with care, they become the perfect approach to solve most of the big problems faced by tech-dependent companies.
It is important to keep the key principles of the microservices-oriented architecture's design in mind, such as resilience, composability, elasticity, and so on; otherwise, you could end up with a monolithic application split across different machines that produces problems rather than an elegant solution.
Shortcomings
There is also some criticism around microservices-oriented architectures, as they introduce some problems to deal with, such as latency, traceability, and configuration management that are not present with monolithic-based software. Some of the problems are described as follows:
In general, engineers should try to evaluate the pros and cons of this approach and make a decision on whether to use microservices or not in order to fit the business needs.
Microservices-oriented architectures have some particularities that need to be taken into consideration. When a software engineer is writing monolithic software, there are some problems that are completely overlooked due to the nature of the software being built.
For example, imagine that our software needs to send e-mails. In a monolithic software, we would just add the functionality to the core of the application. We might even choose to create a dedicated module to deal with e-mails (which seems like a good idea). Now, imagine that we are creating a microservice and, instead of adding a functionality to a big software artifact, we create a dedicated service that can be deployed and versioned independently. In this case, we will have an extra step that we didn't have to take into consideration, the network latency, to reach the new microservice.
In the preceding example, no matter what approach (monolithic or microservices) you are taking to build the software, is not a big deal; for example, if an e-mail is lost, it is not the end of the world. As per definition, the e-mail delivery is not guaranteed, so our application will still work, although we might receive a few complaints from our customers.
Key design principles
There are a few key design principles that need to be taken into consideration when building microservices. There is no golden rule and, as microservices are a recent concept, sometimes there is even a lack of consensus on what practices to follow. In general, we can assume the following design principles:
Business units, no components
One of the most enjoyable sides of software engineering is creating a new project. This is where you can apply all your creativity, try new architectural concepts, frameworks, or methodologies. Unfortunately, it is not a common situation in a mature company. Usually, what we do is create new components inside the existing software. One of the best design principles that you can follow when creating new components is keeping the coupling as low as possible with the rest of the software, so that it works as an independent unit.
Tip
Keeping a low level of coupling allows a software component to be converted into a microservice with little to no effort.
Consider a real-world example: the application of your company now needs to be able to process payments.
The logical decision here would be creating a new module that knows how to deal with the chosen payment provider (credit cards, PayPal, and so on) and allows us to keep all the payment-related business logic inside of it. Let's define the interface in the following code:
This simple interface can be understood by everyone, but it is the key when moving towards microservices. We have encapsulated all the business knowledge behind an interface so that we could theoretically switch the payment provider without affecting the rest of the application—the implementation details are hidden from the outer world.
The following is what we know until now:
We have created a highly cohesive and loosely coupled business unit. Let's justify this affirmation in the following:
The following diagram shows how a system composed of many components gets one of them (payment service) stripped out into a microservice:
Once this module is implemented, we will be able to process the payments and our monolithic application will have another functionality that could be a good candidate to extract into a microservice.
Now, we can rollout new versions of the payment service, as long as the interface does not change, as well as the contract with the rest of the world (our system or third parties), hasn't changed. That is why it is so important to keep the implementation independent from interfacing, even though the language does not provide support for interfaces.
We can also scale up and deploy as many payment services as we require so that we can satisfy the business needs without unnecessarily scaling the rest of the application that might not be under pressure.
Tip
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
Smart services, dumb communication pipes
Hyper Text Transfer Protocol (HTTP) is one of the best things to have ever happened to the Internet. Imagine a protocol that was designed to be state-less, but was hacked through the use of cookies in order to hold the status of the client. This was during the age of Web 1.0, when no one was talking about REST APIs or mobile apps. Let's see an example of an HTTP request:
As you can see, it is a human readable protocol that does not need to be explained in order to be understood.
Nowadays, it is broadly understood that HTTP is not confined to be used in the Web, and as it was designed, it is now used as a general purpose protocol to transfer data from one endpoint to another. HTTP is all you need for the communication between microservices: a protocol to transfer data and recover from transmission errors (when possible).
In the past few years, especially within the enterprise world, there has been an effort to create smart communication mechanisms such as BPEL. BPEL stands for Business Process Execution Language, and instead of focusing on communication actions, it focuses on actions around business steps.
This introduces some level of complexity in the communication protocol and makes the business logic of the application bleed into it from the endpoints, causing some level of coupling between the endpoints.
The business logic should stay within the endpoints and not bleed into the communication channel so that the system can be easily tested and scaled. The lesson learned through the years is that the communication layer must be a plain and simple protocol that ensures the transmission of the data and the endpoints (microservices).These endpoints should embed into their implementation the fact that a service could be down for a period of time (this is called resilience, we will talk about this later in this chapter) or the network could cause communication issues.
HTTP usually is the most used protocol when building microservices-oriented software but another interesting option that needs to be explored is the use of queues, such as Rabbit MQ and Kafka, to facilitate the communication between endpoints.
The queueing technology provides a clean approach to manage the communication in a buffered way, encapsulating the complexities of acknowledging messages on highly transactional systems.
Decentralization
One of the major cons of monolithic applications is the centralization of everything on a single (or few) software components and databases. This, more often than not, leads to huge data stores that needs to be replicated and scaled according to the needs of the company and centralized governance of the flows.
Microservices aim for decentralization. Instead of having a huge database, why not split the data according to the business units explained earlier?
Some of the readers could use the transactionality as one of the main reasons for not doing it. Consider the following scenario:
In a monolithic software, all the calls would be wrapped in a transaction, so if, for some reason, any of the calls fails, the data on the other calls won't be persisted in the database.
When you learn about designing databases, one of the first and the most important principles are summarized by the ACID acronym:
On a microservices-oriented software, the ACID principle is not guaranteed globally. Microservices will commit the transaction locally, but there are no mechanisms that can guarantee a 100% integrity of the global transaction. It would be possible to dispatch the book without processing the payment, unless we factor in specific rules to prevent it.
On a microservices-oriented architecture, the transactionality of the data is not guaranteed, so we need to factor the failure into the implementation. A way to solve (although, workaround is a more appropriate word) this problem is decentralizing the governance and data storage.
When building microservices, we need to embed in the design, the fact that one or more components could fail and degrade the functionality according to the availability of the software.
Let's take a look at the following diagram:
This diagram represents the sequence of execution on a monolithic software. A sequential list of calls that, no matter what, are going to be executed following the ACID principle: either all the calls (transactions) succeed or none of them do.
This is only possible as the framework and database engine designers have developed the concept of transactions to guarantee the transactionality of the data.
When working with microservices, we need to account for what the engineers call eventual consistency. After a partial fail on a transaction, each microservice instance should store the information required to recover the transaction so that the information will be eventually consistent. Following the previous example, if we send the book without processing the payment, the payment gateway will have a failed transaction that someone will process later on, making the data consistent again.
The best way to solve this problem is decentralizing the governance. Every endpoint should be able to take a local decision that affects the global scope of the transaction. We will talk more about this subject in Chapter 3, From the Monolith to Microservices.
Technology alignment
When building a new software, there is always a concept that every developer should keep in mind: standards.
Standards guarantee that your service will be technologically independent so that it will be easy to build the integrations using a different programming language or technologies.
One of the advantages of modeling a system with microservices is that we can choose the right technology for the right job so that we can be quite efficient when tackling problems. When building monolithic software, it is fairly hard to combine technologies like we can do with microservices. Usually, in a monolithic software, we are tied to the technology that we choose in the beginning.
Java Remote Method Invocation (RMI) is one example of the non-standard protocols that should be avoided if you want your system to be open to new technologies. It is a great way of connecting software components written in Java, but the developers will struggle (if not fail) to invoke an RMI method using Node.js. This will tie our architecture to a given language, which from the microservices point of view, will kill one of the most interesting advantages: technology heterogeneity.
How small is too small?
Once we have decided to model our system as a set of microservices, there is always one question that needs an answer: how small is too small?
The answer is always tricky and probably disappointing: it depends.
The right size of the microservices in a given system depends on the structure of the company as well as the ability to create software components that are easily manageable by a small team of developers. It also depends on the technical needs.
Imagine a system that receives and processes banking files; as you are probably aware, all the payments between banks are sent in files with a specific known format (such as Single Euro Payments Area (SEPA) for Euro payments). One of the particularities of this type of systems is the large number of different files that the system needs to know how to process.
The first approach for this problem is tackling it from the microservices point of view, separating it from any other service creating a unit of work, and creating one microservice for each type of file. It will enable us to be able to rollout modifications for the existing file processors without interfering with the rest of the system. It will also enable us to keep processing files even though one of the services is experiencing problems.
The microservices should be as small as needed, but keep in mind that every microservice adds an overhead to the operations team that needs to manage a new service. Try to answer the question how small is too small? in terms of manageability, scalability, and specialization. The microservice should be small enough to be managed and scaled up (or down) quickly without affecting the rest of the system, by a single person; and it should do only one thing.
Tip
As a general rule, a microservice should be small enough to be completely rewritten in a sprint.
Key benefits
In the previous topic, we talked about what a microservices-oriented architecture is. I also exposed the design principles that I have learned from experience, as well as showed a few benefits of this type of architecture.
Now, it is time to outline these key benefits and show how they will help us to improve the quality of our software, as well as be able to quickly accommodate the new business requirements.
Resilience
Resilience is defined in Wikipedia as the ability of a system to cope with change. I like to think about resilience as the ability of a system to gracefully recover from an exception
