41,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Discover practical techniques to build cloud-native apps that are scalable, reliable, and always available.
About This Book
- Build well-designed and secure microservices. Enrich your microservices with continous integration and monitoring.
- Containerize your application with Docker
- Deploy your application to AWS. Learn how to utilize the powerful AWS services from within your application
Who This Book Is For
This book is for developers who want to begin building secure, resilient, robust, and scalable Go applications that are cloud native. Some knowledge of the Go programming language should be sufficient.To build the front-end application, you will also need some knowledge of JavaScript programming.
What You Will Learn
- Understand modern software applications architectures
- Build secure microservices that can effectively communicate with other services
- Get to know about event-driven architectures by diving into message queues such as Kafka, Rabbitmq, and AWS SQS.
- Understand key modern database technologies such as MongoDB, and Amazon's DynamoDB
- Leverage the power of containers
- Explore Amazon cloud services fundamentals
- Know how to utilize the power of the Go language to access key services in the Amazon cloud such as S3, SQS, DynamoDB and more.
- Build front-end applications using ReactJS with Go
- Implement CD for modern applications
In Detail
Cloud computing and microservices are two very important concepts in modern software architecture. They represent key skills that ambitious software engineers need to acquire in order to design and build software applications capable of performing and scaling. Go is a modern cross-platform programming language that is very powerful yet simple; it is an excellent choice for microservices and cloud applications. Go is gaining more and more popularity, and becoming a very attractive skill..
The book will take you on a journey into the world of microservices and cloud computing with the help of Go. It will start by covering the software architectural patterns of cloud applications, as well as practical concepts regarding how to scale, distribute, and deploy those applications. You will also learn how to build a JavaScript-based front-end for your application, using TypeScript and React.
From there, we dive into commercial cloud offerings by covering AWS. Finally, we conclude our book by providing some overviews of other concepts and technologies that the reader can explore to move from where the book leaves off.
Style and approach
This book provides practical techniques, code examples, and architectural patterns required to build cloud native microservices in the Go language. It progresses through the subjects in a step-by-step manner. In each step, we cover the technical concepts behind it, the design considerations, and the code needed to build it.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 444
Veröffentlichungsjahr: 2017
Ähnliche
BIRMINGHAM - MUMBAI
Cloud Native programming with Golang
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: December 2017
Production reference: 1261217
ISBN 978-1-78712-598-8
www.packtpub.com
Credits
Authors
Mina Andrawos
Martin Helmich
Copy Editor
Dhanya Baburaj
Reviewer
Jelmer Snoeck
Project Coordinator
Sheejal Shah
Commissioning Editor
Aaron Lazar
Proofreader
Safis Editing
Acquisition Editor
Nitin Dasan
Indexer
Francy Puthiry
Content Development Editor
Sreeja Nair
Graphics
Jason Monteiro
Technical Editor
Prashant Mishra
Production Coordinator
Arvindkumar Gupta
About the Authors
Mina Andrawos is an experienced engineer who has developed deep experience in Go from using it personally and professionally. He regularly authors articles and tutorials about the language, and also shares Go's open source projects. He has written numerous Go applications with varying degrees of complexity. Other than Go, he has skills in Java, C#, Python, and C++. He has worked with various databases and software architectures. He is also skilled with the agile methodology for software development. Besides software development, he has working experience of scrum mastering, sales engineering, and software product management.
Martin Helmich studied computer science at the University of Applied Sciences in Osnabrück and lives in Rahden, Germany. He works as a software architect, specializing in building distributed applications using web technologies and Microservice Architectures. Besides programming in Go, PHP, Python, and Node.js, he also builds infrastructures using configuration management tools such as SaltStack and container technologies such as Docker and Kubernetes.
He is an Open Source enthusiast and likes to make fun of people who are not using Linux. In his free time, you'll probably find him coding on one of his open source pet projects, listening to music, or reading science-fiction literature.
About the Reviewer
Jelmer Snoeck is a software engineer with a focus on performance, reliability, and scaling. He's very passionate about open source and maintains several open source projects. Jelmer comes from a Ruby background and has been working with the Go language since 2014. He's taken a special interest in containers and Kubernetes, and is currently working on several projects to help with the deployment flow for these tools. Jelmer understands how to operate and scale distributed systems and is excited to share his experience with the world.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/178712598X.
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
Modern Microservice Architectures
Why Go?
Basic design goals
Cloud service models
Cloud application architecture patterns
The twelve-factor app
What are microservices?
Deploying microservices
REST web services and asynchronous messaging
The MyEvents platform
Summary
Building Microservices Using Rest APIs
The background
So, what are microservices?
Microservices internals
RESTful Web APIs
Gorilla web toolkit
Implementing a Restful API
Persistence layer
MongoDB
MongoDB and the Go language
Implementing our RESTful APIs handler functions
Summary
Securing Microservices
HTTPS
Symmetric cryptography
Symmetric-key algorithms in HTTPS
Asymmetric cryptography
Asymmetrical cryptography in HTTPS
Secure web services in Go
Obtaining a certificate
OpenSSL
generate_cert.go
Building an HTTPS server in Go
Summary
Asynchronous Microservice Architectures Using Message Queues
The publish/subscribe pattern
Introducing the booking service
Event collaboration
Implementing publish/subscribe with RabbitMQ
The Advanced Message Queueing Protocol
RabbitMQ quickstart with Docker
Advanced RabbitMQ setups
Connecting RabbitMQ with Go
Publishing and subscribing to AMQP messages
Building an event emitter
Building an event subscriber
Building the booking service
Event sourcing
Implementing publish/subscribe and event sourcing with Apache Kafka
Kafka quickstart with Docker
Basic principles of Apache Kafka
Connecting to Kafka with Go
Publishing messages with Kafka
Consuming messages from Kafka
Summary
Building a Frontend with React
Getting started with React
Setting up Node.js and TypeScript
Initializing the React project
Basic React principles
Kick-starting the MyEvents frontend
Implementing the event list
Bringing your own client
Building the event list components
Enabling CORS in the backend services
Testing the event list
Adding routing and navigation
Implementing the booking process
Summary
Deploying Your Application in Containers
What are containers?
Introduction to Docker
Running simple containers
Building your own images
Networking containers
Working with volumes
Building containers
Building containers for the backend services
Using static compilation for smaller images
Building containers for the frontend
Deploying your application with Docker Compose
Publishing your images
Deploying your application to the cloud
Introduction to Kubernetes
Setting up a local Kubernetes with Minikube
Core concepts of Kubernetes
Services
Persistent volumes
Deploying MyEvents to Kubernetes
Creating the RabbitMQ broker
Creating the MongoDB containers
Making images available to Kubernetes
Deploying the MyEvents components
Configuring HTTP Ingress
Summary
AWS I – Fundamentals, AWS SDK for Go, and EC2
AWS fundamentals
The AWS console
AWS command-line interface (CLI)
AWS regions and zones
AWS tags
AWS Elastic Beanstalk
AWS services
AWS SDK for Go
Configuring the AWS region
Configuring AWS SDK authentication
Creating IAM Users
Creating IAM Roles
The fundamentals of the AWS SDK for Go
Sessions
Service clients
Native datatypes
Shared configuration
Pagination methods
Waiters
Handling Errors
Elastic Compute Cloud (EC2)
Creating EC2 instances
Accessing EC2 instances
Accessing EC2 instances from a Linux or macOS machine
Accessing EC2 from Windows
Security groups
Summary
AWS II–S3, SQS, API Gateway, and DynamoDB
Simple Storage Service (S3)
Configuring S3
Simple Queue Service (SQS)
AWS API gateway
DynamoDB
DynamoDB components
Attribute value data types
Primary keys
Secondary indexes
Creating tables
The Go language and DynamoDB
Summary
Continuous Delivery
Setting up your project
Setting up version control
Vendoring your dependencies
Using Travis CI
Deploying to Kubernetes
Using GitLab
Setting up GitLab
Setting up GitLab CI
Summary
Monitoring Your Application
Setting up Prometheus and Grafana
Prometheus's basics
Creating an initial Prometheus configuration file
Running Prometheus on Docker
Running Grafana on Docker
Exporting metrics
Using the Prometheus client in your Go application
Configuring Prometheus scrape targets
Exporting custom metrics
Running Prometheus on Kubernetes
Summary
Migration
What is a monolithic application?
What are microservices?
Migrating from monolithic applications to microservices
Humans and technology
Cutting a monolithic application to pieces
How do we break the code?
Glue code
Microservices design patterns
Sacrificial architecture
A four-tier engagement platform
Bounded contexts in domain-driven designs
Data consistency
Event-driven architecture for data consistency
Events sourcing
CQRS
Summary
Where to Go from Here?
Microservices communications
Protocol buffers
GRPC
More on AWS
DynamoDB streams
Autoscaling on AWS
Amazon Relational Database Service
Other cloud providers
Microsoft Azure
Google Cloud Platform
OpenStack
Running containers in the cloud
Serverless architectures
Summary
Preface
Cloud computing and microservices are two very important concepts in modern software architecture. They represent key skills that ambitious software engineers need to acquire in order to design and build software applications capable of performing and scaling. Go is a modern cross-platform programming language that is very powerful yet simple; it is an excellent choice for microservices and cloud applications. Go is gaining increasing popularity and becoming a very attractive skill.
The book will take you on a journey into the world of microservices and cloud computing with the help of Go. It will start by covering the software architectural patterns of cloud applications as well as practical concepts regarding how to scale, distribute, and deploy those applications. From there, the book will dive deep into the techniques and design approaches that are needed for writing production-level microservices and their deployment into typical cloud environments.
After completing this book, you will have learned how to write effective production-grade microservices that are deployable to the cloud, practically understand the world of Amazon Web Services, and know how to build non-trivial Go applications.
What this book covers
Chapter 1, Modern Microservice Architectures, opens the book by describing typical features of cloud-based applications and microservice architectures. We will also establish requirements and a high-level architecture for a fictional application that will serve as a continuous example over the following chapters of this book.
Chapter 2, Building Microservices Using REST APIs, discusses how to build modern microservices with the Go language. We will cover important and non-trivial topics. By the end of this chapter, you will have enough knowledge to build microservices that can expose RESTFul APIs, support persistence, and can effectively communicate with other services.
Chapter 3, Securing Microservices, shows you how to secure your microservices. You will get to learn about how to handle certificates and HTTPS in the Go language.
Chapter 4, Asynchronous Microservice Architectures, presents how to implement an asynchronous microservice architecture using message queues. For this, we will give an overview on established message queuing software, such as RabbitMQ and Apache Kafka, and present Go libraries to integrate these components into your software. We will also discuss architectural patterns such as Event Collaboration and Event Sourcing that work well together with asynchronous architectures.
Chapter 5, Building a Frontend with React, takes a small detour from the Go world into the JavaScript world and shows you how to build a web frontend for the microservice-based project using the React framework. For this, we will give a short overview over the basic architectural principles of React and how to build a React-based frontend for existing REST APIs.
Chapter 6, Deploying Your Application in Containers, shows how to deploy Go applications in a portable and reproducible way using application containers. You will learn to install and using Docker and how to build custom Docker images for your own Go applications. Furthermore, we will describe how to use the Kubernetes orchestration engine to deploy containerized applications in large-scale cloud environments.
Chapter 7, AWS – Fundamentals, AWS SDK for Go and AWS EC2, is the first of two chapters to cover the AWS ecosystem. In this chapter, we will cover AWS in practical details. You will get exposed to several important concepts like how to setup AWS server instances , how to utilize the AWS API features, and how to write Go applications that are capable of interacting with AWS.
Chapter 8, AWS – S3, SQS, API Gateway, and DynamoDB, continues to cover the AWS ecosystem in more detail. You will dive deeper into popular services in the AWS world. By the end of this chapter, you will have enough knowledge to build non-trivial Go cloud applications using the powers of Amazon Web Services.
Chapter 9, Continuous Delivery, describes how to implement a basic Continuous Delivery pipeline for your Go applications. For this, we will describe the basic principles of CD and how to implement a simple pipeline using tools such as Travis CI and Gitlab. We will use Docker images as deployment artifacts and deploy these images into a Kubernetes cluster, thus building on the topics and skills covered in Chapter 4, Asynchronous Microservice Architectures.
Chapter 10, Monitoring Your Application, shows you how to monitor your microservice architecture using Prometheus and Grafana. We will cover the basic architecture of Prometheus and describe how to set up a Prometheus instance using Docker. Also, you will learn how to adjust your Go applications to expose metrics that can be scraped by Prometheus. We will also describe how to set up a graphical user interface for Prometheus using Grafana.
Chapter 11, Migration, covers practical factors and approaches to consider when migrating from legacy monolithic applications into modern microservices cloud-ready applications.
Chapter 12, Where to Go from Here?, shows you where to continue the learning journey from here. It will cover other modern cloud-related technologies that deserve to be explored, such as alternative communication protocols, other cloud providers, and new architectural paradigms that might be the next big thing.
What you need for this book
For this book, you should have some basic knowledge of the Go programming language (if you're still looking to get started with Go, we can recommend the book Learning Go Programming by Vladimir Vivien, also published by Packt). To run the code examples provided in this book, you will also need a working Go SDK on your local machine (Go 1.7 or newer). Head to https://golang.org/dl/ for download and installation instructions.
For many of the practical examples in the book, you will need a working Docker installation (although previous experience in working with Docker is not required). Take a look at https://www.docker.com/community-edition for download and installation instructions.
For working with React in Chapter 5, Building a Frontend With React, you will also need some basic knowledge in JavaScript programming and a working Node.JS installation on your local machine. You can download the current release of Node.JS from https://nodejs.org/en/#download.
Who this book is for
This book is targeted at Go developers who want to build secure, resilient, robust, and scalable applications that are cloud native. Some knowledge of web services and web programming should be sufficient to get you through the book.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The react-router-dom package adds a few new components to our application."
A block of code is set as follows:
import * as React from "react"; import {Link} from "react-router-dom"; export interface NavigationProps { brandName: string; } export class Navigation extends React.Component<NavigationProps, {}> { }
Any command-line input or output is written as follows:
$ npm install --save react-router-dom $ npm install --save-dev @types/react-router-dom
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "For this, click on the Create Repository after logging in and choose a new name for your image."
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors .
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
Log in or register to our website using your e-mail address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Cloud-Native-Programming-with-Golang. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/CloudNativeprogrammingwithGolang_ColorImages.pdf.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Modern Microservice Architectures
In the world of computing and software, we hear about many cool new technologies and frameworks almost every week. Some of them stay and persist, whereas others fail the test of time and disappear. Needless to say, cloud computing sits very comfortably in the former category. We live in a world where cloud computing powers almost everything that needs serious backend computing power, from Internet of Things (IOT) devices that check the temperature on a refrigerator to video games that show you real-time stats for your scores compared to your peers in multiplayer games.
Cloud computing benefits huge enterprises with offices all over the world, as well as minimal start-ups of two people writing code in a coffee shop. There is tons of material that cover why cloud computing is so important for modern information technologies. For the sake of efficiency, we'll provide a straightforward answer to this question, without going into long bullet points, graphs, and lengthy paragraphs. For businesses, it's all about making money and saving costs. Cloud computing drives costs down significantly for most organizations. That's because cloud computing saves you the cost of building your own data center. No expensive hardware needs to be bought, and no expensive buildings with fancy air-conditioning systems need to be commissioned. Additionally, almost all cloud computing offerings give you the ability to pay for only what you use and no more. Cloud computing also offers massive flexibility for software engineers and IT administrators to do their jobs quickly and efficiently, thus achieving developer happiness and increased productivity.
In this chapter, we will cover the following topics:
Design goals of cloud-native applications, especially scalability
Different cloud service models
The twelve-factor app
Microservice architectures
Communication patterns, especially synchronous versus asynchronous communication
Why Go?
Go (or Golang) is a relatively new programming language that is taking the software development world by storm. It was developed by Google to facilitate the construction of its backend software services. However, it's now being used by numerous enterprises and start-ups to write powerful applications. What sets Go apart is the fact that it was built from the ground up to provide performance that is destined to compete with very powerful languages, such as C/C++, while supporting a relatively simple syntax that resembles dynamic languages such as JavaScript. The Go runtime offers garbage collection; however, it does not rely on virtual machines to achieve that. Go programs are compiled into native machine code. When invoking the Go compiler, you simply choose the type of platform (Windows, Mac, and so on) that you'd like the binary to run on when you build. The compiler will then produce a single binary that works on that platform. This makes Go capable of cross-compiling and producing native binaries.
Go is perfect for microservice architectures, which we will be seeing a lot of in the future. A microservice architecture is an architecture where you divide the responsibilities of your application between smaller services that only focus on specific tasks. These services can then communicate among themselves to obtain the information they need to produce results.
Go is a fresh programming language, developed in the age of cloud computing, and with modern software technologies in mind. Go is optimized for portable microservice architectures due to the fact that a Go program mostly compiles to a single binary, making the need for dependencies and virtual machines in production environments almost non-existent. Go is also a pioneer in container technologies. Docker, the top name in software containers, is written in none other than Go. Due to Go's popularity, there is work being done by major cloud providers, as well as third-party contributors, to ensure that Go gets the API support it needs for different cloud platforms.
The goal of this book is to build the knowledge bridge between the Go programming language and the cloud technologies of modern computing. In this book, you will gain practical knowledge of Go microservice architectures, message queues, containers, cloud platform Go APIs, SaaS applications design, monitoring cloud applications, and more.
Basic design goals
In order to fully benefit from the advantages of modern cloud platforms, we need to consider their characteristic properties when developing applications that should run on these platforms.
One of the main design goals of cloud applications is scalability. On the one hand, this means growing your application's resources as needed in order to efficiently serve all your users. On the other hand, it also means shrinking your resources back to an appropriate level when you do not need them anymore. This allows you to run your application in a cost-efficient manner without having to constantly overprovision for peak workloads.
In order to achieve this, typical cloud deployments often use small virtual machine instances that host an application and scale by adding (or removing) more of these instances. This method of scaling is called horizontal scaling or scale out—as opposed to vertical scaling or scale up, where you would not increase the number of instances, but provision more resources to your existing instances. Horizontal scaling is often preferred to vertical scaling for several reasons. First, horizontal scaling promises unlimited linear scalability. On the other hand, vertical scaling has its limits due to the fact that the number of resources that you can add to an existing server cannot grow infinitely. Secondly, horizontal scaling is often more cost-efficient since you can use cheap commodity hardware (or, in cloud environments, smaller instance types), whereas larger servers often grow exponentially more expensive.
All major cloud providers offer the ability to perform horizontal scaling automatically, depending on your application's current resource utilization. This feature is called auto-scaling. Unfortunately, you do not get horizontal scalability for free. In order to be able to scale out, your application needs to adhere to some very important design goals that often need to be considered from the start, as follows:
Statelessness
: Each instance of a cloud application should not have any kind of internal state (meaning that any kind of data is saved for later use, either in-memory or on the filesystem). In a scale-out scenario, subsequent requests might be served by another instance of the application and, for this reason, must not rely on any kind of state being present from previous requests. In order to achieve this, it is usually necessary to externalize any kind of persistent storage, such as databases and filesystems. Both database services and file storage are often offered as managed services by the cloud provider that you use in your application.
Ease of deployment
: When scaling out, you will need to deploy new instances of your application quickly. Creating a new instance should not require any kind of manual setup, but should be automated as much as possible (ideally completely).
Resiliency:
In a cloud environment, especially when using auto-scaling, instances may be shut down at a moment's notice. Also, most cloud providers do not guarantee an extremely high availability on individual instances (and suggest scaling out instead, optionally across multiple availability zones). For this reason, termination and sudden death (either intentionally, in case of auto-scaling, or unintentionally, in case of failure) is something that we always need to expect in a cloud environment, and the application must handle it accordingly.
Achieving these design goals is not always easy. Cloud providers often support you in this task by offering managed services (for example, highly scalable database services of distributed file storage) that otherwise you would have to worry about yourself. Concerning your actual application, there is the twelve-factor app methodology (which we will cover in more detail in a later section), which describes a set of rules for building scalable and resilient applications.
Cloud service models
When it comes to cloud computing offerings, there are three main service models to consider for your project:
IaaS
(
Infrastructure as a Service
): This is the model where the cloud service provider gives you access to infrastructure on the cloud, such as servers (virtual and bare metal), networks, firewalls, and storage devices. You use IaaS when all that you need is for the cloud provider to manage the infrastructure for you and take the hassle and the cost of maintaining it out of your hands. IaaS is used by start-ups and organizations that want full control over their application's layer. Most IaaS offerings come with a dynamic or elastic scaling option, which would scale your infrastructure based on your consumption. This, in effect, saves organizations costs since they
only
pay for what they use.
PaaS
(
Platform as a Service
)
: This
is the next layer up from IaaS. PaaS provides the computing platform you need to run your application. PaaS typically includes the operating systems you need to develop your applications, the databases, the web layer (if needed), and the programming language execution environment. With PaaS, you don't have to worry about updates and patches for your application environment; it gets taken care of by the cloud provider. Let's say you wrote a powerful .NET application that you want to see running in the cloud. A PaaS solution will provide the .NET environment you need to run your application, combined with the Windows server operating systems and the IIS web servers. It will
also
take care of load-balancing and scale for larger applications. Imagine the amount of money and effort you could save by adopting a PaaS platform instead of doing the effort in-house.
SaaS
(
Software as a Service
)
: This
is the highest layer offering you can obtain as a cloud solution. A SaaS solution is when a fully functional piece of software is delivered over the web. You access SaaS solutions from a web browser. SaaS solutions are typically used by regular users of the software, as opposed to programmers or software professionals. A very famous example of a SaaS platform is Netflix—a complex piece of software hosted in the cloud, which is available to you via the web. Another popular example is Salesforce. Salesforce solutions get delivered to customers through web browsers with speed and efficiency.
Cloud application architecture patterns
Usually, developing applications that run in a cloud environment is not that different from regular application development. However, there are a few architectural patterns that are particularly common when targeting a cloud environment, which you will learn in the following section.
The twelve-factor app
The twelve-factor app methodology is a set of rules for building scalable and resilient cloud applications. It was published by Heroku, one of the dominant PaaS providers. However, it can be applied to all kinds of cloud applications, independent of concrete infrastructure or platform providers. It is also independent of programming languages and persistence services and can equally be applied to Go programming and, for example, Node.js programming. The twelve-factor app methodology describes (unsurprisingly) twelve factors that you should consider in your application for it to be easily scalable, resilient, and platform independent. You can read up on the full description on each factor on https://12factor.net. For the purpose of this book, we will highlight some factors that we deem especially important:
Factor II: Dependencies—Explicitly declare and isolate dependencies
:
This factor deserves special mention because it is actually not as important in Go programming as in other languages. Typically, a cloud application should never rely on any required library or external tool being already present on a system. Dependencies should be explicitly declared (for example, using an npm
package.json
file for a Node.js application) so that a package manager can pull all these dependencies when deploying a new instance of the application. In Go, an application is typically deployed as a statically compiled binary that already contains all required libraries. However, even a Go application can be dependent on external system tools (for example, it can fork out to tools such as
ImageMagick
) or on existing C libraries. Ideally, you should deploy tools like these alongside your application. This is where container engines, such as Docker, shine.
Factor III: Config—Store config in the environment
:
Configuration is any kind of data that might vary for different deployment, for example, connection data and credentials for external services and databases. These kinds of data should be passed to the application via environment variables.
In a Go application, retrieving these is then as easy as calling
os.Getenv ("VARIABLE_NAME")
. In more complex cases (for example, when you have many configuration variables), you can also resort to libraries such as
github.com/tomazk/envcfg
or
github.com/caarlos0/env
. For heavy lifting, you can use the
github.com/spf13/viper
library.
Factor IV: Backing Services—Treat backing services as attached resources
: Ensure that services that your app depends on (such as databases, messaging systems, or external APIs) are easily swappable by configuration. For example, your app could accept an environment variable, such as
DATABASE_URL
, that might contain
mysql://root:root@localhost/test
for a local development deployment and
mysql://root:[email protected]
in your production setup.
Factor VI:
Processes—
Execute the app as one or more stateless processes
: Running application instances should be stateless; any kind of data that should persist beyond a single request/transaction needs to be stored in an external persistence service.One important case to keep in mind is user sessions in web applications. Often, user session data is stored in the process's memory (or is persisted to the local filesystem) in the expectancy that subsequent requests of the same user will be served by the same instance of your application. Instead, try to keep user sessions stateless or move the session state into an external data store, such as
Redis
or
Memcached
.
Factor IX: Disposability—Maximize robustness with fast startup and graceful shutdown
: In a cloud environment, sudden termination (both intentional, for example, in case of downscaling, and unintentional, in case of failures) needs to be expected. A twelve-factor app should have fast startup times (typically in the range of a few seconds), allowing it to rapidly deploy new instances. Besides, fast startup and graceful termination is another requirement. When a server shut down, the operating system will typically tell your application to shut down by sending a
SIGTERM
signal that the application can catch and react to accordingly (for example, by stopping to listen on the service port, finishing requests that are currently being processed, and then exiting).
Factor XI: Logs—Treat logs as event streams:
Log data is often useful for debugging and monitoring your application's behavior. However, a twelve-factor app should not concern itself with the routing or storage of its own log data. The easiest and simplest solution is to simply write your log stream to the process's standard output stream (for example, just using
fmt.Println(...)
). Streaming events to
stdout
allows a developer to simply watch the event stream on their console when developing the application. In production setups, you can configure the execution environment to catch the process output and send the log stream to a place where it can be processed (the possibilities here are endless—you could store them in your server's
journald
, send them to a syslog server, store your logs in an ELK setup, or send them to an external cloud service).
What are microservices?
When an application is maintained by many different developers over a longer period of time, it tends to get more and more complex. Bug fixes, new or changing requirements, and constant technological changes result in your software continually growing and changing. When left unchecked, this software evolution will lead to your application getting more complex and increasingly difficult to maintain.
Preventing this kind of software erosion is the objective of the microservice architecture paradigm that has emerged over the past few years. In a microservice architecture, a software system is split into a set of (potentially a lot of) independent and isolated services. These run as separate processes and communicate using network protocols (of course, each of these services should in itself be a twelve-factor app). For a more thorough introduction to the topic, we can recommend the original article on the microservice architecture by Lewis and Fowler at https://martinfowler.com/articles/microservices.html.
In contrast to traditional Service-Oriented Architectures (SOA), which have been around for quite a while, microservice architectures focus on simplicity. Complex infrastructure components such as ESBs are avoided at all costs, and instead of complicated communication protocols such as SOAP, simpler means of communication such as REST web services (about which you will learn more in Chapter 2, Building Microservices Using Rest APIs) or AMQP messaging (refer to Chapter 4, Asynchronous Microservice Architectures Using Message Queues) are preferred.
Splitting complex software into separate components has several benefits. For instance, different services can be built on different technology stacks. For one service, using Go as runtime and MongoDB as persistence layer may be the optimal choice, whereas a Node.js runtime with a MySQL persistence might be a better choice for other components. Encapsulating functionality in separate services allows developer teams to choose the right tool for the right job. Other advantages of microservices on an organizational level are that each microservice can be owned by different teams within an organization. Each team can develop, deploy, and operate their services independently, allowing them to adjust their software in a very flexible way.
Deploying microservices
With their focus on statelessness and horizontal scaling, microservices work well with modern cloud environments. Nevertheless, when choosing a microservice architecture, deploying your application will tend to get more complex overall, as you will need to deploy more, different applications (all the more reason to stick with the twelve-factor app methodology).
However, each individual service will be easier to deploy than a big monolithic application. Depending on the service's size, it will also be easier to upgrade a service to a new runtime or to replace it with a new implementation entirely. Also, you can scale each microservice individually. This allows you to scale out heavily used parts of your application while keeping less utilized components cost-efficient. Of course, this requires each service to support horizontal scaling.
Deploying microservices gets (potentially) more complex when different services use different technologies. A possible solution for this problem is offered by modern container runtimes such as Docker or RKT. Using containers, you can package an application with all its dependencies into a container image and then use that image to quickly spawn a container running your application on any server that can run Docker (or RKT) containers. (Let's return to the twelve-factor app—deploying applications in containers is one of the most thorough interpretations of dependency isolation as prescribed by Factor II.)
Running container workloads is a service offered by many major cloud providers (such as AWS'Elastic Container Service, the Azure Container Service, or the Google Container Engine). Apart from that, there are also container orchestration engines such as Docker Swarm, Kubernetes, or Apache Mesosthat you can roll out on IaaS cloud platforms or your own hardware. These orchestration engines offer the possibility to distribute container workloads over entire server clusters, and offer a very high degree of automation. For example, the cluster manager will take care of deploying containers across any number of servers, automatically distributing them according to their resource requirements and usages. Many orchestration engines also offer auto-scaling features and are often tightly integrated with cloud environments.
You will learn more about deploying microservices with Docker and Kubernetes in Chapter 6, Deploying Your Application in Containers.
REST web services and asynchronous messaging
When building a microservice architecture, your individual services need to communicate with one another. One widely accepted de facto standard for microservice communication is RESTful web services (about which you will learn more in Chapter 2, Building Microservices Using Rest APIs, and Chapter 3, Securing Microservices). These are usually built on top of HTTP (although the REST architectural style itself is more or less protocol independent) and follow the client/server model with a request/reply communication model.
This architecture is typically easy to implement and to maintain. It works well for many use cases. However, the synchronous request/reply pattern may hit its limits when you are implementing a system with complex processes that span many services. Consider the first part of the preceding diagram. Here, we have a user service that manages an application's user database. Whenever a new user is created, we will need to make sure that other services in the system are also made aware of this new user. Using RESTful HTTP, the user service needs to notify these other services by REST calls. This means that the user service needs to know all other services that are in some way affected by the user management domain. This leads to a tight coupling between the components, which is something you'd generally like to avoid.
An alternative communication pattern that can solve these issues is the publish/subscribe pattern. Here, services emit events that other services can listen on. The service emitting the event does not need to know which other services are actually listening to these events. Again, consider the second part of the preceding diagram—here, the user service publishes an event stating that a new user has just been created. Other services can now subscribe to this event and are notified whenever a new user has been created. These architectures usually require the use of a special infrastructure component: the message broker. This component accepts published messages and routes them to their subscribers (typically using a queue as intermediate storage).
The publish/subscribe pattern is a very good method to decouple services from one another—when a service publishes events, it does not need to concern itself with where they will go, and when another service subscribes to events, it also does not know where they came from. Furthermore, asynchronous architectures tend to scale better than ones with synchronous communication. Horizontal scaling and load balancing are easily accomplished by distributing messages to multiple subscribers.
Unfortunately, there is no such thing as a free lunch; this flexibility and scalability are paid for with additional complexity. Also, it becomes hard to debug single transactions across multiple services. Whether this trade-off is acceptable for you needs to be assessed on a case-by-case basis.
In Chapter 4, Asynchronous Microservice Architectures Using Message Queues, you will learn more about asynchronous communication patterns and message brokers.
The MyEvents platform
Throughout this book, we will build a useful SaaS application called MyEvents. MyEvents will utilize the technologies that you'll be learning in order to become a modern, scalable, cloud-native, and snappy application. MyEvents is an event management platform that allows users to book tickets for events all over the world. With MyEvents, you will be able to book tickets for yourself and your peers for concerts, carnivals, circuses, and more. MyEvents will keep a record of the bookings, the users, and the different locations where the events are taking place. It will manage your reservations efficiently.
We will make use of microservices, message queues, ReactJS, MongoDB, AWS, and more to construct MyEvents. In order to understand the application better, let's take a look at the logical entities that our overall application will be managing. They will be managed by multiple microservices in order to establish a clear separation of concerns and to achieve the flexibility and scalability that we need:
We will have multiple users; each User can have multiple bookings for events, and each Booking will correspond to a single Event. For each one of our events, there will be a Location where the event is taking place. Inside the Location, we will need to identify the Hall or room where the event is taking place.
Now, let's take a look at the microservice architecture and the different components that make our application:
We will use a ReactJS frontend to interface with the users of our applications. The ReactJS UI will use an API gateway (AWS or local) to communicate with the different microservices that form the body of our application. There are two main microservices that represent the logic of MyEvents:
Event Service
: This is the service that handles the events, their locations, and changes that happen to them
Booking Service
: This service handles bookings made by users
All our services will be integrated using a publish/subscribe architecture based on message queues. Since we aim to provide you with practical knowledge in the world of microservices and cloud computing, we will support multiple types of message queues. We will support Kafka, RabbitMQ, and SQS from AWS.
The persistence layer will support multiple database technologies as well, in order to expose you to various practical database engines that empower your projects. We will support MongoDB, and DynamoDB</span>.
All of our services will support metrics APIs, which will allow us to monitor the statistics of our services via Prometheus.
The MyEvents platform is designed in a way that will build strong foundations of knowledge and exposure to the powerful world of microservices and cloud computing.
Summary
In this introductory chapter, you learned about the basic design principles of cloud-native application development. This includes design goals, such as supporting (horizontal) scalability and resilience, and also architectural patterns, such as the twelve-factor app and microservice architectures.
Over the course of the following chapters, you will learn to apply many of these principles while building the MyEvents application. In Chapter 2, Building Microservices Using Rest APIs, you will learn how to implement a small microservice that offers a RESTful web service using the Go programming language. In the next chapters, you will continue to extend this small application and learn how to handle the deployment and operate on of this application in various cloud environments.
Building Microservices Using Rest APIs
In this chapter, we'll go on a journey to learn about the world of microservices. We'll learn about how they are structured, how they communicate, and how they persist data. The concept of microservices is a key concept to cover due to the fact that most of the modern cloud applications in production today rely on microservices to achieve resiliency and scalability.
In this chapter, we will cover the following topics :
Deep diving into the microservices architecture
RESTful web APIs
Building RESTful APIs in the Go language
The background
We provided a practical definition for microservices in the first chapter. In this chapter, let's define microservices a bit more.
To fully appreciate microservices, let's start by telling the story of their rise. Before the idea of microservices became popular, most applications used to be monolithic. A monolithic application is a single application that tries to get numerous tasks accomplished at once. Then, as new features are needed, the application will get bigger and bulkier. This, in effect, produced unmaintainable applications in the long run. With the emergence of cloud computing, and distributed applications with massive loads, the need for a more flexible application architecture became obvious.
In Chapter 1, Modern Microservice Architectures, we provided an introduction to the MyEvents application, which we will be expecting to build in this book. The MyEvents application is used to manage event bookings for concerts, plays, and so on. The main tasks for the application include the following:
Process bookings
: For example, a user makes a booking for a concert next month. We will need to store this reservation, ensure that there are seats available for this event, and confirm no prior reservations were made with the same name, among other things.
Handle events
: Our application needs to be aware of all the concerts, plays, and other types of events that we're expecting to support. We need to know the event addresses, the total number of seats, their duration, and so on.
Handle search
: Our application needs to be capable of performing efficient searches to retrieve our bookings and events.
The following image shows how a monolithic application design for MyEvents would look like:
We'll build multiple software layers within the application to handle each distinct task needed. Our application will become a program with a large code base. Since the code is all connected, there will always be a risk of change in one layer affecting code on the other layers.
Since it's a single program, it won't be easy to write some of the software layers in different programming languages. This is typically a very good option to have when you know there is a really good library in language X to support feature Y, however, language X is not good for feature Z.
Also, as you add new features or layers, your single program will keep growing with no good scalability options. Wouldn't it be better to be able to run different software layers on different servers so that you can control your application load without throwing more hardware on one or two servers?
Software engineers have tried to solve the monolithic application's dilemma for a long time. Microservices is one approach to address the issues that come with monolithic applications. Before the term microservices became popular, there was the concept of SOA, which was similar in principle to microservices.
Before we dive more into microservices, it is worth mentioning that monolithic applications are not always bad. It all depends on what you are trying to achieve. If you are trying to build an application that is expected to have a limited set of tasks, and not expected to grow by much, then a single well-built application might be all you need. If on the other hand, you are looking to build a complex application that is expected to perform numerous independent tasks, being maintained by multiple people, while handling massive data loads, then the microservices architecture is your friend.
So, what are microservices?
Simply put, microservices is the idea that instead of putting all of your code in one basket (monolithic application), you write multiple small software services or microservices. Each service is expected to focus on one task and perform it well. The accumulation of those services will make up your application.
For the MyEvents application, each software layer from the monolithic application will translate into a software service. This will then communicate together to form our application. Each one of those software services will be, in effect, a microservice.
Since those services collaborate to build a complex application, they need to be able to communicate via protocols that they all understand. Microservices that use web Restful APIs for communication make use of the HTTP protocol extensively. We'll cover Restful APIs in more detail in this chapter.
