Docker High Performance E-Book

Allan Espinosa

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 166

Veröffentlichungsjahr: 2019

Beschreibung

Leverage Docker to unlock efficient and rapid container deployments to improve your development workflow

Key Features:

Reconfigure Docker hosts to create a logging system with the ElasticSearch-Logstash-Kibana (ELK) stackTackle the challenges of large-scale container deployment with this fast-paced guideBenchmark the performance of your Docker containers using Apache JMeter

Book Description:

Docker is an enterprise-grade container platform that allows you to build and deploy your apps. Its portable format lets you run your code right from your desktop workstations to popular cloud computing providers. This comprehensive guide will improve your Docker workflows and ensure your application's production environment runs smoothly.

This book starts with a refresher on setting up and running Docker and details the basic setup for creating a Docker Swarm cluster. You will then learn how to automate this cluster by using Chef Server and Cookbook. After that, you will run the Docker monitoring system with Prometheus and Grafana, and deploy the ELK stack. You will also learn some tips for optimizing Docker images.

After deploying containers with the help of Jenkins, you will then move on to a tutorial on using Apache JMeter to analyze your application's performance. You will learn how to use Docker Swarm and NGINX to load-balance your application and how common debugging tools in Linux can be used to troubleshoot Docker containers.

By the end of this book, you will be able to integrate all the optimizations that you have learned and put everything into practice in your applications.

What you will learn:

Automate provisioning and setting up nodes in a Docker Swarm clusterConfigure a monitoring system with Prometheus and GrafanaUse Apache JMeter to create workloads for benchmarking the performance of Docker containersUnderstand how to load-balance an application with Docker Swarm and NginxDeploy strace, tcdump, blktrace, and other Linux debugging tools to troubleshoot containersIntegrate Docker optimizations for DevOps, Site Reliability Engineering, CI, and CD

Who this book is for:

If you are a software developer with a good understanding of managing Docker services and the Linux file system and are looking for ways to optimize working with Docker containers, then this is the book for you. Developers fascinated with containers and workflow automation with benefit from this book.

Allan Espinosa is a DevOps practitioner and an active open source contributor to various distributed system tools, such as Docker and Chef. Allan maintains several Docker images for popular open source software that were popular even before their official release from the upstream open source groups. Throughout his career, Allan has worked on large distributed systems containing hundreds to thousands of servers in production. He has built scalable applications on various platforms, ranging from large supercomputing centers to production clusters in the enterprise. He is currently managing distributed systems at scale for Bloomberg, where he oversees the company's Hadoop infrastructure. Allan can be contacted through his Twitter handle, @AllanEspinosa. Russ McKendrick is an experienced system administrator who has been working in IT and related industries for over 25 years. During his career, he has had varied responsibilities, from looking after an entire IT infrastructure to providing first-line, second-line, and senior support in both client-facing and internal teams for large organizations. Russ supports open source systems and tools on public and private clouds at N4Stack, a Node4 company, where he is the practice manager (SRE and DevOps). In his spare time, he has written several books including Mastering Docker, Learn Ansible and Kubernetes for Serverless Applications, all published by Packt Publishing.

Leseprobe

Docker High Performance

Second Edition

Complete your Docker journey by optimizing your application's workflows and performance

Allan Espinosa

Russ McKendrick

BIRMINGHAM - MUMBAI

Docker High Performance Second Edition

Copyright © 2019 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor:Vijin BorichaAcquisition Editor:Aditi GourContent Development Editor:Roshan KumarTechnical Editor:Adya AnandCopy Editor: Safis EditingProject Coordinator:Namrata SwettaProofreader: Safis EditingIndexer:Tejal Daruwale SoniGraphics:Jisha ChirayilProduction Coordinator:Jisha Chirayil

First published: January 2016 Second edition: April 2019

Production reference: 1300419

Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.

ISBN 978-1-78980-721-9

www.packtpub.com

 

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

Packt.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [e-mail] for more details.

At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. 

Contributors

About the authors

Allan Espinosa is a DevOps practitioner and an active open source contributor to various distributed system tools, such as Docker and Chef. Allan maintains several Docker images for popular open source software that were popular even before their official release from the upstream open source groups.

Throughout his career, Allan has worked on large distributed systems containing hundreds to thousands of servers in production. He has built scalable applications on various platforms, ranging from large supercomputing centers to production clusters in the enterprise. He is currently managing distributed systems at scale for Bloomberg, where he oversees the company's Hadoop infrastructure. Allan can be contacted through his Twitter handle, @AllanEspinosa. 

I would like to thank my wife, Kana, for the continuous support that allowed me to spend significant time with this project.

Russ McKendrick is an experienced system administrator who has been working in IT and related industries for over 25 years. During his career, he has had varied responsibilities, from looking after an entire IT infrastructure to providing first-line, second-line, and senior support in both client-facing and internal teams for large organizations.

Russ supports open source systems and tools on public and private clouds at N4Stack, a Node4 company, where he is the practice manager (SRE and DevOps). In his spare time, he has written several books including Mastering Docker, Learn Ansible and Kubernetes for Serverless Applications, all published by Packt Publishing.

About the reviewer

Shashikant Bangera is a lead DevOps architect working for a multinational IT service provider. He specializes in architecting automated, efficient delivery pipelines to help build software rapidly and effectively. He has extensive experience with the cloud, containers, and DevOps tools and process. He has worked across a variety of domains, and designed DevOps processes and automation for large-scale projects. He is an open source enthusiast and has multiple projects under his name on GitHub. He has a master's degree from Welingkar University. 

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Title Page

Copyright and Credits

Docker High Performance Second Edition

About Packt

Why subscribe?

Packt.com

Contributors

About the authors

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Preparing Docker Hosts

Preparing a Docker host

Enabling remote access

Setting up a certificate authority

Enabling remote access in Docker Engine

Connecting remotely from the Docker client

Building a Docker Swarm cluster

Summary

Configuring Docker with Chef

Importance of configuration management

Using Chef

Signing up for a Chef server

Setting up our workstation

Bootstrap nodes

Configuring the Docker host

Writing Chef recipes

Push Chef policies

Initializing Docker Swarm

Alternative methods

Summary

Monitoring Docker

The importance of monitoring

Collecting metrics with Prometheus

Exposing Prometheus's metrics

Scraping and visualizing metrics

Consolidating logs in an ELK stack

Deploying Elasticsearch, Logstash, and Kibana

Forwarding Docker container logs

Other monitoring and logging solutions

Summary

Optimizing Docker Images

Reducing deployment time

Improving image build time

Using registry mirrors

Reusing image layers

Reducing the build context size

Using caching proxies

Reducing Docker image size

Chaining commands

Separating build and deployment images

Guide to Optimization

Summary

Deploying Containers

Deploying and configuring Jenkins

Deploying the Jenkins container

Finishing the Jenkins Configuration

Setting up our Docker credentials within Jenkins

Building and deploying a container

Preparing our application

Creating a Jenkins job

Running the Pipeline

Summary

Benchmarking

Setting up Apache JMeter

Deploying a sample application

Installing JMeter

Building a benchmark workload

Creating a test plan in JMeter

Analyzing benchmark results

Viewing the results of JMeter runs

Calculating throughput

Plotting response time

Observing performance in Grafana and Kibana

Tuning the benchmark

Increasing concurrency

Running distributed tests

Other benchmarking tools

Summary

Load Balancing

Preparing application backends

Balancing load with NGINX

Scaling out our Docker applications

Deploying with zero downtime

Other load balancers

Summary

Troubleshooting Containers

Inspecting containers with the docker exec command

Debugging from outside Docker

Tracing system calls

Analyzing network packets

Observing block devices

Other container debugging tools

Summary

Onto Production

Performing web operations

Supporting web applications with Docker

Deploying applications

Scaling applications

Further reading

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

Docker is an enterprise-grade container platform that allows you to build and deploy your apps. Its portable format lets you run your code anywhere, from your desktop workstation, to popular cloud computing providers. This book will help you realize the full potential of Docker.

In this book, you will learn a lot about how Docker works. You will learn the basics of Docker, along with some of the fundamental concepts of web operations. You will gain knowledge of Docker and the relevant operating system concepts to get a deeper understanding of what is happening behind the scenes. You will also learn a lot about the tools to deploy and troubleshoot our Docker containers in production in a scalable and manageable fashion.

Who this book is for

If you are a software developer with a good understanding of managing Docker services and the Linux file system, and are looking for ways to optimize working with Docker containers, then this is the book for you. Developers fascinated by containers and workflow automation will also benefit from this book.

What this book covers

Chapter 1, Preparing Docker Hosts, helps you familiarize yourself with Docker Engine and how to prepare a Docker host. We will then build a PKI to ensure secure communication between our Docker host and our Docker client workstation. We will also build a small Docker Swarm cluster, consisting of multiple Docker hosts.

Chapter 2, Configuring Docker with Chef, shows how to automate the configuration of our Docker deployments. We will also use Chef, a piece of configuration management software, to manage Docker hosts in scale.

Chapter 3, Monitoring Docker, helps us to understand the importance of monitoring and collecting metrics in Prometheus. We will also learn how to consolidate logs in an ELK stack.

Chapter 4, Optimizing Docker Images, discusses optimizing our Docker images and improving our development workflow. We will also learn to reduce image deployment time and improve image build time.

Chapter 5, Deploying Containers, looks at how we can use Jenkins to build, distribute, and deploy our containerized application. We will also learn to build and deploy a simple application using a three-stage pipeline.

Chapter 6, Benchmarking, helps us create benchmarks to gauge the performance of our Docker application.

Chapter 7, Load Balancing, talks about how to scale out our Docker applications to increase our capacity. We will use load balancers, which are a key component in the architecture of various web scale applications. We will also learn to balance load with NGINX.

Chapter 8, Troubleshooting Containers, inspects containers with Docker exec, and help us understand what debugging is, along with other container debugging tools.

Chapter 9, Onto Production, wraps up the book by teaching us how to perform web operations. We will also learn to deploy and scale our applications.

To get the most out of this book

A Linux workstation with a recent kernel is needed to serve as a host for Docker CE 18.09.0. This book uses

CentOS 7

as its base operating system to install and set up Docker.

More details on how to get Docker up and running is covered in

Chapter 1

, Preparing Docker Hosts.

Download the example code files

You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

Log in or register at

www.packt.com

.

Select the

SUPPORT

tab.

Click on

Code Downloads & Errata

.

Enter the name of the book in the

Search

box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR/7-Zip for Windows

Zipeg/iZip/UnRarX for Mac

7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub athttps://github.com/PacktPublishing/Docker-High-Performance-Second-Edition. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781789807219_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "First, we will create a directory for our PKI and generate the CA's private key in a file called ca-key.pem."

A block of code is set as follows:

{ "tlsverify": true, "tlscacert": "/etc/docker/ca.pem", "tlskey": "/etc/docker/server-key.pem", "tlscert": "/etc/docker/server.pem"}

Any command-line input or output is written as follows:

dockerhost$ systemctl daemon-reload

dockerhost$ systemctl restart docker.service

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: " In the form field labeled Name, set the value to Unicorn Capacity."

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [e-mail].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.

Preparing Docker Hosts

Docker allows us to deliver applications to our customers faster. It simplifies the workflows needed to get code from development to production by being able to easily create and launch Docker containers. This chapter will be a quick refresher on how to get our environment ready to run Docker-based development and operations workflow by doing the following:

Preparing a Docker host

Enabling remote access to Docker hosts

Building a Docker Swarm cluster

Most of the parts of this chapter are concepts that we are already familiar with and these  are readily available in the Docker documentation website. This chapter shows selected commands and interactions with the Docker host that will be used in the succeeding chapters.

Preparing a Docker host

It is assumed that we are already familiar on how to set up a Docker host. For most of the chapters of this book, we will run against the following environment unless mentioned explicitly:

Operating System: CentOS 7.5

Docker version: 18.09.0

The following commands display the operating system and Docker Engine version running inside our Docker host:

$ ssh dockerhost

dockerhost$ lsb_release -a

LSB Version: :core-4.1-amd64:core-4.1-noarch

Distributor ID: CentOS

Description: CentOS Linux release 7.5.1804 (Core)

Release: 7.5.1804

Codename: Core

dockerhost$ docker version

Version: 18.09.0

API version: 1.39

Go version: go1.10.4

Git commit: 4d60db4

Built: Wed Nov 7 00:48:22 2018

OS/Arch: linux/amd64

Experimental: false

Server: Docker Engine - Community

Engine:

Version: 18.09.0

API version: 1.39 (minimum version 1.12)

Go version: go1.10.4

Git commit: 4d60db4

Built: Wed Nov 7 00:19:08 2018

OS/Arch: linux/amd64

Experimental: false

If we haven't set up our Docker environment, we can follow the instructions in the Docker website found in https://docs.docker.com/install/linux/docker-ce/centos/ to prepare our Docker host.

Enabling remote access

Instead of logging in remotely to our Docker host to run containers, we will access the Docker host by enabling the remote API in Docker Engine. This allows us to manage our Docker containers from our client workstation or continuous delivery server. We will then interact with our Docker host to represent our production environment. The remote API will then be used from our client workstation to perform deployments of our Docker containers.

This section will cover the steps to secure and enable remote access to our Docker host:

Setting up a certificate authority

Reconfiguring Docker Engine to enable remote access

Configuring the Docker client for remote access

Setting up a certificate authority

For the rest of this section, we will be installing TLS certificates in both our Docker host server and client. To ensure trusted communication between the server and client, we will be setting up a Public Key Infrastructure (PKI). This will allow both the Docker engine running in our host and our Docker client to make the connection to verify the identity of each other. 

The first step in building the PKI is setting up the Certificate Authority (CA). A CA is a trusted third party that issues digital certificates to members of our PKI, namely our Docker host and client.

In the next few steps, we will set up our CA inside of our client workstation:

First, we will create a directory for our PKI and generate the CA's private key in a file called

ca-key.pem

:

client$ mkdir ~/ca

client$ cd ~/ca

client$ openssl genrsa -aes256 -out ca-key.pem 4096

Generating RSA private key, 4096 bit long modulus

......................................++

................................................++

e is 65537 (0x10001)

Enter pass phrase for ca-key.pem: ****

Verifying - Enter pass phrase for ca-key.pem: ****

We need to remember the passphrases set for our CA's private key, as we will always need them for the rest of this chapter.

Next, we make sure that this private key is secure by restricting read and write access to us:

client$ chmod 600 ca-key.pem

client$ ls -l ca-key.pem

-rw-------. 1 dockeruser group 3326 Dec 2 20:45 ca-key.pem

Finally, we will generate a certificate for our CA that is self-signed. Let's type the following command to place the self-signed certificate in a file called

ca.pem

:

client$ openssl req -key ca-key.pem -new -x509 \

-subj '/CN=Certificate Authority' \

-sha256 -days 365 -out ca.pem

Enter pass phrase for ca-key.pem: ****

We now have a CA, and we will be distributing its ca.pemcertificate to our Docker host and client later in this section. For now, let's inspect the generate certificate:

client$ cat ca.pem

-----BEGIN CERTIFICATE-----

MIIFEzCCAvugAwIBAgIJAM19ce5sap+kMA0GCSqGSIb3DQEBCwUAMCAxHjAcBgNV

BAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xODEyMDYwMzQ5MTNaFw0xOTEy

MDYwMzQ5MTNaMCAxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTCCAiIw

DQYJKoZIhvcNAQEBBQADggIPADCCAgoCggIBAKCESs7QpRZ78v8p2nKomCGABqCN

b3E0vBpjveTnjA4kOEWVsHloq2o66yuuNff75GNWghzq791KyKJOy/dehNL9DauA

DD3DJh0+uaOGn547W827Z37wJ64acNyvIQjyiyeLrpF4BzzxaZ/AJFVgqar5Kuqc

qiOG3GUYcnfu6mpmlKoa1XqBtSQ+A2fd4/mpXC0zrDrz9MSEOCs5/Xm6/faexYae

V8gBkCYWiUVUi+RRRc2vU1LzuiI5FsXmD3kNHCjNIbYIoyqKMzbTJjEffhN+5B/V

Rc3qfRmfoEv8P0Hc4Wx55qH8BLWwhvNFAZ+nre+j7zPz+dTfLVyveOPxErHaI1V8

WH9qEVf+haNqUBrjNCuL+xyVNx7evPygD88jyZDWLK5Y0JTh2GSPqMeVi3hSKzNP

GbVjT8tmkCUEsYbSJg2vkPYJR4aC8LLdJsjr7wkWBF1IcYYZpLo3EsUnkjNi7MGS

pGdLob3UToekXaA1D6esDhlEB+3Tt/RWJkS91ijUiDs2kTSmDfnxUQGyeD4wx/rj

lPFRSLdUUYiFcdI5VegZVSqYxW/Qw2/t+GvoLkrOrggqY1f++XUgK5hSoT8EqgiG

SjapkgphMEquVP8UlZ3jC0VmgwFnRUEdqau6yLWMYG6TvLkyVi1Vmfam7CoB1aDn

TccUszk+rezX+1nJAgMBAAGjUDBOMB0GA1UdDgQWBBTrqfPKO0i2peZ6Hd/BYOMq

WXD9kDAfBgNVHSMEGDAWgBTrqfPKO0i2peZ6Hd/BYOMqWXD9kDAMBgNVHRMEBTAD

AQH/MA0GCSqGSIb3DQEBCwUAA4ICAQCa6SPGncEZSWu0WLfkh1mERa9JfBQzJFpv

1E7M3tZeFyJS7LfXdcf9WEAaWqTpha87A+5g9uBi/whYk47dyTik07/k+CyF112i

9GXK8j/UNCjAMOSluOCxpIsmMXp2Dn+ma21msN1K/lHK0ZhGWB9ZDggvdzRRPjic

Dq3aQ49ATHQHGg9cqgZO0zXtcQYaHfCNds5YLNVL66eDhuN91V2MEqWtRDHfr0vA

F3KldXfQ/clnrjGLqo7a3oR1R4QofQ03bV+PRIgub+l3Fee1D68BqF9dLRjUABd2

zm5OzNAmmHPSGWGvOxylvPrUS0ulUzMUWdoXN85SDdLHFXTXwpbD/GgqK+Y3BTgO

7d+mOoTHVEdw2gUXLaqeEchBge2Kh/LQtiN7Zp8OY7snX66Z8tF6W2MKhnSpDzcW

J4WMbmaRqsTEeaRk0aTWkhBZukSZf4zjaa/abF+iRvU5c1OGS9GmYfuGq3Tlj+Xo

JZNuKp9HzOPaj8qiD0DJW9EnuZ24zzpDSiSdmOdARcaaFFKhW8i+SVP6VqrAR3Nb

OL8ne6w6kdoiq4+hPKfWVS9Yh0aQstJMNP91Nnw3J+aRz9eN03jpl/z18vHhW/xl

nYJrB2KlC7SOnUT7TMJr4O5Aw1SidxMH6NLiiC1jbTWXDMuYL8UghDIk9Ne/WhBd

-----END CERTIFICATE-----

Enabling remote access in Docker Engine

Now that we have a CA for our PKI, we can use this CA to verify the identity of our Docker host. The following steps will prepare the identity of our Docker host: