31,19 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Enhance your software deployment workflow using containers
Key Features
- ●Get up-and-running with basic to advanced concepts of Docker
- ●Get acquainted with concepts such as Docker containers, Docker images, orchestrators and so on.
- ●Practical test-based approach to learning a prominent containerization tool
Book Description
Docker containers have revolutionized the software supply chain in small and big enterprises. Never before has a new technology so rapidly penetrated the top 500 enterprises worldwide. Companies that embrace containers and containerize their traditional mission-critical applications have reported savings of at least 50% in total maintenance cost and a reduction of 90% (or more) of the time required to deploy new versions of those applications. Furthermore they are benefitting from increased security just by using containers as opposed to running applications outside containers.
This book starts from scratch, introducing you to Docker fundamentals and setting up an environment to work with it. Then we delve into concepts such as Docker containers, Docker images, Docker Compose, and so on. We will also cover the concepts of deployment, orchestration, networking, and security. Furthermore, we explain Docker functionalities on public clouds such as AWS.
By the end of this book, you will have hands-on experience working with Docker containers and orchestrators such as SwarmKit and Kubernetes.
What you will learn
- ●Containerize your traditional or microservice-based application
- ●Share or ship your application as an immutable container image
- ●Build a Docker swarm and a Kubernetes cluster in the cloud
- ●Run a highly distributed application using Docker Swarm or Kubernetes
- ●Update or rollback a distributed application with zero downtime
- ●Secure your applications via encapsulation, networks, and secrets
- ●Know your options when deploying your containerized app into the cloud
Who this book is for
This book is targeted at system administrators, operations engineers, DevOps engineers, and developers or stakeholders who are interested in getting started with Docker from scratch. No prior experience with Docker Containers is required.
Gabriel N. Schenker has more than 25 years of experience as an independent consultant, architect, leader, trainer, mentor, and developer. Currently, Gabriel works as Senior Curriculum Developer at Confluent after coming from a similar position at Docker. Gabriel has a Ph.D. in Physics, and he is a Docker Captain, a Certified Docker Associate, and an ASP Insider. When not working, Gabriel enjoys time with his wonderful wife Veronicah and his children.Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 447
Veröffentlichungsjahr: 2018
Ähnliche
Learn Docker – Fundamentals of Docker 18.x
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Gebin GeorgeAcquisition Editor: Shrilekha InaniContent Development Editor: Ronn KurienTechnical Editor: Swathy MohanCopy Editor:Safis EditingProject Coordinator: Judie JoseProofreader: Safis EditingIndexer: Priyanka DhadkeGraphics: Tom ScariaProduction Coordinator: Nilesh Mohite
First published: April 2018
Production reference: 1240418
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78899-702-7
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Gabriel N. Schenker has more than 25 years of experience as an independent consultant, architect, leader, trainer, mentor, and developer. Currently, Gabriel works as Senior Curriculum Developer at Confluent after coming from a similar position at Docker. Gabriel has a Ph.D. in Physics, and he is a Docker Captain, a Certified Docker Associate, and an ASP Insider. When not working, Gabriel enjoys time with his wonderful wife Veronicah and his children.
About the reviewer
Peter McKee is a Software Architect and Senior Software Engineer at Docker, Inc. He leads the technical team that delivers the Docker Success Center. He's been leading and mentoring teams for more than 20 years. When not building things with software, he spends his time with his wife and seven kids in beautiful Austin, TX.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Learn Docker – Fundamentals of Docker 18.x
Packt Upsell
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
What Are Containers and Why Should I Use Them?
Technical requirements
What are containers?
Why are containers important?
What's the benefit for me or for my company?
The Moby project
Docker products
Docker CE
Docker EE
The container ecosystem
Container architecture
Summary
Questions
Further reading
Setting up a Working Environment
Technical requirements
The Linux command shell
PowerShell for Windows
Using a package manager
Installing Homebrew on a Mac
Installing Chocolatey on Windows
Choosing a code editor
Docker Toolbox
Docker for Mac and Docker for Windows
Installing Docker for Mac
Installing Docker for Windows
Using docker-machine on Windows with Hyper-V
Minikube
Installing Minikube on  Mac and Windows
Testing Minikube and kubectl
Summary
Questions
Further reading
Working with Containers
Technical requirements
Running the first container
Starting, stopping, and removing containers
Running a random quotes container
Listing containers
Stopping and starting containers
Removing containers
Inspecting containers
Exec into a running container
Attaching to a running container
Retrieving container logs
Logging drivers
Using a container-specific logging driver
Advanced topic – changing the default logging driver
Anatomy of containers
Architecture
Namespaces
Control groups (cgroups)
Union filesystem (UnionFS)
Container plumbing
Runc
Containerd
Summary
Questions
Further reading
Creating and Managing Container Images
What are images?
The layered filesystem
The writable container layer
Copy-on-write
Graph drivers
Creating images
Interactive image creation
Using Dockerfiles
The FROM keyword
The RUN keyword
The COPY and ADD keywords
The WORKDIR keyword
The CMD and ENTRYPOINT keywords
A complex Dockerfile
Building an image
Multistep builds
Dockerfile best practices
Saving and loading images
Sharing or shipping images
Tagging an image
Image namespaces
Official images
Pushing images to a registry
Summary
Questions
Further reading
Data Volumes and System Management
Technical requirements
Creating and mounting data volumes
Modifying the container layer
Creating volumes
Mounting a volume
Removing volumes
Sharing data between containers
Using host volumes
Defining volumes in images
Obtaining Docker system information
Listing resource consumption
Pruning unused resources
Pruning containers
Pruning images
Pruning volumes
Pruning networks
Pruning everything
Consuming Docker system events
Summary
Questions
Further reading
Distributed Application Architecture
What is a distributed application architecture?
Defining the terminology
Patterns and best practices
Loosely coupled components
Stateful versus stateless
Service discovery
Routing
Load balancing
Defensive programming
Retries
Logging
Error handling
Redundancy
Health checks
Circuit breaker pattern
Running in production
Logging
Tracing
Monitoring
Application updates
Rolling updates
Blue-green deployments
Canary releases
Irreversible data changes
Rollback
Summary
Questions
Further reading
Single-Host Networking
Technical requirements
The container network model
Network firewalling
The bridge network
The host network
The null network
Running in an existing network namespace
Port management
Summary
Questions
Further reading
Docker Compose
Technical requirements
Demystifying declarative versus imperative
Running a multi-service app
Scaling a service
Building and pushing an application
Summary
Questions
Further reading
Orchestrators
What are orchestrators and why do we need them?
The tasks of an orchestrator
Reconciling the desired state
Replicated and global services
Service discovery
Routing
Load balancing
Scaling
Self-healing
Zero downtime deployments
Affinity and location awareness
Security
Secure communication and cryptographic node identity
Secure networks and network policies
Role-based access control (RBAC)
Secrets
Content trust
Reverse uptime
Introspection
Overview of popular orchestrators
Kubernetes
Docker Swarm
Apache Mesos and Marathon
Amazon ECS
Microsoft ACS 
Summary
Questions
Further reading
Introduction to Docker Swarm
Architecture
Swarm nodes
Swarm managers
Swarm workers  
Stacks, services, and tasks
Services
Task
Stack
Multi-host networking
Creating a Docker Swarm
Creating a local single node swarm
Creating a local swarm in VirtualBox or Hyper-V
Using Play with Docker (PWD) to generate a Swarm
Creating a Docker Swarm in the cloud
Deploying a first application
Creating a service
Inspecting the service and its tasks
Logs of a service
Reconciling the desired state
Deleting a service or a stack
Deploying a multi-service stack
The swarm routing mesh
Summary
Questions
Further reading
Zero Downtime Deployments and Secrets
Technical requirements
Zero downtime deployment
Popular deployment strategies
Rolling updates
Health checks
Rollback
Blue–green deployments
Canary releases
Secrets
Creating secrets
Using a secret
Simulating secrets in a development environment
Secrets and legacy applications
Updating secrets
Summary
Questions
Further reading
Introduction to Kubernetes
Technical requirements
Architecture
Kubernetes master nodes
Cluster nodes
Introduction to Minikube
Kubernetes support in Docker for Desktop
Pods
Comparing Docker container and Kubernetes pod networking
Sharing the network namespace
Pod life cycle
Pod specification
Pods and volumes
Kubernetes ReplicaSet
ReplicaSet specification
Self-healing
Kubernetes deployment
Kubernetes service
Context-based routing
Comparing SwarmKit with Kubernetes
Summary
Questions
Further reading
Deploying, Updating, and Securing an Application with Kubernetes
Technical requirements
Deploying a first application
Deploying the web component
Deploying the database
Streamlining the deployment
Zero downtime deployments
Rolling updates
Blue–green deployment
Kubernetes secrets
Manually defining secrets
Creating secrets with kubectl
Using secrets in a pod
Secret values in environment variables
Summary
Questions
Further reading
Running a Containerized App in the Cloud
Technical requirements
Deploying our application into AWS ECS
Introduction to ECS
Creating a Fargate ECS cluster of AWS
Authoring a task definition
Running a task in ECS
Modifying the task definition
Adding the database component to the application
Deploying and using Docker EE on AWS
Provisioning the infrastructure
Installing Docker
Installing Docker UCP
Remote admin the UCP cluster
Deploying to Docker Swarm
Deploying to Kubernetes
A short peek into Azure’s container offerings
A short peek into Google’s container offerings
Summary
Questions
Further reading
Assessment
Chapter 1
Chapter 2
Chapter 3
Chapter 4
Chapter 5
Chapter 6
Chapter 7
Chapter 8
Chapter 9
Chapter 10
Chapter 11
Chapter 12
Chapter 13
Chapter 14
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Docker containers have revolutionized the software supply chain in small and big enterprises. Never before has a new technology so rapidly penetrated the top 500 enterprises worldwide. Companies that embrace containers and containerize their traditional mission-critical applications have reported savings of at least 50% in total maintenance costs and a reduction of 90% (or more) in deploying new versions of those applications. Furthermore, they are benefiting from increased security by using containers rather than running applications outside containers.This book starts from scratch, introducing you to Docker fundamentals and setting up an environment to work with it. Then, we delve into concepts such as Docker containers, Docker images, and Docker Compose. We will also cover the concepts of deployment, orchestration, networking, and security. Furthermore, we explain Docker functionalities on public clouds, such as AWS.By the end of this book, you will have hands-on experience working with Docker containers and orchestrators, such as SwarmKit and Kubernetes.
Who this book is for
This book is targeted at system administrators, operations engineers, DevOps engineers, and developers or stakeholders who are interested in getting started with Docker from scratch. No prior experience with Docker containers is required.
What this book covers
Chapter 1, What Are Containers and Why Should I Use Them?, focuses on the software supply chain and the friction within it. It then presents containers as a means to reduce this friction and add enterprise-grade security on top of it. In this chapter, we also look into how containers and the ecosystem around them are assembled. We specifically point out the distinction between the upstream OSS components (Moby) that form the building blocks of the downstream products of Docker and other vendors.
Chapter 2, Setting up a Working Environment, discusses in detail how to set up an ideal environment for developers, DevOps engineers, and operators that can be used when working with Docker containers.
Chapter 3, Working with Containers, teaches how start, stop, and remove containers. The chapter also teaches how to inspect containers to retrieve additional metadata. Furthermore, it introduces how to run additional processes and how to attach to the main process in an already running container. It also shows how to retrieve logging information from a container that is produced by the processes running inside it.
Chapter 4, Creating and Managing Container Images, introduces the different ways to create container images, which serve as templates for containers. It introduces the inner structure of an image and how it is built.
Chapter 5, Data Volumes and System Management, introduces data volumes that can be used by stateful components running in containers. The chapter also introduces system-level commands that are used to gather information about Docker and the underlying OS, as well as commands to clean the system from orphaned resources. Finally, it introduces the system events generated by the Docker engine.
Chapter 6, Distributed Application Architecture, introduces the concept of a distributed application architecture and discusses the various patterns and best practices that are required to run a distributed application successfully. Finally, it discusses the additional requirements that need to be fulfilled to run such an application in production.
Chapter 7, Single-Host Networking, introduces the Docker container networking model and its single-host implementation in the form of the bridge network. The chapter introduces the concept of software-defined networks (SDNs) and how they are used to secure containerized applications. Finally, it introduces how container ports can be opened to the public and thus how to make containerized components accessible from the outside world.
Chapter 8, Docker Compose, introduces the concept of an application consisting of multiple services, each running in a container, and how Docker Compose allows us to easily build, run, and scale such an application using a declarative approach.
Chapter 9, Orchestrators, introduces the concept of orchestrators. It teaches why orchestrators are needed and how they work. The chapter also provides an overview of the most popular orchestrators and explores a few of their pros and cons.
Chapter 10, Introduction to Docker Swarm, introduces Docker's native orchestrator called SwarmKit. It elaborates on all the concepts and objects SwarmKit uses to deploy and run a distributed, resilient, robust, and highly available application in a cluster on-premise, or in the cloud. The chapter also introduces how SwarmKit ensures secure applications using SDNs to isolate containers and secrets to protect sensitive information.
Chapter 11, Zero Downtime Deployments and Secrets, teaches how to deploy services or applications onto a Docker swarm with zero downtime and automatic rollback capabilities. It also introduces secrets as a means to protect sensitive information.
Chapter 12, Introduction to Kubernetes, introduces the currently most popular container orchestrator. It introduces the core Kubernetes objects that are used to define and run a distributed, resilient, robust, and highly available application in a cluster. Finally, it introduces Minikube as a way to locally deploy a Kubernetes application and also the integration of Kubernetes with Docker for Mac and Docker for Windows.
Chapter 13, Deploying, Updating, and Securing an Application with Kubernetes, teaches how to deploy, update, and scale applications into a Kubernetes cluster. It also explains how zero-downtime deployments are achieved to enable disruption free updates and rollbacks of mission-critical applications. This chapter also introduces Kubernetes secrets as a means to configure services and protect sensitive data.
Chapter 14, Running a Containerized App in the Cloud, gives an overview over some of the most popular ways of running containerized applications in the cloud. We have a closer look to what the most popular cloud vendor, AWS, offers in this regard. We include self-hosting and hosted solutions and discuss their pros and cons. Offerings of other vendors, such as Microsoft Azure and Google Cloud Engine, are also briefly discussed.
To get the most out of this book
Ideally you have access to a laptop or personal computer with Windows 10 Professional or a recent version of Mac OS X installed. A computer with any popular Linux OS installed works too. If you're on a Mac you should install Docker for Mac and if you're on Windows then install Docker for Windows. You can download them from here: https://www.docker.com/community-edition
If you are on an older version of Windows or are using Windows 10 Home edition, then you should install Docker Toolbox. You can find the Docker Toolbox here: https://docs.docker.com/toolbox/toolbox_install_windows/
On the Mac, use the Terminal application, and on Windows, use a PowerShell console to try out the commands you will be learning. You also need a recent version of a browser such as Google Chrome, Safari or Internet Explorer. Of course you will need internet access to download tools and container images that we are going to use and explore in this book.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packtpub.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/fundamentalsofdocker/labs. If there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it from https://www.packtpub.com/sites/default/files/downloads/LearnDockerFundamentalsofDocker18x_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "The content of each layer is mapped to a special folder on the host system, which is usually a subfolder of /var/lib/docker/."
A block of code is set as follows:
COPY . /appCOPY ./web /app/webCOPY sample.txt /data/my-sample.txtADD sample.tar /app/bin/ADD http://example.com/sample.txt /data/
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
FROM python:2.7RUN mkdir -p /appWORKDIR /appCOPY ./requirements.txt /app/
RUN pip install -r requirements.txt
CMD ["python", "main.py"]
Any command-line input or output is written as follows:
$ mkdir ~/FundamentalsOfDocker
$ cd ~/FundamentalsOfDocker
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Select System info from the Administration panel."
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
What Are Containers and Why Should I Use Them?
This first chapter of this book will introduce you to the world of containers and their orchestration. The book starts from the beginning, assuming no prior knowledge in the area of containers, and will give you a very practical introduction into the topic.
In this chapter, we are focusing on the software supply chain and the friction within it. We then present containers as a means to reduce this friction and add enterprise-grade security on top of it. In this chapter, we also look into how containers and the ecosystem around them are assembled. We specifically point out the distinction between the upstream Operations Support System (OSS) components, united under the code name Moby, that form the building blocks of the downstream products of Docker and other vendors.
The chapter covers the following topics:
What are containers?
Why are containers important?
What's the benefit for me or for my company?
The Moby project
Docker products
The container ecosystem
Container architecture
After completing this module, you will be able to:
Explain in a few simple sentences to an interested layman what containers are, using an analogy such as physical containers
Justify to an interested layman why containers are so important, using an analogy such as physical containers versus traditional shipping, or apartment homes versus single family homes, and so on
Name at least four upstream open source components that are used by the Docker products, such as Docker for Mac/Windows
Identify at least three Docker products
Technical requirements
This chapter is a theoretical introduction into the topic. Therefore, there are no special technical requirements for this chapter.
What are containers?
A software container is a pretty abstract thing and thus it might help if we start with an analogy that should be pretty familiar to most of the readers. The analogy is a shipping container in the transportation industry. Throughout history, people have been transporting goods from one location to another by various means. Before the invention of the wheel, goods would most probably have been transported in bags, baskets, or chests on the shoulders of the humans themselves, or they might have used animals such as donkeys, camels, or elephants to transport them.
With the invention of the wheel, transportation became a bit more efficient as humans would built roads on which they could move their carts along. Many more goods could be transported at a time. When we then introduced the first steam-driven machines, and later gasoline driven engines, transportation became even more powerful. We now transport huge amounts of goods in trains, ships, and trucks. At the same time, the type of goods became more and more diverse, and sometimes complex to handle.
In all these thousands of years, one thing did not change though, and that was the necessity to unload the goods at the target location and maybe load them onto another means of transportation. Take, for example, a farmer bringing a cart full of apples to a central train station where the apples are then loaded onto a train, together with all the apples from many other farmers. Or think of a winemaker bringing his barrels of wine with a truck to the port where they are unloaded, and then transferred to a ship that will transport the barrels overseas.
This unloading from one means of transportation and loading onto another means of transportation was a really complex and tedious process. Every type of good was packaged in its own way and thus had to be handled in its own way. Also, loose goods risked being stolen by unethical workers, or goods could be damaged in the process.
Then, there came the container, and it totally revolutionized the transportation industry. The container is just a metallic box with standardized dimensions. The length, width, and height of each container is the same. This is a very important point. Without the world agreeing on a standard size, the whole container thing would not have been as successful as it is now.
Now, with standardized containers, companies who want to have their goods transported from A to B package those goods into these containers. Then, they call a shipper which comes with a standardized means for transportation. This can be a truck that can load a container or a train whose wagons can each transport one or several containers. Finally, we have ships that are specialized in transporting huge amounts of containers. The shippers never need to unpack and repackage goods. For a shipper, a container is a black box and they are not interested in what is in it nor should they care in most cases. It is just a big iron box with standard dimensions. The packaging of goods into containers is now fully delegated to the parties that want to have their goods shipped, and they should know best on how to handle and package those goods.
Since all containers have the same standardized shape and dimensions, the shippers can use standardized tools to handle containers, that is, cranes that unload containers, say from a train or a truck, and load them onto a ship or vice versa. One type of crane is enough to handle all the containers that come along over time. Also, the means of transportation can be standardized, such as container ships, trucks, and trains.
Because of all this standardization, all the processes in and around shipping goods could also be standardized and thus made much more efficient than they were before the age of containers.
I think by now you should have a good understanding of why shipping containers are so important and why they revolutionized the whole transportation industry. I chose this analogy purposefully, since the software containers that we are going to introduce here fulfill the exact same role in the so-called software supply chain as shipping containers do in the supply chain of physical goods.
In the old days, developers would develop a new application. Once that application was completed in the eyes of the developers, they would hand this application over to the operations engineers that were then supposed to install it on the production servers and get it running. If the operations engineers were lucky, they even got a somewhat accurate document with installation instructions from the developers. So far so good, and life was easy.
But things got a bit out of hand when in an enterprise, there were many teams of developers that created quite different types of applications, yet all needed to be installed on the same production servers and kept running there. Usually, each application has some external dependencies such as which framework it was built on or what libraries it uses and so on. Sometimes, two applications would use the same framework but in different versions that might or might not be compatible between each other. Our operations engineer's life became much harder over time. They had to be really creative on how they could load their ship, which is of course their servers with different applications without breaking something.
Installing a new version of a certain application was now a complex project on its own and often needed months of planning and testing. In other words, there was a lot of friction in the software supply chain. But these days, companies rely more and more on software and the release cycles become shorter and shorter. We cannot afford anymore to just have a new release maybe twice a year. Applications need to be updated in a matter of weeks or days, or sometimes even multiple times per day. Companies that do not comply risk going out of business due to the lack of agility. So, what's the solution?
A first approach was to use virtual machines (VMs). Instead of running multiple applications all on the same server, companies would package and run a single application per VM. With it, the compatibility problems were gone and life seemed good again. Unfortunately, the happiness didn't last for long. VMs are pretty heavy beasts on their own since they all contain a full-blown OS such as Linux or Windows Server and all that for just a single application. This is as if in the transportation industry you would use a gigantic ship just to transport a truck load of bananas. What a waste. That can never be profitable.
The ultimate solution to the problem was to provide something much more lightweight than VMs but also able to perfectly encapsulate the goods it needed to transport. Here, the goods are the actual application written by our developers plus (and this is important) all the external dependencies of the application, such as framework, libraries, configurations, and more. This holy grail of a software packaging mechanism was the Docker container.
Developers use Docker containers to package their applications, frameworks, and libraries into them, and then they ship those containers to the testers or to the operations engineers. For the testers and operations engineers, the container is just a black box. It is a standardized black box, though. All containers, no matter what application runs inside them, can be treated equally. The engineers know that if any container runs on their servers, then any other containers should run too. And this is actually true, apart from some edge cases which always exist.
Thus, Docker containers are a means to package applications and their dependencies in a standardized way. Docker then coined the phrase—Build, ship and run anywhere.
Why are containers important?
These days, the time between new releases of an application become shorter and shorter, yet the software itself doesn't become any simpler. On the contrary, software projects increase in complexity. Thus, we need a way to tame the beast and simplify the software supply chain.
We also hear every day how much more cyber crimes are on the rise. Many well-known companies are affected by security breaches. Highly sensitive customer data gets stolen, such as social security numbers, credit card information, and more. But not only customer data is compromised, sensitive company secrets are also stolen.
Containers can help in many ways. First of all, Gartner has found in a recent report that applications running in a container are more secure than their counterparts not running in a container. Containers use Linux security primitives such as Linux kernel namespaces to sandbox different applications running on the same computers and control groups (cgroups), to avoid the noisy neighbor problem where one bad application is using all available resources of a server and starving all other applications.
Due to the fact that container images are immutable, it is easy to have them scanned for known vulnerabilities and exposures, and in doing so, increase the overall security of our applications.
Another way we can make our software supply chain more secure when using containers is to use content trust. Content trust basically ensures that the author of a container image is who they pretend to be and that the consumer of the container image has a guarantee that the image has not been tampered with in transit. The latter is known as a man-in-the-middle (MITM) attack.
All that I have just said is of course technically also possible without using containers, but since containers introduce a globally accepted standard, it makes it so much easier to implement those best practices and enforce them.
OK, but security is not the only reason why containers are important. There are other reasons:
One of them is the fact that containers make it easy to simulate a production-like environment, even on a developer's laptop. If we can containerize any application, then we can also containerize, say, a database such as Oracle or MS SQL Server. Now, everyone who has ever had to install an Oracle database on a computer knows that this is not the easiest thing to do and it takes a lot of space away on your computer. You wouldn't want to do that to your development laptop just to test whether the application you developed really works end to end. With containers at hand, I can run a full-blown relational database in a container as easily as saying 1, 2, 3. And when I'm done with testing, I can just stop and delete the container and the database is gone without leaving a trace on my computer.
Since containers are very lean compared to VMs, it is not uncommon to have many containers running at the same time on a developer's laptop without overwhelming the laptop.
A third reason why containers are important is that operators can finally concentrate on what they are really good at, provisioning infrastructure, and running and monitoring applications in production. When the applications they have to run on a production system are all containerized, then operators can start to standardize their infrastructure. Every server becomes just another Docker host. No special libraries of frameworks need to be installed on those servers, just an OS and a container runtime such as Docker.
Also, the operators do not have to have any intimate knowledge about the internals of the applications anymore since those applications run self-contained in containers that ought to look like black boxes to the operations engineers, similar to how the shipping containers look to the personnel in the transportation industry.
What's the benefit for me or for my company?
Somebody once said that today, every company of a certain size has to acknowledge that they need to be a software company. Software runs all businesses, period. As every company becomes a software company, there is a need to establish a software supply chain. For the company to remain competitive, their software supply chain has to be secure and efficient. Efficiency can be achieved through thorough automation and standardization. But in all three areas, security, automation, and standardization, containers have shown to shine. Large and well-known enterprises have reported that when containerizing existing legacy applications (many call them traditional applications) and establishing a fully automated software supply chain based on containers, they can reduce the cost used for maintenance of those mission-critical applications by a factor of 50 to 60% and they can reduce the time between new releases of these traditional applications by up to 90%.
That said, the adoption of container technology saves these companies a lot of money, and at the same time it speeds up the development process and reduces the time to market.
The Moby project
Originally, when the company Docker introduced Docker containers, everything was open source. Docker didn't have any commercial products at this time. The Docker engine which the company developed was a monolithic piece of software. It contained many logical parts, such as the container runtime, a network library, a RESTful API, a command-line interface, and much more.
Other vendors or projects such as Red Hat or Kubernetes were using the Docker engine in their own products, but most of the time they were only using part of its functionality. For example, Kubernetes did not use the Docker network library of the Docker engine but provided its own way of networking. Red Hat in turn did not update the Docker engine frequently and preferred to apply unofficial patches to older versions of the Docker engine, yet they still called it the Docker engine.
Out of all these reasons and many more, the idea emerged that Docker had to do something to clearly separate the Docker open source part from the Docker commercial part. Furthermore, the company wanted to prevent competitors from using and abusing the name Docker for their own gains. This was the main reason why the Moby project was born. It serves as the umbrella for most of the open source components Docker developed and continues to develop. These open source projects do not carry the name Docker in them anymore.
Part of the Moby project are components for image management, secret management, configuration management, and networking and provisioning, to name just a few. Also, part of the Moby project are special Moby tools that are, for example, used to assemble components into runnable artifacts.
Some of the components that technically would belong to the Moby project have been donated by Docker to the Cloud Native Computing Foundation (CNCF) and thus do not appear in the list of components anymore. The most prominent ones are containerd and runc which together form the container runtime.
Docker products
Docker currently separates its product lines into two segments. There is the Community Edition (CE) which is closed source yet completely free, and then there is the Enterprise Edition (EE) which is also a closed source and needs to be licensed on a yearly basis. The enterprise products are backed by 24 x 7 support and are supported with bug fixes much longer than their CE counterparts.
Docker CE
Part of the Docker community edition are products such as the Docker Toolbox, Docker for Mac, and Docker for Windows. All these three products are mainly targeting developers.
Docker for Mac and Docker for Windows are easy-to-install desktop applications that can be used to build, debug, and test Dockerized applications or services on a Mac or on Windows. Docker for Mac and Docker for Windows are complete development environments which deeply integrated with their respective hypervisor framework, networking, and filesystem. These tools are the fastest and most reliable way to run Docker on a Mac or on Windows.
Under the umbrella of the CE, there are also two products that are more geared towards operations engineers. Those products are Docker for Azure and Docker for AWS.
For example, with Docker for Azure, which is a native Azure application, you can set up Docker in a few clicks, optimized for and integrated to the underlying Azure Infrastructure as a Service (IaaS) services. It helps operations engineers to accelerate time to productivity in building and running Docker applications in Azure.
Docker for AWS works very similar but for Amazon's cloud.
Docker EE
The Docker EE consists of the two products Universal Control Plane (UCP) and Docker Trusted Registry (DTR) that both run on top of Docker Swarm. Both are Swarm applications. Docker EE builds on top of the upstream components of the Moby project and adds enterprise-grade features such as role-based access control (RBAC), multi tenancy, mixed clusters of Docker Swarm and Kubernetes, web-based UI, and content trust, as well as image scanning on top of it.
The container ecosystem
There has never been a new technology introduced in IT that penetrated the landscape so quickly and so thoroughly than containers. Any company that doesn't want to be left behind cannot ignore containers. This huge interest in containers from all sectors of the industry has triggered a lot of innovation in this sector. Numerous companies have specialized in containers and either provide products that build on top of this technology or build tools that support it.
Initially, Docker didn't have a solution for container orchestration thus other companies or projects, open source or not, tried to close this gap. The most prominent one is Kubernetes which was initiated by Google and then later donated to the CNCF. Other container orchestration products are Apache Mesos, Rancher, Red Hat's Open Shift, Docker's own Swarm, and more.
More recently, the trend goes towards a service mesh. This is the new buzz word. As we containerize more and more applications, and as we refactor those applications into more microservice-oriented applications, we run into problems that simple orchestration software cannot solve anymore in a reliable and scalable way. Topics in this area are service discovery, monitoring, tracing, and log aggregation. Many new projects have emerged in this area, the most popular one at this time being Istio, which is also part of the CNCF.
Many say that the next step in the evolution of software are functions, or more precisely, Functions as a Service (FaaS). Some projects exist that provide exactly this kind of service and are built on top of containers. One prominent example is OpenFaaS.
We have only scratched the surface of the container ecosystem. All big IT companies such as Google, Microsoft, Intel, Red Hat, IBM, and more are working feverishly on containers and related technologies. The CNCF that is mainly about containers and related technologies, has so many registered projects, that they do not all fit on a poster anymore. It's an exciting time to work in this area. And in my humble opinion, this is only the beginning.
Container architecture
Now, let's discuss on a high level how a system that can run Docker containers is designed. The following diagram illustrates what a computer on which Docker has been installed looks like. By the way, a computer which has Docker installed is often called a Docker host, because it can run or host Docker containers:
In the preceding diagram, we see three essential parts:
On the bottom, we have the Linux operating system
In the middle dark gray, we have the container runtime
On the top, we have the Docker engine
Containers are only possible due to the fact that the Linux OS provides some primitives, such as namespaces, control groups, layer capabilities, and more which are leveraged in a very specific way by the container runtime and the Docker engine. Linux kernel namespaces such as process ID (pid) namespaces or network(net) namespaces allow Docker to encapsulate or sandbox processes that run inside the container. Control groups make sure that containers cannot suffer from the noisy neighbor syndrome, where a single application running in a container can consume most or all of the available resources of the whole Docker host. Control groups allow Docker to limit the resources, such as CPU time or the amount of RAM that each container gets maximally allocated.
The container runtime on a Docker host consists of containerd and runc. runc is the low-level functionality of the container runtime and containerd, which is based on runc, provides the higher-level functionality. Both are open source and have been donated by Docker to the CNCF.
The container runtime is responsible for the whole life cycle of a container. It pulls a container image (which is the template for a container) from a registry if necessary, creates a container from that image, initializes and runs the container, and eventually stops and removes the container from the system when asked.
The Docker engine provides additional functionality on top of the container runtime, such as network libraries or support for plugins. It also provides a REST interface over which all container operations can be automated. The Docker command-line interface that we will use frequently in this book is one of the consumers of this REST interface.
Summary
In this chapter, we looked at how containers can massively reduce the friction in the software supply chain and on top of that, make the supply chain much more secure.
In the upcoming chapter, we will familiarize ourselves with containers. We will learn how to run, stop, and remove containers and otherwise manipulate them. We will also have a pretty good overview over the anatomy of containers. For the first time, we're really going to get our hands dirty and play with these containers, so stay tuned.
Questions
Please solve the following questions to assess your learning progress:
Which statements are correct (multiple answers are possible)?
A container is kind of a lightweight VM
A container only runs on a Linux host
A container can only run one process
The main process in a container always has PID 1
A container is one or more processes encapsulated by Linux namespaces and restricted by cgroups
Explain to an interested layman in your own words, maybe using analogies, what a container is.
Why are containers considered to be a game changer in IT? Name three to four reasons.
What does it mean when we claim:
If a container runs on a given platform then it runs anywhere...
? Name two to three reasons why this is true.
True or False:
Docker containers are only really useful for modern greenfield applications based on microservices
. Please justify your answer.
How much does a typical enterprise save when containerizing their legacy applications?
20%
33%
50%
75%
Which two core concepts of Linux are containers based on?
Further reading
Here is a list of links that lead to more detailed information regarding topics we have discussed in this chapter:
Docker overview
at
https://docs.docker.com/engine/docker-overview/
The Moby project
at
https://mobyproject.org/
Docker products
at
https://www.docker.com/get-docker
Cloud Native Computing Foundation
at
https://www.cncf.io/
containerd – industry standard container runtime
at
https://containerd.io/
Setting up a Working Environment
In the last chapter, we learned what Docker containers are and why they're important. We learned what kinds of problem containers solve in a modern software supply chain.
In this chapter, we are going to prepare our personal or working environment to work efficiently and effectively with Docker. We will discuss in detail how to set up an ideal environment for developers, DevOps, and operators that can be used when working with Docker containers.
This chapter covers the following topics:
The Linux command shell
PowerShell for Windows
Using a package manager
Choosing a code editor
Docker Toolbox
Docker for Mac and Docker for Windows
Minikube
After completing this chapter, you will be able to do the following:
Use an editor on your laptop that is able to edit simple files such as a Dockerfile or a
docker-compose.yml
file
Use a shell such as Bash on Mac and PowerShell on Windows to execute Docker commands and other simple operations, such as navigating the folder structure or creating a new folder
Install Docker for Mac or Docker for Windows on your computer
Execute simple Docker commands such as
docker version
or
docker container run
on your Docker for Mac or Docker for Windows
Successfully install Docker Toolbox on your computer
Use
docker-machine
to create a Docker host on VirtualBox
Configure your local Docker CLI to remote access a Docker host running in VirtualBox
Technical requirements
For this chapter, you will need a laptop or a workstation with either macOS or Windows, preferably Windows 10 Professional, installed. You should also have free internet access to download applications and the permission to install those applications on your laptop.
The Linux command shell
Docker containers were first developed on Linux for Linux. It is thus natural that the primary command-line tool used to work with Docker, also called a shell, is a Unix shell; remember, Linux derives from Unix. Most developers use the Bash shell. On some lightweight Linux distributions, such as Alpine, Bash is not installed and consequently one has to use the simpler Bourne shell, just called sh. Whenever we are working in a Linux environment, such as inside a container or on a Linux VM, we will use either /bin/bash or /bin/sh, depending on their availability.
Although macOS X is not a Linux OS, Linux and OS X are both flavors of Unix and thus support the same types of tools. Among those tools are the shells. So, when working on a Mac, you will probably be using the Bash shell.
In this book, we expect from the readers a familiarity with the most basic scripting commands in Bash, and PowerShell if you are working on Windows. If you are an absolute beginner, then we strongly recommend that you familiarize yourself with the following cheat sheets:
Linux Command Line Cheat Sheet
by Dave Child at
http://bit.ly/2mTQr8l
PowerShell Basic Cheat Sheet
at
http://bit.ly/2EPHxze
PowerShell for Windows
On a Windows computer, laptop, or server, we have multiple command-line tools available. The most familiar is the command shell. It has been available on any Windows computer for decades. It is a very simple shell. For more advanced scripting, Microsoft has developed PowerShell. PowerShell is very powerful and very popular among engineers working on Windows. On Windows 10, finally, we have the so-called Windows Subsystem for Linux, which allows us to use any Linux tool, such as the Bash or Bourne shells. Apart from this, there also exist other tools that install a Bash shell on Windows, for example, the Git Bash shell. In this book, all commands will use Bash syntax. Most of the commands also run in PowerShell.
Our recommendation for you is thus to either use PowerShell or any other Bash tool to work with Docker on Windows.
Using a package manager
The easiest way to install software on a Mac or Windows laptop is to use a good package manager. On a Mac, most people use Homebrew and on Windows, Chocolatey is a good choice.
Installing Homebrew on a Mac
Installing Homebrew on a Mac is easy; just follow the instructions at https://brew.sh/.
The following is the command to install Homebrew:
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Once the installation is finished, test whether Homebrew is working by entering brew --version in the Terminal. You should see something like this:
$ brew --version
Homebrew 1.4.3
Homebrew/homebrew-core (git revision f4e35; last commit 2018-01-11)
Now, we are ready to use Homebrew to install tools and utilities. If we, for example, want to install the Vi text editor, we can do so like this:
$ brew install vim
This will then download and install the editor for you.
Installing Chocolatey on Windows
To install the Chocolatey package manager on Windows, please follow the instructions at https://chocolatey.org/ or just execute the following command in a PowerShell Terminal that you have run as administrator:
PS> Set-ExecutionPolicy Bypass -Scope Process -Force; iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))
Once Chocolatey is installed, test it with the command choco without additional parameters. You should see output similar to the following:
PS> choco
Chocolatey v0.10.3
To install an application such as the Vi editor, use the following command:
PS> choco install -y vim
The -y parameter makes sure that the installation happens without asking for reconfirmation. Please note that once Chocolatey has installed an application, you need to open a new PowerShell window to use it.
Choosing a code editor
Using a good code editor is essential to working productively with Docker. Of course, which editor is the best is highly controversial and depends on your personal preference. A lot of people use Vim, or others such as Emacs, Atom, Sublime, or Visual Studio (VS
