A Developer's Essential Guide to Docker Compose E-Book

Emmanouil Gkatziouras

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 231

Veröffentlichungsjahr: 2022

Beschreibung

Software development is becoming increasingly complex due to the various software components used. Applications need to be packaged with software components to facilitate their operations, making it complicated to run them. With Docker Compose, a single command can set up your application and the needed dependencies.
This book starts with an overview of Docker Compose and its usage and then shows how to create an application. You will also get to grips with the fundamentals of Docker volumes and network, along with Compose commands, their purpose, and use cases. Next, you will set up databases for daily usage using Compose and, leveraging Docker networking, you will establish communication between microservices. You will also run entire stacks locally on Compose, simulate production environments, and enhance CI/CD jobs using Docker Compose. Later chapters will show you how to benefit from Docker Compose for production deployments, provision infrastructure on public clouds such as AWS and Azure, and wrap up with Compose deployments on said infrastructure.
By the end of this book, you will have learned how to effectively utilize Docker Compose for day-to-day development.

Leseprobe

A Developer’s Essential Guide to Docker Compose

Simplify the development and orchestration of multi-container applications

Emmanouil Gkatziouras

BIRMINGHAM—MUMBAI

A Developer’s Essential Guide to Docker Compose

Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Rahul Nair

Publishing Product Manager: Niranjan Naikwadi

Senior Editor: Shazeen Iqbal

Content Development Editor: Romy Dias

Technical Editor: Arjun Varma

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Joshua Misquitta

Marketing Coordinator: Nimisha Dua

First published: September 2022

Production reference: 1150922

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80323-436-6

www.packt.com

To the amazing tech community in London. The vibrant tech community of London and its ecosystem helps me to be updated with the latest trends, to be motivated, and to interact with great engineers. Also, to my colleagues at Yapily.

Contributors

About the author

Emmanouil Gkatziouras started his software engineering journey when he joined a computer engineering and informatics department in Patras, Greece. He then worked as a software engineer for various companies. In 2015, he joined Oseven where he started working with cloud providers such as AWS and Azure, and container orchestration tools such as ECS and Kubernetes. He has fulfilled many roles, most recently as a cloud architect for the platform team.

He loves to give back to the developer community by contributing to open source projects such as InfluxDB, Spring Cloud GCP, and Alpakka and by blogging on various software topics. He is committed to continuous learning and is a holder of certifications such as CKA, CCDAK, PSM, CKAD, and PSO.

I want to thank myself, for giving me the time and support I’ve needed to write this book, on top of everyday responsibilities and priorities. I’d also like to thank my partner Viv for her patience while I was writing this book and the whole Packt editing team who assisted me: Romy Dias, Ashwin Dinesh Kharwa, and Niranjan Naikwadi.

About the reviewer

Werner Dijkerman is a freelance cloud, Kubernetes (certified), and DevOps engineer. He's currently focused on, and working with, cloud-native solutions and tools including AWS, Ansible, Kubernetes, and Terraform. He is also focused on Infrastructure as Code and monitoring the correct “thing” with tools such as Zabbix, Prometheus, and the ELK Stack, with a passion for automating everything and avoiding doing anything that resembles manual work.

Big thanks, hugs, and a shoutout to Ernst Vorsteveld!

Table of Contents

Preface

Part 1: Docker Compose 101

1

Introduction to Docker Compose

Technical requirements

Introducing Docker Compose and its usage

Installing Docker Compose

Docker Desktop

Installing Docker

docker compose versus docker-compose

Understanding how Docker Compose works

Your first Docker Compose file

Using your Docker image on Docker Compose

Summary

2

Running the First Application Using Compose

Technical requirements

Creating a core application

Installing Go

A REST API in Go using Gin

The application

Running Redis using Compose

Shelling into a container managed by Compose

Interacting with a Docker Compose service

Packaging your application with Docker and Compose

Enabling environment configuration

Docker image creation

Running the image

Build an image using Compose

Running your multi-container application using Compose

Health check

Depending on services

Labels

Images

Containers

Summary

3

Network and Volumes Fundamentals

Technical requirements

Explaining Docker volumes

Attaching a Docker volume to a container

Shared volumes

Read-only volumes

Docker volume drivers

Using a volume driver versus mounting locally

Declaring Docker volumes on Compose files

Attaching Docker volumes to an existing application

Creating a configuration file

Mounting a file using volume

Mounting read-only volumes

Docker networking

Bridge

Host

Overlay

Defining networks on a Compose configuration

Adding an extra network to the current application

Summary

4

Executing Docker Compose Commands

Technical requirements

Introducing Compose commands

The Docker CLI versus Compose commands

Setting up the target application

Provisioning commands

build

create

up

Container commands

exec

run

pause

unpause

start and stop

restart

kill

ps

Cleanup commands

down

rm

Image commands

List images

Pulling images

Pushing images

Local Docker registry on Compose

Pushing to the local registry

Monitoring commands

Logs

top

Events

Other commands

help

version

port

config

Summary

Part 2: Daily Development with Docker Compose

5

Connecting Microservices

Technical requirements

Introducing the location microservice

Adding a location service to Compose

Adding a network for the location microservice

Executing requests to the location microservice

Streaming task events

Adding a task events processing microservice

Summary

6

Monitoring Services with Prometheus

What is Prometheus?

Adding an endpoint for Prometheus

Adding the metrics endpoint to the Task Manager

Adding the metrics endpoint to the location service

Exporting metrics from the Event Service

Configuring Prometheus to parse metrics

Adding Prometheus to the Compose network

Pushing metrics to Prometheus

Creating your first metrics query

Adding an alert

Summary

7

Combining Compose Files

Technical requirements

Splitting Compose files

Task Manager base

Location service

Event service

Task Manager

Prometheus

Combining Compose files

Selecting the Compose files to run

Using Hoverfly

Extending services

Capturing traffic with Hoverfly

Creating mock applications using Ηoverfly

Creating different environments

Running with capturing enabled

Running with monitoring disabled

Running applications individually

Combining multiple Compose files into one

Using config

Summary

8

Simulating Production Locally

Technical requirements

Segregating private and public workloads

Setting up DynamoDB locally

Creating DynamoDB tables

Interacting with the Local DynamoDB

Setting up SQS locally

Setting up S3 locally

Setting up a REST-based Lambda function

Setting up an SQS-based Lambda function

Docker Compose links

Connecting the Lambda functions

Summary

9

Creating Advanced CI/CD Tasks

Technical requirements

Introduction to CI/CD

Using Docker Compose with GitHub Actions

Creating your first GitHub Action

Caching built images

Building application images

Testing your Compose application

Using Docker Compose with Bitbucket pipelines

Creating your first Bitbucket pipeline

Caching Compose and Docker images

Building application images

Testing your Compose application

Using Docker Compose with Travis

Creating your first Travis job

Caching Compose

Building application images

Testing your Compose application

Summary

Part 3: Deployment with Docker Compose

10

Deploying Docker Compose Using Remote Hosts

Technical requirements

Docker remote hosts

Creating a remote Docker host

Creating a Docker host on AWS EC2

Using the remote Docker host

Docker Contexts

Deploying Compose to remote hosts

Executing remote host deployments through your IDE

Summary

11

Deploying Docker Compose to AWS

Technical requirements

Introduction to AWS ECS

Hosting your Docker images on AWS ECR

Provision ECR using AWS CLI

Provision ECR using Terraform

Storing a Terraform state file

Pushing images to ECR

Adapting the Compose application images

Deploying your application to an ECS cluster

Running your Compose application to an existing cluster

Creating a log group

Creating a private network

Security groups

Configuring the ECS cluster and the load balancer

Updating the Compose file

Running your Compose application on existing infrastructure

Advanced Docker Compose concepts on ECS

Updating the application

Scaling the application

Using secrets

Summary

12

Deploying Docker Compose to Azure

Technical requirements

An introduction to ACI

Pushing to an Azure container registry

Storing the Terraform state file

Deploying on ACI

Summary

13

Migrating to Kubernetes Configuration Using Compose

Technical requirements

Introduction to Kubernetes

Kubernetes components and Compose

Compose applications versus namespaces

Compose services versus Kubernetes services

Labels

Compose networks versus network policies

Using Kompose to convert files

Introduction to Minikube

Deploying to Kubernetes

Summary

Index

Other Books You May Enjoy

Preface

The book explains the fundamentals of Docker Compose and its usage. You will discover the usage of Docker components under Compose along with Compose commands, their purpose, and their use cases. Further on, you will explore setting up databases for daily usage, leveraging Docker networking, and establishing communication between microservices. You will also run entire stacks locally on Compose, simulate production environments, and enhance CI/CD jobs using Docker Compose. Finally, you will learn about advanced topics such as Docker Compose on production deployments, provisioning infrastructure on public clouds such as AWS and Azure, and also pave the way for a migration to the Kubernetes orchestration engine.

Who this book is for

This book is for software engineers, developer advocates, and DevOps engineers looking to set up multi-container Docker applications using Compose without the need to set up a Docker orchestration engine and the expertise required. It is also for team leads looking to increase the productivity of an organization’s software teams by streamlining the provisioning of complex development environments locally using Docker Compose.

What this book covers

Chapter 1, Introduction to Docker Compose, provides an overview of how Compose works and its various usages. There will be a brief explanation of the Docker Compose file format and a Compose example will be run.

Chapter 2, Running the First Application Using Compose, shows you how to create a simple Golang application that interacts with a Redis database. At the end of the chapter, you will have managed to run a multi-container application through Compose.

Chapter 3, Network and Volumes Fundamentals, dives into the fundamentals of Docker volumes and networks. At the end of the chapter, you will have defined and used a network for the existing application.

Chapter 4, Executing Docker Compose Commands, takes you through the Compose commands, their purpose, and the use cases.

Chapter 5, Connecting Microservices, explores creating new microservices. At the end of the chapter, you should have developed new microservices within the same network and established connectivity between them.

Chapter 6, Monitoring Services with Prometheus, covers adding monitoring to the services backed by the monitoring solution Prometheus.

Chapter 7, Combining Compose Files, looks at modularizing the Compose file and splitting it into multiple parts.

Chapter 8, Simulating Production Locally, provides an overview of complex Compose configurations with the goal of simulating production partially or fully in a local environment.

Chapter 9, Creating Advanced CI/CD Tasks, shows you how to create more advanced CI/CD tasks by simulating cases using Compose.

Chapter 10, Deploying Docker Compose Using Remote Hosts, covers deploying to remote hosts using Compose.

Chapter 11, Deploying Docker Compose to AWS, covers utilizing the knowledge acquired on Compose to achieve a deployment on AWS using ECS.

Chapter 12, Deploying Docker Compose to Azure, focuses on another popular cloud provider, Azure. At the end of the chapter, you should achieve a deployment on Azure ACI.

Chapter 13, Migrating to Kubernetes Configuration Using Compose, shows you how to translate the Compose files to a Kubernetes Deployment.

To get the most out of this book

You are expected to understand containerization and must possess fundamental Docker knowledge. Also, you should be comfortable with shell scripting. Ideally, a UNIX workstation would be the best option to progress through the book. Most of the code and commands presented should also be able to run on Windows machines.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/A-Developer-s-Essential-Guide-to-Docker-Compose. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/kD3i4.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Mount the created nginx.conf configuration file as another file in your system."

A block of code is set as follows:

type Task struct { Id string `json:"id"` Name string `json:"name"` Description string `json:"description"` Timestamp int64 `json:"timestamp"` }

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

services: redis: image: redis ports: - 6379:6379

Any command-line input or output is written as follows:

$ curl --location --request POST 'localhost:8080/task/'

$ cat /etc/nginx/nginx.conf

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: "Select System info from the Administration panel."

Tips or Important Notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read A Developer’s Essential Guide to Docker Compose, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Part 1: Docker Compose 101

This part will introduce us to Docker Compose and how it works behind the scenes. We will familiarize ourselves with Compose by developing and deploying a set of applications using Compose. We will also find out how Docker concepts that we use daily (such as networking and volumes) map to Compose. Lastly, we will have an overview of the available Compose commands and familiarize ourselves with executing them.

The following chapters will be covered under this section:

Chapter 1, Introduction to Docker ComposeChapter 2, Running the First Application Using ComposeChapter 3, Network and Volumes FundamentalsChapter 4, Executing Docker Compose Commands