AWS for System Administrators E-Book

Prashant Lakhera

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 282

Veröffentlichungsjahr: 2021

Beschreibung

Amazon Web Services (AWS) is one of the most popular and efficient cloud platforms for administering and deploying your applications to make them resilient and robust. AWS for System Administrators will help you to learn several advanced cloud administration concepts for deploying, managing, and operating highly available systems on AWS.
Starting with the fundamentals of identity and access management (IAM) for securing your environment, this book will gradually take you through AWS networking and monitoring tools. As you make your way through the chapters, you’ll get to grips with VPC, EC2, load balancer, Auto Scaling, RDS database, and data management. The book will also show you how to initiate AWS automated backups and store and keep track of log files. Later, you’ll work with AWS APIs and understand how to use them along with CloudFormation, Python Boto3 Script, and Terraform to automate infrastructure.
By the end of this AWS book, you’ll be ready to build your two-tier startup with all the necessary infrastructure, monitoring, and logging components in place.

Leseprobe

AWS for System Administrators

Build, automate, and manage your infrastructure on the most popular cloud platform – AWS

Prashant Lakhera

BIRMINGHAM—MUMBAI

AWS for System Administrators

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Wilson D'souza

Publishing Product Manager: Vijin Boricha

Acquisition Editor: Shrilekha Inani

Senior Editor: Arun Nadar

Content Development Editor: Romy Dias

Technical Editor: Yoginee Marathe

Copy Editor: Safis Editing

Project Coordinator: Neil Dmello

Proofreader: Safis Editing

Indexer: Priyanka Dhadke

Production Designer: Nilesh Mohite

First published: January 2021

Production reference: 1130121

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80020-153-8

www.packt.com

Packt.com

Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionalsImprove your learning with Skill Plans built especially for youGet a free eBook or video every monthFully searchable for easy access to vital informationCopy and paste, print, and bookmark content

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the author

Prashant Lakhera (lakhera2015 on Twitter) is an X-RHCA (Red Hat Certified Architect) and a seasoned Linux and open source specialist with over 15 years of enterprise open source experience.

Having a positive impact on the world is important to him, which is why he shares his knowledge with others through his website, blog posts, and YouTube channel, which also helps him to dig deep into topics and build on his expertise.

I would like to thank my wife, Pratima, for her support while writing this book, and my furry boy, Prince. Also, to my mother, who always supports and encourages me throughout my life.

About the reviewer

Saurabh Dhawan is an AWS- and Azure-certified cloud solution architect with over 16 years of IT experience. He has first-hand knowledge of building cloud-native solutions and a knack for Alexa programming. Saurabh has worked in India's largest IT company in the past and is currently part of the architecture team for the world's most iconic telecom company.

I would like to thank my wife for letting me get lost in my home office for hours on end!

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Table of Contents

Preface

Section 1: AWS Services and Tools

Chapter 1: Setting Up the AWS Environment

Technical requirements

Setting up the environment

Installing the AWS CLI

Configuring command-line completion

Configuring the AWS command line

Understanding the AWS CLI command structure

Introducing Python Boto3

Installing Python Boto3

Verifying the Boto3 setup

Introducing CloudFormation

Writing your first CloudFormation template

Creating a CloudFormation stack using the AWS console

Creating a CloudFormation stack using the AWS CLI

Introducing Terraform

Installing Terraform

Creating resources using Terraform

Installing tools in an automated way

Summary

Chapter 2: Protecting Your AWS Account Using IAM

Technical requirements

Creating IAM users and groups

Introducing IAM users

Introducing IAM groups

Understanding IAM policies

IAM policy structure

Introducing ARN

IAM policy evaluation

Creating the IAM policy using the AWS CLI

Creating IAM roles

Advantages of using an IAM role

Creating an IAM role using Terraform

Introducing AWS Security Token Service (AWS STS)

Advantages of AWS STS

Use cases

IAM cross-account access

Real-time use case of launching a specific instance using CloudFormation

Rotating IAM credentials using Boto3

Prerequisites

Creating a Boto3 script to rotate credentials

Summary

Section 2: Building the Infrastructure

Chapter 3: Creating a Data Center in the Cloud Using VPC

Technical requirements

Setting up two VPCs

Creating your first VPC using the AWS console

Creating a second VPC using CloudFormation

Introducing AWS Transit Gateway

Creating your first transit gateway using the AWS console

Creating a second transit gateway using Terraform

Real-time use case to enable a VPC flow log

Summary

Chapter 4: Scalable Compute Capacity in the Cloud via EC2

Technical requirements

Setting up EC2 instances

Creating an EC2 instance using AWS CloudFormation

Creating an AWS billing alarms

Real-time use case to clean up an unused AMI

Real-time use case to detach unused EBS volumes

Real-time use case to shutdown instances on a daily basis

Summary

Section 3: Adding Scalability and Elasticity to the Infrastructure

Chapter 5: Increasing an Application's Fault Tolerance with Elastic Load Balancing

Technical requirements

Different load balancers offered by AWS

Setting up the application load balancer

Setting up the application load balancer

Automating the application load balancer using Terraform

Summary

Chapter 6: Increasing Application Performance Using AWS Auto Scaling

Technical requirements

Setting up Auto Scaling

Creating a launch template

Creating an AWS Auto Scaling group

Verifying an Auto Scaling group

Understanding Auto Scaling policies

Scaling an application based on demand

Testing the Auto Scaling group

Creating an Auto Scaling group using Terraform

Summary

Chapter 7: Creating a Relational Database in the Cloud using AWS Relational Database Service (RDS)

Technical requirements

The different database offerings in AWS RDS

Setting up AWS RDS in high availability mode

Setting up a MySQL read replica

Automating AWS RDS MySQL creation using Terraform

Summary

Section 4: The Monitoring, Metrics, and Backup Layers

Chapter 8: Monitoring AWS Services Using CloudWatch and SNS

Technical requirements

CloudWatch monitoring

Monitoring custom metrics using CloudWatch

Downloading and installing the CloudWatch agent

Creating an IAM role used by CloudWatch agent

Running the CloudWatch agent on your server

Introduction to SNS

Introduction to CloudWatch Events

Automating alarm notification using email and a Slack channel

Configuring Slack

Configuring CloudWatch

Creating a Lambda function

Testing the integration

Summary

Chapter 9: Centralizing Logs for Analysis

Technical requirements

Why do we need log management?

Setting up the CloudWatch agent

Setting up AWS Elasticsearch and Kibana

Summary

Chapter 10: Centralizing Cloud Backup Solution

Technical requirements

The v backup options offered by AWS

Why do we back up data?

Setting up the AWS DLM

Backing up your data to S3 using the AWS CLI

Transitioning S3 data to Glacier using a lifecycle policy

Automating transitioning S3 data to Glacier using Terraform

Summary

Chapter 11: AWS Disaster Recovery Solutions

Technical requirements

Discussing the various DR solutions offered by AWS

Backup and restore

Pilot light

Warm standby in AWS

Hot standby (with multi-site)

Configuring a website to fail over to an S3 bucket

Summary

Chapter 12: AWS Tips and Tricks

Technical requirements

Some common pitfalls – VPC limitations

Which VPC subnets to choose while building a VPC

Dedicated instance versus dedicated host – which should you choose?

The power of the IAM permission boundary

Custom CloudWatch metrics

Tagging, tagging, and tagging – why is tagging important?

Protecting your EC2 instances and EBS volumes using termination protection

How to reduce your AWS bill

Choosing an AWS bucket name and how to create a random bucket name

Automating AMI creation

Creating an AMI using the AWS console

Creating an AMI using the AWS CLI

Automating AMI creation using Packer

Summary

Other Books You May Enjoy

Leave a review - let other readers know what you think

Preface

AWS for System Administrators will teach you how to deploy, manage, and operate highly available systems on AWS. You'll start with the fundamentals of Identity and Access Management (IAM) to secure your environment before moving on to AWS networking and monitoring tools. As you make your way through the chapters, you'll get to grips with concepts such as Virtual Private Cloud (VPC), Elastic Compute Cloud (EC2), load balancers, auto-scaling, Relational Database Service (RDS) databases, CloudWatch, deployment, data management, and security. In the concluding chapters, you'll initiate AWS automated backups and learn how to keep track of and store log files. You will also acquire a knowledge of AWS APIs and how to use them, along with CloudFormation, Python Boto3 scripts, and Terraform to automate the infrastructure.

By the end of this book, you will be confident in building up your two-tier start-up with all the infrastructure, monitoring, and logging components in place. You will also acquire knowledge of AWS APIs and how to use them, along with Python Boto3 scripts and Terraform to automate the infrastructure.

Who this book is for

This book is aimed at the following people:

System administrators and solution architects who want to build highly flexible and available AWS cloud platforms for their applicationsSoftware engineers and programmers who want to automate their AWS infrastructure using APIsIT project managers who want to understand technical aspects as well as billing requirements before adopting AWS in their organizationIT architects who want to design their infrastructure using various solutions and then come up with an optimum solution

If you are planning to use AWS in your organization, this book is for you. It will show you how to build a highly available AWS environment from scratch.

What this book covers

Chapter 1, Setting Up the AWS Environment, provides a brief introduction to various AWS offerings. It's always a good idea to get a brief introduction to the various AWS services. We will start by exploring various services using the AWS console and then set up our environment to install tools such as the AWS CLI, Boto3, CloudFormation, and Terraform, which we can use in future chapters to automate the entire infrastructure.

Chapter 2, Protecting Your AWS Account Using IAM, provides a brief introduction to IAM and an in-depth overview of IAM policies and roles. Security is job zero for all of us, so it's important to understand IAM policies, such as how to make sure we assign only the minimum privileges to a user to do their job. We will also discuss two real-world scenarios where we will see how to restrict the user to launch only a particular instance and rotate their credentials on a regular basis to reduce the risk of leaking their access and secret keys.

Chapter 3, Creating a Data Center in the Cloud Using VPC, covers building two VPCs for high-availability and disaster recovery. We will use two subnets: public for setting up two EC2 instances and private to host databases. Once the VPCs are up, we will create a transit gateway so that services in these two VPCs communicate with each other. Finally, we will look at a real-world scenario to enable VPC flow logs.

Chapter 4, Scalable Compute Capacity in the Cloud via EC2, is the last chapter of the Building Infrastructure section, where the VPCs built in the previous chapter are used to create four instances in two availability zones. We need these four instances for high availability as well as for disaster recovery. We will also explore three real-world scenarios to save costs by shutting down instances in the development environment after XPM, clean up unused Amazon Machine Images (AMIs), and remove unattached volumes.

Chapter 5, Increasing an Application's Fault Tolerance with Elastic Load Balancing, explores how, to make our application robust, we add the layer of an application load balancer in front of instances. This helps to distribute the load to the backend EC2 instances, which make the application highly available as well as serve as the single point of contact for clients.

Chapter 6, Increasing Application Performance using AWS Auto Scaling, covers setting up the on-demand scaling of our application based on criteria such as load, I/O, and network. It provides a uniform user experience to our users by spinning up the instances in the backend when the load on the application increases and similarly tears down those instances when the load is back to normal.

Chapter 7, Creating a Relational Database in the Cloud Using AWS Relational Database Service (RDS), looks at adding a database layer to our application by using AWS RDS. As databases are a critical piece of our application, we will set them up in high-availability mode, both as primary and secondary as well as read-only replicas in different AWS regions to reduce the load on the main master server.

Chapter 8, Monitoring AWS Services Using CloudWatch and SNS, looks at monitoring critical pieces to maintain the uptime of the application, such as CPU, I/O, system uptime, as well as custom metrics such as memory and disk space. In the end, we will set up an automated alarm as well as notifications via email, SMS, and Slack.

Chapter 9, Centralizing Logs for Analysis, shows how to store logs in one centralized place (CloudWatch logs) and then forward them to Elasticsearch to perform anomaly detection.

Chapter 10, Centralizing Cloud Backup Solution, looks at how to back up our instances or databases using AWS solutions (DLM snapshots) and a custom solution, such as S3 scripts.

Chapter 11, AWS Disaster Recovery Solutions, shows how to use backups to perform data recovery in case of failure. Besides that, AWS offers various disaster recovery solutions, and we will see which solution to use in which scenario.

Chapter 12, AWS Tips and Tricks, teaches you 10 tips and tricks to get the most out of AWS. Some of these tricks are based on my experience, while others are derived from AWS blogs.

To get the most out of this book

Throughout this book, we will cover several AWS examples with a number of demonstrations. As a result, I suggest using an AWS account that is not used for a production workload. To follow along, you can use any Unix-based system as all these examples are already tested on Ubuntu:

To create a new AWS account, please follow this link:

https://aws.amazon.com/premiumsupport/knowledge-center/create-and-activate-aws-account/

Installation of the AWS CLI, Boto3, and Terraform is covered in Chapter 1, Setting Up the AWS Environment.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/AWS-for-System-Administrators/.

If there's an update to the code, it will be updated on the existing GitHub repository.

Code in Action

Code in Action videos for this book can be viewed at http://bit.ly/3ptc50K.

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781800201538_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "The Principal parameter (*) used within the resource-based policies is used to identify the user, account, or role."

A block of code is set as follows:

       {

           "Sid": "Stmt1604259864802",

           "Action": "s3:*",

           "Effect": "Deny",

           "Resource": "arn:aws:s3:::myexamplebucket/*",

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

           "Condition": {

               "NotIpAddress": {

                   "aws:SourceIp": "192.168.1.10/24"

               }

           },

           "Principal": "*"

       }

Any command-line input or output is written as follows:

$ cd AWS-for-System-Administrators/Chapter4/html

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "If you are creating a new user, click on Add user."

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packt.com.

Section 1: AWS Services and Tools

This part will give you a brief introduction to various Amazon Web Services (AWS) services. After completion of Section 1, you will have skills in AWS services and understand the various ways to manage your AWS infrastructure. Then, we will look at Identity and Access Management (IAM) to fine-grain the user permissions and follow the principle of least privilege. We will then explore some real-world scenarios to solidify the concepts.

The following chapters are included in this section:

Chapter 1, Setting Up the AWS EnvironmentChapter 2, Protecting Your AWS Account Using IAM