AWS Cloud Projects E-Book

Ivo Pinto

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 297

Veröffentlichungsjahr: 2024

Beschreibung

Tired of resumes getting lost in the pile? This book is your roadmap to building an in-demand AWS portfolio that grabs attention and gets you hired.
This comprehensive guide unlocks AWS’s full potential through eight real-world projects designed for developers of all levels. Inside, you'll find invaluable guidance for crafting stunning websites with S3, CloudFront, and Route53. You’ll build robust and scalable applications, such as recipe-sharing platforms, using DynamoDB and Elastic Load Balancing. For streamlined efficiency, the book will teach you how to develop serverless architectures with AWS Lambda and Cognito. Gradually, you’ll infuse your projects with artificial intelligence by creating a photo analyzer powered by Amazon Rekognition. You’ll also automate complex workflows for seamless content translation using Translate, CodePipeline, and CodeBuild. Later, you’ll construct intelligent virtual assistants with Amazon Lex and Bedrock to answer web development queries. The book will also show you how to visualize your data with insightful dashboards built using Athena, Glue, and QuickSight.
By the end of this book, you’ll have a portfolio of AWS projects to showcase your cloud skills, making you stand out in today’s competitive job market.

Leseprobe

AWS Cloud Projects

Strengthen your AWS skills through practical projects, from websites to advanced AI applications

Ivo Pinto

Pedro Santos

AWS Cloud Projects

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Preet Ahuja

Publishing Product Manager: Prachi Rana

Book Project Manager: Ashwini Gowda

Senior Editor: Sayali Pingale

Technical Editor: Rajat Sharma

Copy Editor: Safis Editing

Indexer: Manju Arasan

Production Designer: Vijay Kamble

DevRel Marketing Coordinator: Rohan Dobhal

First published: October 2024

Production reference: 1130924

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83588-928-2

www.packtpub.com

To my amazing wife, Celia, your love and support mean the world to me. Thanks for always believing in me and for putting up with my late-night writing sessions. And to our two adorable furballs, Oscar and Tofu, your purrs and playful antics made the writing process so much more fun. Thanks for keeping me company on yet another writing journey.

– Ivo Pinto

This book is dedicated to my wife, Juliana, for her continuous support and motivation that have been a driving force behind this work, and for being a source of strength. To my mother, whose life was an example of perseverance, resilience, and determination, and for all the invaluable lessons of never giving up and always pushing forward, even in the face of adversity. And lastly, to my mentor, with whom I’m lucky enough to have had the opportunity to co-author this book.

– Pedro Santos

Contributors

About the authors

Ivo Pinto, CCIE No. 57162 (R&S, Security, and Data Center), CISSP, is a principal solutions architect with many years of experience in the fields of cloud, automation, and enterprise and data center networking. Ivo has worked at Cisco in different roles and different geographies, and he has led the architecture and deployment of many automated global-scale solutions for Fortune 500 companies that are in production today. In his latest role, he is responsible for the architecture of multiple ISV products at Amazon Web Services (AWS). Ivo has authored two books, Network Automation Made Easy and Automating and Orchestrating Networking with NetDevOps.

Pedro Santos is a senior solutions architect with over a decade of experience in the technology industry. With a background in data engineering and cloud computing, he focuses on designing and implementing innovative cloud-based solutions. After experiencing different cloud providers and companies of different sizes, Pedro is now working for AWS where he helps SaaS providers design and improve their products.

About the reviewers

Manuel Pata, at the time of this writing, is a senior technical account manager for AWS based in Portugal. In this role, he helps large ISV customers get the most out of AWS, and he loves doing that. You’ll also find him regularly working with Edge services, such as Amazon CloudFront, AWS Global Accelerator, or AWS WAF. He loves a good technical challenge and will happily spend his time prototyping solutions for his customers.

Kuldeep Singh is a seasoned project manager with over 16 years of experience in ICTS, specializing in infrastructure projects, tech upgrades, and service enhancements. He has delivered secure cloud BI platforms for banks and led data center migrations. Kuldeep has collaborated with major telecom providers globally. As a tech reviewer for Packt, he has enriched readers with insights in Cloud Native Software Security Handbook and Architecting Serverless Solutions. Currently at Birlasoft, he focuses on cloud platforms, cybersecurity, and IT governance. Kuldeep holds AWS and Azure architect certifications, is PMP certified, and is a valuable PMI member.

I extend my heartfelt thanks to my son, Aadvik, and my wife, Ankita—my sunshine on cloudy days—for their unwavering support during the creation of this book. Their patience and encouragement have been invaluable. I also wish to thank my colleagues at Birlasoft for their insights and collaboration, which have greatly contributed to this work.

Roman Ceresnak is an accomplished AWS architect and AWS Community Builder with deep expertise in cloud technologies, holding over 30 AWS certifications. With an HPD title, Roman brings a wealth of knowledge and hands-on experience in designing and implementing scalable, secure, and efficient cloud solutions. Throughout his career, he has worked extensively with a variety of advanced technologies, including AWS Glue, Apache Airflow, Grafana, Prometheus, IAM, and Kubernetes, among many others.

As an AWS Community Builder, Roman actively contributes to the AWS ecosystem, sharing his knowledge and expertise with a broader audience. He is committed to fostering a collaborative environment where professionals can learn and grow together.

I am deeply grateful to the vibrant AWS community and open source contributors who generously share their knowledge and expertise. Their collective efforts have been instrumental in shaping my journey. I owe special thanks to the AWS Community Builders program for fostering a culture of collaboration and learning.

Table of Contents

Preface

Part 1: Beginner Level Projects

1

Deploying and Interacting with AWS Services

Technical requirements

Architecting on AWS

Requirements gathering

Choosing an architecture pattern

Selecting a service

Diagramming

Exploring the Well-Architected Framework

Getting started with AWS Console

Using the Console

Navigating AWS CLI and AWS SDK

AWS CLI

AWS SDK

Understanding IaC

Using CloudFormation

Using Terraform

Summary

2

Creating a Personal Website

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon Simple Storage Service (S3)

Amazon CloudWatch metrics

Coding the solution

Editing the website

Publishing the website

Monitoring the website

Cleaning up

Future work

Implementing custom DNS

Taking security to the next level

Having better observability

Final architecture

Summary

Part 2: Intermediate Level Projects

3

Building a Recipe-Sharing Application

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon S3

Amazon CloudFront

Amazon Virtual Private Cloud (VPC)

Amazon EC2

Application Load Balancer (ALB)

Amazon DynamoDB

AWS CloudFormation

Coding the solution

Cloning the project

Section 1 – DNS configuration and certificate issuing (option 1 only)

Section 2 – Solution deployment

Section 3 – Additional configurations (option 1 only)

Section 4 – Frontend configuration and deployment

Section 5 – Testing and exploring your application

Section 6 – Cleaning up

Future work

Using secure protocols

Infrastructure auto-scaling

Managed hosting and CI/CD

Authentication

Logging and monitoring

Caching

Summary

4

Building a Serverless Recipe-Sharing Application

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon Simple Storage Service (S3)

Amazon CloudFront

Amazon DynamoDB

AWS CloudFormation

Amazon Cognito

Amazon Lambda

API Gateway

Coding the solution

Cloning the project

Solution deployment

Frontend configuration and deployment

Test and explore your application

Clean up

Future work

Enrich your application with media content

User profile

Summary

5

Implementing an Image Analyzer to Detect Photo Friendliness

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon Rekognition

Amazon API Gateway and AWS Lambda

Coding the solution

Building the infrastructure

Understanding the image analyzer code

Testing your application

Cleaning up

Future work

Implementing authentication and authorization

Improving your security posture

Implementing custom names

Improving the image analysis algorithm

Hosting your own ML model

Summary

6

Architecting a Content Translation Pipeline

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Lambda@Edge

Amazon Translate

AWS CodePipeline and AWS CodeBuild

Coding the solution

Building the web application

Building the CI/CD pipeline

Testing the solution

Cleaning up

Future work

Implementing custom names

Expanding your application functionality

Adopting CI/CD for infrastructure code

Summary

Part 3: Advanced Level Projects

7

Implementing a Chatbot Using Machine Learning

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon CloudFront and Amazon Simple Storage Service (S3)

Amazon DynamoDB

Amazon Cognito, Amazon API Gateway, and Amazon Lambda

Amazon Lex

Coding the solution

Cloning the project

Solution deployment

Frontend configuration and deployment

Amazon Lex configuration and build

Test and explore your application

Clean up

Future work

Extend the actions performed by your chatbot

Multilingual support

Maintain user profiles for different sessions

Summary

8

Building a Business Intelligence Application

Technical requirements

Scenario

Requirements

Architecture patterns

Architecture

AWS services

Amazon Simple Storage Service (S3)

AWS Glue

Amazon Athena

Amazon QuickSight

Coding the solution

Section 1 – Cloning the project

Section 2 – Solution deployment

Section 3 – Clickstream event generator

Section 4 – The Glue ETL job

Section 5 – Data exploration with Athena

Section 6 – Data visualization with QuickSight

Section 7 – Clean up

Future work

Automate the ETL pipeline

Data life cycle management

Summary

9

Exploring Future Work

Technical requirements

AWS services overview

Containers

Other API types

Generative AI

Other communication patterns

AWS Pricing Calculator

Pricing the solution from Chapter 2

Pricing the solution from Chapter 6

AWS re:Post

AWS documentation, Solutions Library and Prescriptive Guidance

AWS documentation

AWS Solutions Library

AWS Prescriptive Guidance

Summary

Index

Other Books You May Enjoy

Part 1: Beginner Level Projects

In Part 1 of this book, you are going to learn how to interact with AWS services using the AWS Console, AWS CLI, and Terraform. Then, you will build a simple web application powered by S3 and CloudFront following a step-by-step approach. These are beginner-level projects, but they will get you started on the path to autonomously building your own web applications.

This part has the following chapters:

Chapter 1, Deploying and Interacting with AWS ServicesChapter 2, Creating a Personal Website

1

Deploying and Interacting with AWS Services

Embarking on the journey to build solutions on the Amazon Web Services (AWS) platform requires a comprehensive understanding of the available tools and approaches. This chapter introduces various methodologies for architecting on AWS, beginning with preparatory activities such as requirements gathering, service selection, and diagramming.

You will then explore the various methods and tools available for deploying and interacting with AWS services, including the AWS Console, AWS Command Line Interface (CLI), AWS Software Development Kits (SDKs), and Infrastructure as Code (IaC).

This is a theoretical chapter, structured around the following main topics:

Architecting on AWSGetting started with AWS ConsoleNavigating AWS CLI and SDKUnderstanding IaC

By the end of this chapter, you will possess the knowledge and skills necessary to create, operate, and monitor AWS services using the approach that best aligns with your requirements and preferences, whether it be through the user-friendly AWS Console, the CLI, programmatic access via SDKs, or the powerful IaC tools.

Technical requirements

Although this is a theoretical chapter, you will find code snippets in the GitHub repository of this chapter at https://github.com/PacktPublishing/AWS-Cloud-Projects/tree/main/chapter1/code.

To follow along, you will need an AWS account.

Architecting on AWS

Architecting on AWS refers to the process of designing and planning cloud-based solutions using AWS. It involves understanding the various AWS services, their capabilities, and how they can be combined to build scalable, secure, and cost-effective architectures.

When architecting on AWS, the following four aspects should be considered, each detailed later in this chapter:

Requirements gathering: This is a crucial step in the process of architecting solutions on AWS. It involves understanding the business needs, functional requirements, non-functional requirements, and constraints that will shape the design and implementation of the AWS architecture.Architecture patterns: AWS provides various architecture patterns and reference architectures that serve as starting points for common use cases, such as web applications, data processing pipelines, or serverless architectures. You can leverage these patterns and customize them to meet their specific requirements.Service selection: AWS offers a broad range of services, including compute, storage, databases, networking, analytics, machine learning, and more. You must carefully evaluate the requirements of the applications and select the appropriate AWS services that best fit those needs.Diagramming: Creating visual representations of the proposed architecture is a crucial step in the architecting process. There are no AWS official tools, but draw.io or simply Microsoft PowerPoint can be used to create architecture diagrams, which helps communicate the design and facilitate collaboration and implementation.

Let’s look at these aspects in detail.

Requirements gathering

Clear and well-defined requirements are crucial for architects to design AWS solutions that meet the specific needs of the organization and provide the desired outcomes. Gathering requirements can involve collaborating with stakeholders, conducting workshops, analyzing existing systems and data, and understanding the business context.

However, if your project has a smaller scope, not all these steps may apply. Nonetheless, it is important to understand what type of requirements can be gathered before a project starts:

Business requirements: The first step is to understand the business objectives, goals, and drivers behind the solution being architected. This includes factors such as the target market, expected growth, revenue models, and any specific business constraints or regulations that need to be considered.Functional requirements: These requirements define the specific features, functionalities, and capabilities that the solution must provide. This could include requirements related to user interfaces, data processing, integration with existing systems, or specific business logic.Non-functional requirements: Non-functional requirements define the qualitative attributes that the solution must possess, such as performance, scalability, availability, security, and compliance. These requirements are often critical in determining the appropriate AWS services and architectural patterns to be used.Technical requirements: Technical requirements encompass the specific technologies, programming languages, frameworks, and tools that need to be used or integrated with the AWS solution. This could include requirements for specific databases, messaging systems, or third-party services.Data requirements: Understanding the data requirements is essential when architecting on AWS. This includes the types of data (structured, unstructured, or semi-structured), data volumes, data sources, data processing needs, and any specific data governance or compliance requirements.Integration requirements: If the AWS solution needs to integrate with existing on-premises systems, third-party services, or other cloud environments, the integration requirements must be clearly defined. This includes identifying the integration points, data formats, protocols, and security considerations.Security and compliance requirements: Depending on the industry and the nature of the data being handled, there may be specific security and compliance requirements that need to be addressed in the AWS architecture. These could include regulatory standards, data protection laws, or industry-specific certifications.Financial requirements: AWS offers a pay-as-you-go pricing model. Understanding the budget constraints and cost requirements is essential for selecting the appropriate AWS services and implementing cost-effective architectures.

Bear in mind that some folks consider costs or security requirements part of the umbrella of functional and non-functional requirements. Don’t be pedantic about naming; just gather all your requirements.

Choosing an architecture pattern

Architecture patterns and reference architectures serve as starting points for designing and implementing cloud-based solutions. These patterns encapsulate best practices, proven designs, and architectural principles tailored to specific use cases and requirements. You can find many of these in the AWS Architecture Center: https://aws.amazon.com/architecture.

By leveraging AWS architecture patterns and reference architectures, you can build upon proven designs, accelerate the development process, and ensure that your solutions align with AWS best practices and industry standards.

Architecture patterns address common scenarios and requirements. These include patterns for web applications, data processing pipelines, serverless architectures, microservices, event-driven architectures, and more. You can leverage these patterns as a foundation and customize them to meet specific needs.

In addition to general patterns, AWS provides reference architectures for specific domains and industries, such as e-commerce, media and entertainment, healthcare, financial services, and more. These reference architectures offer detailed guidance on how to design and implement solutions using AWS services and best practices specific to those domains.

To select a pattern or architecture, you must carefully evaluate the requirements, constraints, and use cases of their solutions to select the most appropriate one. This selection process involves understanding the strengths, weaknesses, and trade-offs of each pattern, as well as their alignment with the rest of the technical stack. This is often detailed in their description.

While architecture patterns provide a solid starting point, they are rarely implemented as-is. You must customize and adapt the patterns to fit your specific requirements, integrating additional AWS services, adjusting configurations, and incorporating security, monitoring, and operational considerations.

It can also happen that what you need is a hybrid or multi-pattern architecture. Some solutions require a combination of multiple architecture patterns or a hybrid approach that combines components of different patterns. There is an extra challenge in determining how to effectively integrate and orchestrate the different patterns into a cohesive and scalable architecture. This is an advanced topic, which you will learn more about in later chapters of this book.

Selecting a service

By now, you have a well-defined problem and a generic architectural pattern. The next step is service selection. This is a critical aspect of architecting solutions in AWS. With over 200 services available, AWS provides a vast array of building blocks that can be combined to create scalable, secure, and cost-effective architectures.

Service selection is an iterative process that involves balancing architectural best practices and all kinds of requirements: non-functional, functional, cost, security, and so on. You must continuously evaluate and refine service selections as the solution evolves and new requirements or constraints emerge.

The first step in service selection is to map the gathered requirements to the available AWS services. This involves understanding the capabilities and use cases of each service and identifying the ones that can address the specific functional, non-functional, and technical requirements of the solution. To make this mapping, you will need to first understand what category of service you should look at. Services are organized into different categories, such as compute, storage, databases, networking, security, analytics, and more. You will see different services in each of these categories in the following chapters of this book.

After you have identified the service category that maps to your requirements, you will need to evaluate the different service capabilities. Each AWS service offers a unique set of capabilities and features. For example, if the solution requires a highly available and scalable database, services such as Amazon RDS or Amazon Aurora might be suitable choices. This will become clearer as you advance through the chapters of this book.

Some interesting non-functional capabilities that you should consider are as follows:

Service integrations: AWS services are designed to work together seamlessly. You should consider the integration points between different services and ensure that the selected services can be integrated effectively to deliver the desired functionality.Managed versus self-managed services: AWS offers both managed services, where AWS handles the underlying infrastructure and maintenance, and self-managed services, where the customer has more control but also more responsibility. You must evaluate the trade-offs between these service types based on factors such as operational overhead, cost, and compliance requirements.Pricing and cost optimization: AWS services have different pricing models, and you must consider the cost implications of their service selections. Cost optimization strategies, such as leveraging reserved instances, spot instances, or auto-scaling, should be evaluated and incorporated into the architecture.Roadmap: AWS services are constantly evolving, with new features and services being released regularly. You should consider the future roadmap of the services they select and ensure that the architecture can accommodate potential changes or new service offerings.

Important note

Did you know that not every AWS service is available in all regions? That’s right. AWS services are not uniformly available across all AWS regions. You must therefore also consider the regional availability of the services you plan to use.

Sometimes, you will not find a suitable AWS service for your requirements and that is okay. This is where third-party services come in. Don’t be scared to leverage third-party tooling if it fits your needs. However, consider all the previously mentioned dimensions such as cost or service integrations.

Diagramming

Visual representations of the proposed architecture help communicate the design, facilitate collaboration among team members, and ensure a shared understanding of the solution’s components and their interactions. It also provides a map of the implementation.

There is no standard tool to diagram AWS solutions. The most well-accepted tools are PowerPoint using AWS Architecture Icons (https://aws.amazon.com/architecture/icons/) or draw.io (https://app.diagrams.net). These tools include icons and shapes that represent the different AWS services.

Figure 1.1 shows a high-level diagram, using PowerPoint, of the communication flow between two EC2 instances in different regions.

Figure 1.1 – Cross-region EC2 communication flow

Diagrams serve as a universal language for communicating the architecture design to stakeholders, developers, operations teams, and other involved parties. You should not only represent the various AWS services, but also their relationships, as well as the overall flow of data and processes within the solution. You can create several diagrams for the same solution, each with a different level of detail.

Important note

Diagramming is not over in the design phase. Rather, it is an ongoing process throughout the architecture life cycle. Regularly updating and reviewing diagrams ensures that they accurately reflect the current state of the architecture.

Diagrams are an essential part of the architecture documentation process. They serve as a reference for the design decisions, component interactions, and rationale behind the chosen architecture. However, documentation is not limited to diagrams. Creating thorough documentation is invaluable for future maintenance, troubleshooting, and knowledge transfer within the team or organization. This book will not cover in-depth documentation because it is a practical book focused on hands-on building.

Exploring the Well-Architected Framework

AWS provides a set of best practices and design principles known as the Well-Architected Framework (WAR), (https://aws.amazon.com/architecture/well-architected/). This framework covers six pillars: operational excellence, security, reliability, performance efficiency, cost optimization, and sustainability.

Cloud architects use this framework to ensure that their solutions align with AWS best practices, usually after they are built, but it can also be used during the design phase. Although the WAR could be a chapter of its own, we just want you to be aware of it for now, and see how we later refer to it while building the projects throughout this book.

Now that we’ve covered the designing and planning phase, let’s delve into the implementation phase. You will explore multiple tools while getting some hands-on experience.

Getting started with AWS Console

The AWS Console is a web-based user interface provided by AWS that allows users to access and manage various AWS services and resources through a Graphical User Interface (GUI).

The AWS Console is designed to be user-friendly and accessible from any web browser, allowing users to manage their AWS resources from anywhere with an internet connection. It provides a visual representation of AWS services and resources, making it easier for users to understand and interact with the AWS ecosystem.

Using the Console

During this section, you will deploy an EC2 instance, which is an AWS virtual machine.

To use the Console, you must have an AWS account. The creation process of an AWS account is outside the scope of this book, but you can find all the necessary information on the AWS website at https://aws.amazon.com/free.

Important note

Every example in this book assumes that you have a standalone AWS account that is not part of AWS Organizations.

Let’s get started:

The first step to access the AWS Console is to navigate to https://console.aws.amazon.com/.

You will be greeted by a login screen like the one shown in Figure 1.2.

Figure 1.2 – AWS Console login screen

If you are using a root user, insert the email of your user followed by the password. If you are using an IAM user, you will also need to input the 12-digit account ID.Upon successful login, you will see the AWS Console home page, as shown in Figure 1.3.

Figure 1.3 – Console Home

Sections worth highlighting for this figure are as follows:

The Search bar at the top, which you can use to search for a specific service.The AWS Region you are currently managing shows at the top-right corner; it’s N. Virginia in this case.The user or role you are currently logged in as, which is also in the top-right corner, hidden under the red square.The recently visited section, which will be empty if you haven’t opened the AWS Console before.

Important note

Why are the role and account ID hidden under a red square in Figure 1.3? Although AWS account IDs, users, and roles are not considered sensitive, it’s an AWS best practice not to publicly disclose them.

Navigate to the EC2 service console. To do this, enter ec2 in the search bar and select EC2, as shown in Figure 1.4.

Figure 1.4 – Search for the EC2 service using the AWS Console

To launch the simplest possible virtual machine, without any customization, select Launch instance, as shown in Figure 1.5.

Figure 1.5 – EC2 dashboard

In the following menu, Launch an instance, select Proceed without a key pair (Not recommended), from the Key pair name drop-down menu. Lastly, select Launch instance again, this time on the right menu.Navigate to your running instances. You can do this by selecting Instances on the EC2 left menu or by navigating to https://console.aws.amazon.com/ec2/#Instances. You should see something similar to Figure 1.6: a running EC2 instance with a funny-looking instance ID and some other attributes.

Figure 1.6 – EC2 instance status

This is not a very useful instance because you cannot connect to it. The reason for that is you that did not select a key pair. This was just a simple demo to show you how the AWS Console works.

You can terminate the instance by selecting Terminate (delete) instance, as shown in Figure 1.7.

Figure 1.7 – Terminate EC2 instance using the AWS Console