Google Cloud Certified Professional Cloud Developer Exam Guide E-Book

Sebastian Moreno

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 390

Veröffentlichungsjahr: 2021

Beschreibung

Google Cloud Platform is one of the three major cloud providers in the industry, exhibiting great leadership in application modernization and data management. This book provides a comprehensive introduction for those who are new to cloud development and shows you how to use the tools to create cloud-native applications by integrating the technologies used by Google.
The book starts by taking you through the basic programming concepts and security fundamentals necessary for developing in Google Cloud. You'll then discover best practices for developing and deploying applications in the cloud using different components offered by Google Cloud Platform such as Cloud Functions, Google App Engine, Cloud Run, and other GCP technologies. As you advance, you'll learn the basics of cloud storage and choosing the best options for storing different kinds of data as well as understand what site reliability engineers do. In the last part, you'll work on a sample case study of Hip Local, a community application designed to facilitate communication between people nearby, created by the Google Cloud team.
By the end of this guide, you'll have learned how to design, develop, and deploy an end-to-end application on the Google Cloud Platform.

Leseprobe

Google Cloud Certified Professional Cloud Developer Exam Guide

Modernize your applications using cloud-native services and best practices

Sebastian Moreno

BIRMINGHAM—MUMBAI

Google Cloud Certified Professional Cloud Developer Exam Guide

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Vijin Boricha

Publishing Product Manager: Shrilekha Inani

Senior Editor: Alexander Powell

Content Development Editor: Mrudgandha Kulkarni

Technical Editor: Nithik Cheruvakodan

Copy Editor: Safis Editing

Project Coordinator: Shagun Saini

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Ponraj Dhandapani

First published: July 2021

Production reference: 1070721

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80056-099-4

www.packt.com

To my girlfriend, Belén Sarrás, for her love, support, and inspiration.

To my family, Luisa Espinosa, Francisca Moreno, Josefa Saavedra, and Puppy, for showing me how talent and creativity evolve.

To my friends, James Martinson, Nicolás Nilo, and Rudighert Schalchli, for their unconditional support.

– Sebastian Moreno

Contributors

About the author

Sebastian Moreno is a former Google Developer Expert for Google Cloud Platform. He is working on multiple projects using cloud technologies, such as cloud innovation, virtual agents, application modernization, and data analytics. He has more than eight cloud certifications with multiple cloud providers. He is a Google Cloud Certified Professional Cloud Developer.

About the reviewer

David Saleh is an IT professional and holds a BSc. in electrical engineering and other professional certifications in cloud architecture, data engineering, and cybersecurity.

With over 20 years of experience in IT, including software architecture and development, David is currently the director of cloud architecture at one of the largest banks in Canada. He has extensive competencies in cloud-native application design and development, enterprise architecture, and cybersecurity.

To my parents, who raised me to be who I am today. You are always there for me, you are amazing, and your efforts have not gone unnoticed. I greatly appreciate and love you.

Table of Contents

Preface

Section 1: Welcome to the Google Cloud Developers' Guide

Chapter 1: Google Cloud Platform Developer Fundamentals

Technical requirements

The basics that every developer should know about Google Cloud infrastructure

Regions and zones

What is X as a Service?

How to reduce latency to your end users

Graceful shutdowns

Top tips for developing and implementing resilient and scalable applications

Microservice ecosystems

Handling user sessions and the importance of stateless applications in the autoscaling world

Application logging, your best friend in error troubleshooting

Why should your microservices handle retries?

How to handle high traffic with autoscaling

Avoiding overload caching your data

Loosely coupled microservices with topics

Don't waste your time – use cloud management services and securely run your applications

Don't reinvent the wheel

Accessing services in a secure way

Summary

Chapter 2: Security Fundamentals and Best Practices

Technical requirements

Reducing the attack surface with POLP

POLP

How to authenticate client-to-service and service-to-service solutions

IAM

IAM hierarchy

Service accounts

Authenticating with Google services

OAuth 2.0

Identity-Aware Proxy

Managing secrets with Google Secret Manager

How to store your sensitive data in a secure way

Google Secret Manager

Cloud Key Management Service

Google Cloud best practices

POLP and roles

Create one service account per microservice

Avoid using Owner, Editor, and Viewer roles

allUsers versus allAuthenticatedUsers

Understand how the IAM hierarchy works

Use Google management keys whenever possible

Use OAuth 2.0 instead of sharing credentials between applications

Use IAP when possible to authenticate and authorize users inside your organization

Always use a key vault for the storage of secrets

Summary

Section 2: Developing and Modernizing Applications on Google Cloud Platform

Chapter 3: Application Modernization Using Google Cloud

Technical requirements

Preparing your environment for developing applications on Google Cloud

Emulating Google Cloud services for local application development

Creating Google Cloud projects

Using the command-line interface withCloud SDK

Using developer tooling

Be sure that your application will work fine on the cloud with testing

Unit testing

Integration testing

End-to-end testing

Load testing

Improving your delivery speed with continuous integration and delivery pipelines

Source control management

Creating secure container images from code

Application modernization best practices

Modern application design

Refactoring a monolith to microservices

Summary

Chapter 4: Using Cloud Functions and Google App Engine

Technical requirements

Welcome to serverless applications

What is serverless?

Advantages and disadvantages of serverless applications

Event-based applications

What is Google App Engine?

Introducing Google Cloud Functions

Different flavors for different situations

HTTP functions

Background functions

Triggering cloud functions from Cloud Storage

Triggering cloud functions from Pub/Sub

Triggering cloud functions from Firestore

Terminating HTTP functions

Terminating background cloud functions

App Engine standard environment

App Engine flexible environment

Exposing your services to the internet

Google App Engine invoked via HTTP

Cloud functions invoked via HTTP

Deployment and security considerations

Location considerations

Securing cloud functions

Securing App Engine

How to make a canary release strategy with A/B testing

Static versus dynamic data considerations

Using traffic splitting in App Engine

Summary

Chapter 5: Virtual Machines and Container Applications on Google Cloud Platform

Technical requirements

Introduction to Docker and Kubernetes

What is Docker?

What are containers and Docker container images?

What is Kubernetes?

Clusters, nodes, and pods in Kubernetes

How to use virtual machines on GCP

Google Compute Engine fundamentals

Managing service accounts for VMs

Bootstrapping applications

Managing Compute Engine VM images

Reading instance metadata to obtain application configuration

Forgetting the complexity of managing a Kubernetes cluster with GKE and Cloud Run

GKE fundamentals

Cloud Run fundamentals

Configuring Kubernetes namespaces

Pod life cycle and resource configuration

Managing Kubernetes RBAC and Google Cloud IAM relationships

The full deployment life cycle of an application on Google Compute Engine

Installing an application in a virtual machine

Creating an instance template

Creating a managed instance group

Configuring a load balancer

Hands-on tutorials in Google Compute Engine

The full deployment life cycle of an application on Google Kubernetes Engine and Cloud Run

Building a container image using Cloud Build

Defining workload specifications

Deploying a containerized application to GKE

Deploying a containerized application to Cloud Run

Hands-on tutorials in Cloud Run

Summary

Chapter 6: Managing APIs on Google Cloud Platform

Technical requirements

The basics before implementing an API management solution in Google Cloud

APIs

API management

The most common authentication methods

OpenAPI Specification

Swagger UI

Swagger Editor

How to protect your APIs using Cloud Endpoints on your deployments

ESP implementation for Cloud Endpoints

OpenAPI configuration for Cloud Endpoints

Discovering the enterprise API management world with Apigee

Deploying a serverless API using Cloud Functions

Implementing an API management solution with Apigee

Adding policies

Creating an API product

Creating a developer app

Securing and managing your serverless workloads with the new Cloud API Gateway

Deploying a serverless API using Cloud Functions

Creating an API in API Gateway

Exposing our service

Protecting our services

API development best practices

An overview of DDD

REST

Summary

Section 3: Storage Foundations

Chapter 7: Handling Unstructured Data

Technical requirements

Different real-world use cases for cloud storage

Worldwide delivery

Analytics

Backup

Disaster recovery

Audit

Optimizing costs regarding storage use cases

How to reduce costs using different tiers

Standard

Nearline

Coldline

Archive

Explaining storage pricing

Cloud storage summary diagrams

Backing up and deleting files automatically

Managing life cycles

Rule conditions

Implementing object life cycles

Object versioning

Bucket retention policy

Object hold protection

Protecting my data with security best practices

Access control

Data encryption

Signed URLs

Integrating cloud storage with my application

Storing and retrieving objects from cloud storage

How to avoid CORS problems

Summary

Chapter 8: Databases and Event Messages in Google Cloud

Technical requirements

First steps in the Cloud SQL and NoSQL world with Firestore

What is Cloud SQL?

Firestore

Differences in designing and integrating a SQL database versus a NoSQL database in your application

SQL versus NoSQL databases

Which database is better for my workload?

Traditional schema versus semi-structured database in Firestore

Understanding how to connect to a Cloud SQL instance and the Firestore dashboard

Creating and connecting to a Cloud SQL instance

Configuring a Firestore instance and viewing the Firestore dashboard

Decoupling applications using Pub/Sub

Pub/Sub types

Pub/Sub

Pub/Sub Lite

Relationship options

Publishing a message

Consuming a message using pull and push subscriptions

Creating, updating, and showing data in your frontend application

Firestore dependencies

Creating a document

Updating a document

Transactions

Batch writes

Getting a document from a collection

Getting a list of documents from a collection

Query cursors

Compound queries

Sub-collection queries

Deleting a document

Some points to remember when using Firestore

Summary

Chapter 9: Data Management and Database Strategies

Technical requirements

How to decide which database fits your needs

Cloud SQL

Cloud Spanner

Bigtable

Firestore

Choosing the right database

Data management strategies for your databases in Google Cloud

What is database sharding?

What is hot-spotting, and why should you avoid it?

Defining a key structure for high-write applications in Bigtable

Key design in Cloud Spanner

Defining keys for Firestore to avoid hot-spotting

Database and application management best practices

Keep your Cloud SQL small

Be prepared to fail

Improve the speed of your database to import data

Have a recovery plan

Optimize query execution in Cloud Spanner

Optimize bulk loading in Cloud Spanner

Optimize update operations in Cloud Spanner

Bigtable test best practices

Firestore location cases

Avoid hot-spotting due to read and write operations in collections

Reading and writing documents from Firestore

Reduce costs and remove limits with indexing exemption

Summary

Chapter 10: Optimizing Applications with Caching Strategies on Google Cloud Platform

Technical requirements

How, when, and why you should use a caching strategy in your applications

Why is having a cache strategy important?

When can we implement a cache strategy?

How can we implement a caching strategy?

Handling high traffic volumes with Memorystore for Memcached

Creating a Memcached instance

Connecting to a Memcached instance

Optimizing your application with Memorystore for Redis

Creating a Redis instance

Connecting to a Redis instance

Summary

Section 4: SRE for Developers

Chapter 11: Logging on Google Cloud Platform

Technical requirements

Introduction to Cloud Logging, the logging solution of GCP

Learning logging best practices in the cloud

Use libraries in your application to record logs

Don't create all logs on a single level

Categorize your logs correctly

Add meaningful descriptions

Make your logs understandable to both humans and machines

Add unique event IDs

Review your logging ingestion metrics

Exclude logs for cost optimization

How to enable an application to create custom logs

Enabling an app to create custom logs

Using Cloud Storage for cost optimization

Using Pub/Sub to trigger events

Using BigQuery for logs analysis

How to watch logs and make advanced logging queries on Cloud Logging

Our first search query

Advanced queries

Real-case logging examples using Cloud Logging and Cloud Monitoring

Centralization of multiple logs in a GCP project

Automatic error alerts

Summary

Chapter 12: Cloud Monitoring, Tracing, and Debugging

Technical requirements

Your first operational dashboard and metrics

Creating your first workspace

Creating your first dashboard

Monitoring your application uptime 24/7

Creating an uptime check

Finding bugs in your application with cloud debugging

Creating our base application

Using Cloud Debugger

Creating a snapshot

Creating a logpoint

Optimizing your application with cloud tracing and profiling

Cloud Trace

Cloud Profiler

Using Cloud Trace in our application

Using Cloud Profiler in our application

Real case examples using cloud monitoring, debugging, and tracing for applications on GCP

Cloud monitoring SLOs

Cloud Debugger in the cloud

Cloud Trace cases

Official documentation resources

Summary

Section 5: Analyzing a Sample Case Study

Chapter 13: HipLocal Sample Case Study Preparation

Technical requirements

Pro tips to ace the sample case study

Sample case structure

How to order the case information

Map services and technologies with Google Cloud

Reviewing the executive statement, existing solution, and requirements

Executive statement

Existing technical environment

Reviewing the technical requirements

Reviewing the business requirements

Architecting the solution

Summary

Chapter 14: Questions and Answers

Technical requirements

Questions

Google Cloud Platform (GCP) developer fundamentals

Security fundamentals and best practices

Application modernization using Google Cloud

Using Cloud Functions and GAE

Virtual machines (VMs) and container applications on GCP

Managing APIs on GCP

Handling unstructured data

Databases and event messages in Google Cloud

Data management and database strategy

Optimizing applications with caching strategies on GCP

Logging on GCP

Cloud monitoring, tracing, and debugging

Answers

GCP developer fundamentals

Security fundamentals and best practices

Application modernization using Google Cloud

Using Cloud Functions and GAE

VMs and container applications on GCP

Managing APIs on GCP

Handling unstructured data

Databases and event messages in Google Cloud

Data management and database strategy

Optimizing applications with caching strategies on GCP

Logging on GCP

Cloud monitoring, tracing, and debugging

Summary

Other Books You May Enjoy

Preface

Google Cloud Certified Professional Cloud Developer Exam Guide is a book for learning more about Google Cloud services used by those working in a cloud developer role on a daily basis. Covering the most important services and best practices, this book will help you to design and develop cloud-native applications, reduce operational tasks, and improve the deployment speed of new features.

This content will allow you to prepare for the Google Cloud Certified Professional Cloud Developer exam by reviewing the entire exam guide through the course of the book, providing sample questions, and by covering a sample case study included in the exam.

Who this book is for

This book is intended for developers, cloud engineers, and anyone preparing for the Google Cloud Certified Professional Cloud Developer exam. Basic to intermediate development skills are required to make the best of this book.

What this book covers

Chapter 1, Google Cloud Platform Developer Fundamentals, covers the fundamentals for developers and good programming cloud practices on Google Cloud.

Chapter 2, Security Fundamentals and Best Practices, covers the essentials tips and acceptable security practices for your cloud applications.

Chapter 3, Application Modernization Using Google Cloud, covers how to take advantage of all the cloud benefits, such as autoscaling, fully managed services, and cost saving.

Chapter 4, Using Cloud Functions and Google App Engine, covers resource optimization and cost saving using two of the most popular serverless solutions on Google Cloud.

Chapter 5, Virtual Machines and Container Applications on Google Cloud Platform, covers three different compute options for deploying containerized applications on Google Cloud.

Chapter 6, Managing APIs on Google Cloud Platform, covers how to implement, expose, manage, and secure APIs on Google Cloud using three different services.

Chapter 7, Handling Unstructured Data, covers how to store and consume unstructured data in an efficient way using Google Cloud services.

Chapter 8, Databases and Event Messages in Google Cloud, covers topics, SQL, and NoSQL database services on Google Cloud for storing your data and decoupling your applications.

Chapter 9, Data Management and Database Strategies, covers different use cases and best practices regarding data management in the cloud.

Chapter 10, Optimizing Applications with Caching Strategies on Google Cloud Platform, covers how to create, implement, and manage caching strategies using Google Cloud services.

Chapter 11, Logging on Google Cloud Platform, covers how to detect errors and diagnose issues in your application faster using Google Cloud services.

Chapter 12, Cloud Monitoring, Tracing, and Debugging, covers how to create dashboards, automated alerts, and trace, profile, and debug applications to understand your application's issues and optimization opportunities.

Chapter 13, Hip Local Sample Case Study Preparation, covers a detailed review of the sample case study with a draft of a solution that meets the needs of the case.

Chapter 14, Questions and Answers, covers questions and answers related to the public exam guide for the Google Cloud Certified Professional Cloud Developer.

To get the most out of this book

To get the most out of reading this book, install the following software in order to be able to run the multiple code examples from each chapter:

You can find the instructions to install Node by following this link: https://nodejs.org/en/.

You can find the instructions to install the Google Cloud SDK by following this link: https://cloud.google.com/sdk.

If you are using the digital version of this book, we advise you to type the code yourself or access the code via the GitHub repository (link available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Google-Cloud-Certified-Professional-Cloud-Developer-Exam-Guide. In case there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781800560994_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Where YOUR_IMAGE is the path to the image of your application in Google Container Registry."

A block of code is set as follows:

{

  "steps": [

    {

      "name": "gcr.io/cloud-builders/gke-deploy",

      "args": [

        "run",

        "--filename=kubernetes-resource-file",

        "--location=location",

        "--cluster=cluster"

      ]

    }

  ]

}

Any command-line input or output is written as follows:

gcloud iam service-accounts set-iam-policy sa-id policy-file

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "In the Host and path rules and Frontend configuration sections, we do not make any changes."

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you've read Google Cloud Certified Professional Cloud Developer Exam Guide, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.

Section 1: Welcome to the Google Cloud Developers' Guide

In this section, you will understand the basic programming concepts to consider when developing in Google Cloud and the most crucial security fundamentals and best practices.

This section comprises the following chapters:

Chapter 1, Google Cloud Platform Developer FundamentalsChapter 2, Security Fundamentals and Best Practices

Chapter 1: Google Cloud Platform Developer Fundamentals

In this first chapter, you will learn about the fundamentals and best practices of developing in Google Cloud, and how these differ from traditional on-premises application development. You will learn about concepts that allow you to take your developments to decoupled, resilient, highly available, and scalable architectures, in addition to delegating the most significant responsibilities to Google's self-managed services in a secure way.

In this chapter, we're going to cover the following main topics:

The differences between IaaS, CaaS, PaaS, and serverlessThe fundamentals of a microservices ecosystemDelegating responsibilities to Google Cloud services

Technical requirements

There are no specific technical requirements for this chapter.

The basics that every developer should know about Google Cloud infrastructure

There are many people who think that programming in the cloud is simply a matter of programming in another environment, but I could not disagree more with that statement.

Depending on which cloud service you use, for example, a virtual machine, Platform as a Service (PaaS), or Function as a Service (FaaS), your code could need to handle life cycles and unexpected program terminations.

When you program in the cloud, you should pay attention not only to good coding but also to which platform or service you are using, since many things can change depending on this factor.

For example, programming in an IaaS service is different from programming in a CaaS or FaaS service. (We will explain what these acronyms mean very shortly.)

In this section, you will learn about the most important differences between the different services offered in the cloud, what the regions and zones are, and why concepts such as high availability and latency are so important in a cloud solution.

Regions and zones

A region is a specific location where you can choose to host your services and computing resources with one or more zones.

A compute cluster (a layer between regions and zones) is a physical infrastructure in a data center with independent power, cooling, network, and security services.

A zone can be hosted in one or more clusters and allows the resource load to be handled and balanced within a region.

Choosing multiple zones and regions allows the application to reduce latency to final users and handle failures, transforming your application into a high-availability service.

If a specific zone presents an outage, your application can keep operating using another zone.

If a specific region presents an outage, your application can keep operating using another zone in another region.

Having instances of your application in multiple regions and zones increases your costs as well as your application's availability.

What is X as a Service?

One of the most important things before starting development in the cloud is to understand what the different types of service are and how the shared responsibility model works. But when we start working in the cloud, we see acronyms everywhere, such as the following:

IaaS: Infrastructure as a ServiceCaaS: Container as a ServicePaaS: Platform as a ServiceFaaS: Function as a ServiceSaaS: Software as a Service

We will start with IaaS. In this case, the cloud provider gives you an infrastructure, generally represented as a virtual machine with an operating system based on a virtualized image. In this kind of service, there is a charge for use. An example of IaaS on Google Cloud Platform (GCP) is GCE, or Google Compute Engine.

In the case of CaaS, the cloud provider gives us an environment where we can deploy our application images. In this kind of service, there is a charge for use.

With PaaS, the cloud provider will provide us with a platform where we can load a previously compiled artifact and configure exposure rules for services and versions. In this kind of service, there is a charge for use. An example of PaaS on GCP is Google App Engine.

If we decide to use FaaS, the cloud provider will give us a platform where we will only have to code and configure the corresponding dependencies, without the need to compile or generate an image. In this kind of service, there is a charge for the number of executions and the time of these executions. An example of FaaS on GCP is Cloud Functions.

And finally, in the case of SaaS, the cloud provider will deliver the software in such a way that the user can directly consume the functionality. For this kind of service, there may be a charge for use or a subscription. An example of SaaS on GCP is Data Studio:

Figure 1.1 – From IaaS to SaaS

When you choose to code using IaaS, you should take care of operating system patches and updates, accepting more responsibilities but also having more options to customize your server. That means more management tasks.

When you choose to code using PaaS, you only need to take care of the business logic of your application, accepting fewer infrastructure responsibilities but also having fewer options to customize your server.

As explained in the introduction to this chapter, each service involves different considerations when we start programming.

Among the most important concepts from the preceding list of services is the control of the execution life cycle of our applications. This was not a concept we needed when we programmed on IaaS, because normally, the servers were always on and it was not necessary to worry about telling the server that our asynchronous executions had finished successfully to turn it off.

This is just one of many points that we will review in this chapter and that will help you to program applications in the different GCP services.

How to reduce latency to your end users

Generally, when we have to solve a problem through coding, the first thing we worry about is that the code does what it has to do, no matter how we achieve it. After this, we focus on ensuring that the code does not have any security vulnerabilities, and finally, we might try to optimize or refactor various methods or functions in the interest of efficiency.

In some cases, this is not enough, and we must plan not only how our code will run in the cloud but also where it will run.

You will probably wonder why some streaming or video services are so fast and others so slow. Various factors may be responsible, such as the speed of the user's internet connection, which unfortunately is not something we can control. However, there is a factor that we can control, and that is how close we can bring the content to our end users.

When a user enters a web page to consume content, and in this particular case we are going to assume that the user decides to consume a video, the user has to go to the source, in this case, the server, to access the content.

Depending on how far the user is from the server where the video is stored, the speed with which they download the content to their computer may be higher or lower. This is known as latency.

The distance between the source and the consumer directly affects the latency, and that is why the closer we bring the source to the consumer, the lower the latency and the higher the speed of consumption of the information.

In GCP (and in most clouds), there is the concept of a Content Delivery Network (CDN), which acts as a content cache. This means that the content is replicated on a different server than the originating one, in order to reduce the requests to the original server and also, in this case, to bring the content closer to the end consumer in order to increase performance.

When content is consumed for the first time, Cloud CDN, the Google Cloud solution CDN implementation, will consult the content on the server to find the source of origin and will replicate it in its nodes so that in future requests, the content is available to be delivered directly to users.

For this feature, Cloud CDN uses a cache key, based on the query URL, to determine whether or not the content the user is trying to access is already replicated on the CDN nodes. This is called a cache hit. When the content is not found, this action is called a cache miss. If you need to configure how long the cache content will exist on the CDN node before revalidating the content at the origin, you can use the Time to Live (TTL) configuration in seconds. The default TTL for content caching is 3,600 seconds (1 hour) and the maximum allowed value is 1 year:

Figure 1.2 – CDN workflow

In summary, Cloud CDN is a solution that allows us to bring content efficiently from the source to consumers, such as images, videos, and files, by replicating the content in different nodes worldwide in order to reduce response times for the final consumer.

Graceful shutdowns

In the world of microservices applications, containerized applications controlled by an orchestrator are generally used. The best-known one is Kubernetes. Kubernetes has the ability to shut down any of the existing microservices (called pods in a Kubernetes cluster) at any time in order to free up resources or maintain the health of the application.

If the design of the application does not support graceful shutdowns, we could run into problems, such as the execution of a call to our application not being completed. That is why it is important to control shutdowns gracefully via the SIGTERM execution termination code.

When our application receives the termination code from SIGTERM execution, our application must close all the connections it has open at that moment and save any information that is useful for maintaining the current state prior to the end of execution.

In Kubernetes, this can be done with the preStop hook, which we will explore in Chapter 5, Virtual Machines and Container Applications on Google Cloud Platform.

But Kubernetes workloads are not the only case where we need to handle graceful shutdowns. For example, GCP has a GCE virtual machine type called a preemptible VM instance. A preemptible VM instance can be purchased with a discount of up to 80% as long as you accept that the instance will terminate after 24 hours.

In that case, you can handle a graceful shutdown using a shutdown script. This script will be executed right before a virtual machine instance is stopped or restarted, allowing instances to perform tasks such as syncing with other applications or exporting logs.

You can also use a shutdown script when you are using other GCP solutions, such as Managed Instance Groups (MIGs), in order to gracefully shut down your instances when the MIG performs a scale-out operation (deleting instances inside the group).

Top tips for developing and implementing resilient and scalable applications

Having presented some of the fundamentals of programming in the cloud, it is important to now address one of the topics that is probably the most mentioned but least understood: microservices. In this section, we briefly introduce the microservices ecosystem and explain how to correctly manage user sessions, create traceability logs in order to detect and solve errors, implement retries in order to make our solution reliable, support consumption peaks through autoscaling, and reduce server consumption through information caching.

Microservice ecosystems

Microservices are an architectural design pattern that allows the separation of responsibilities into smaller applications, which can be programmed in different programming languages, can be independent of each other, and have the possibility of scaling both vertically (increasing CPU and memory) and horizontally (creating more instances) independently in order to optimize the cluster resources.

Although microservices have been widely accepted by both start-ups and established organizations, on account of their great advantages, such as decoupling business logic and better code intelligibility, they also have their disadvantages, which are mostly focused on problems of communication management, networking, and observability.

It is therefore important to decide whether the use of a microservices pattern is really necessary. The size of the solution and clarity in the separation of responsibilities are fundamental factors when making this decision. For simpler or smaller solutions, it is perfectly acceptable to use a monolithic-type architecture pattern, where instead of having many small, orchestrated applications, you have just one large application with full responsibility for the solution.

In Chapter 3, Application Modernization using Google Cloud, we will go into greater detail on the technologies used in GCP in order to support an ecosystem of microservices, mitigating the disadvantages mentioned previously:

Figure 1.3 – Monolith versus microservice pattern

Handling user sessions and the importance of stateless applications in the autoscaling world

Probably, if you are a developer who comes from an old-school background, you will be familiar with managing user sessions on the server side. For a long time, this was how you knew whether or not a user was already authenticated in an application, consulting the server for this information. These types of applications are called stateful.

With the advent of the cloud and the ease of being able to create and destroy instances, horizontal autoscaling became very popular, but it also brought with it a big problem for applications that already existed and were based on querying the server status.

Horizontal scaling consists of the ability to create and destroy instances of our application to deal with high demand at peak times. One example of a trigger for horizontal scaling could be the number of calls received in a specific period of time.

Stateful applications don't work well using horizontal scaling because in this kind of application, the state information is stored in the instance itself, leaving all the other instances inside the horizontal scaling group without the state information. One example of this is when a user with a user session created and stored in one instance of the application accesses another instance of the group without the information from this user session.

To solve this problem, we can use access tokens, which enable session information to be maintained by the user, not by the server. Applications that store state information outside the instance itself are called stateless.

When a user logs in for the first time, an access token is generated, which is signed by an authorizing entity with a private key (which is only stored in that server or instance) and is verified in each of the other instances or servers with a public key, to verify that the access token has been effectively generated by our application and not by a third party:

Figure 1.4 – Authentication flow

There are also access tokens of the JSON Web Token (JWT) type, which, in addition to having information about the signature of the token, can contain non-sensitive information such as the user's name, email, and roles, among other things.

In conclusion, although there are still applications that maintain their states based on the server, if an application requires horizontal scaling capabilities, it is necessary to adopt an authentication strategy using access tokens that are stored on the client side, using a stateless application solution.

Application logging, your best friend in error troubleshooting

Before the advent of microservices, there were only monolithic applications, in which the complete solution was kept in a single instance. This allowed application debugging and troubleshooting to be much simpler, since all the logs were stored in one place.

With the advent of microservices, solutions stopped relying on just one instance and this began to make debugging and troubleshooting activities more complex.

Now we need to be able to stream the logs generated by the various microservices to a centralized logging service in order to analyze all the information in one place, transforming our application to stateless.

We need also to revise the way in which logs are generated in our applications to facilitate debugging.

Now the logs have to not only indicate whether or not an error existed, but also somehow allow the generation of a trace from whichever service the call was initiated because an error now depends not only on a single instance but also on multiple actions executed on different servers.

Among the most common solutions to address these problems is the addition of custom headers to each of the calls that are generated between microservices, resulting in the traceability of the origin of the system and unique event codes:

Figure 1.5 – Trace example

These options are an excellent element in microservices ecosystems to facilitate both the debugging and troubleshooting of applications.

Why should your microservices handle retries?

In any computer system, anything can fail and that is why our application has to be prepared to control these errors. In the world of microservices, this problem is even greater since, unlike an application built with the monolithic pattern, where all communications are generated in the same instance, each of the calls between microservices can fail. This makes it necessary to implement retry policies and error handling in each of the calls that are made.

When retrying a call to a microservice, there are some important factors to consider, such as the number of times we are going to retry the call and how long to wait between each retry.

In addition, it is necessary to allow the microservice to recover in the event that it is not available due to call saturation. In this case, the truncated exponential backoff strategy is used, which consists of increasing the time between each of the retry calls to the microservice in order to allow its recovery, with a maximum upper limit of growth of the time between calls.

Some service mesh solutions (a dedicated layer to handle service-to-service communications) such as Istio come with automatic retry functionality, so there is no need to code the retry logic. However, if your microservices ecosystem does not have a service mesh solution, you can use one of the available open source libraries, depending on which programming language you are using.

Understanding that service-to-service communication can sometimes fail, you should always consider implementing retries in applications that work with microservices ecosystems, along with the truncated exponential backoff algorithm, to make your solutions more reliable.

How to handle high traffic with autoscaling

Before the arrival of the cloud, it was necessary, when starting a computer project, to first evaluate the resources that would be required before buying the server on which the application would live. The problem with this was, and still is, that estimating the resources needed to run an application is very complex. Real load tests are generally necessary to effectively understand how many resources an application will use.

Furthermore, once the server was purchased, the resources could not be changed, meaning that in the event of an error in the initial estimate, or some increase in traffic due to a special event such as Cyber Day, the application could not obtain more resources. The result could be a bad experience for users and lost sales.

This is why most of the applications that are still deployed on on-premises servers have problems when high-traffic events arise. However, thanks to the arrival of the cloud, and the ease of creating and destroying instances, we have the arrival of autoscaling.

Autoscaling is the ability of an application under a load balancer to scale its instances horizontally depending on a particular metric. Among the most common metrics to manage autoscaling in an application are the number of calls made to the application and the percentage of CPU and RAM usage:

Figure 1.6 – Scaling out or horizontal scaling

For applications implemented in services of the IaaS or CaaS type, it is necessary to consider, for the correct management of the applications' autoscaling, the warm-up and cool-down time, in addition to application health checks.

Warm-up is the time it takes for the application to be available to receive post-deployment calls. This time must be configured to avoid an application receiving calls when it is not yet available.

Cool-down is the period of time where the rules defined in the autoscaling metrics are ignored. This configuration helps avoid the unwanted creation and destruction of instances due to short variations in defined metrics such as CPU, memory, and user concurrency calls.

For example, if you have a 60-second cool-down configuration, all the metrics received in that period of time, after the creation of new instances, will not be considered to trigger the autoscaling policy. If you don't have a cool-down configuration, the autoscaling policy could trigger the creation of new instances before the warm-up of the application in those first created instances, generating the unwanted triggering of the autoscaling policy.

The health check is a status verification used for determining whether a service is available to receive calls (it is related to warming up) so that the load balancer has the possibility of sending traffic only to instances that are able to receive traffic at that time.

For applications implemented as serverless-type services, no special configuration is required, since they support the ability to auto-scale automatically.

Depending on the services supported by the cloud provider you use to implement your application, it may be necessary to perform additional configuration steps to correctly control autoscaling policies.

Avoiding overload caching your data

Although it is possible to control an increase in traffic by autoscaling the application, this action is not free, since it means both the temporary creation of new instances to be able to support this load and, in the case of serverless services, an increase in costs for each call made.

To reduce costs and also increase the performance of the calls made, it is therefore necessary to use a data caching strategy:

Figure 1.7 – Caching example

Implementing a caching strategy consists of saving in a database, in memory, queries that are made recurrently (using a key that corresponds to the query). To see whether a user returns to make a query that was made previously, it is saved in the database and thus there is no need to re-process the request, thereby reducing both response times and costs.

The first thing to do before implementing a caching strategy is to evaluate whether the scenario allows this strategy to be implemented. For this, it is necessary to identify the frequency with which changes are made to the data source that the application service consults. If the data change frequency of the source is 0 or very low, this is the ideal scenario to implement a caching strategy. Otherwise, if the data change frequency of the source is very high, it is recommended not to implement this caching strategy since the user could get outdated data.

Depending on the frequency with which the data changes in the source, it will be necessary to configure the lifetime of the cache. This is to avoid the data in the cache becoming outdated relative to the data in the source due to modification of the source, in which case users would get outdated information.

This strategy allows us to optimize our application to achieve better use of resources, increase responsiveness for users, and reduce the costs associated with both the use of resources and money for serverless solutions.

Loosely coupled microservices with topics

When designing a microservices architecture, it is important to ensure that calls made between each of the applications are reliable. This means that the information sent between microservices has retry policies and is not lost in cases of failure. It is also important to design the application solution so that if a microservice needs to be replaced, this does not affect the other microservices. This concept is known as loose coupling.

To implement a loosely coupled strategy, and also to ensure that messages sent between microservices are not lost due to communication failures or micro-cuts, there are messaging services.

In GCP, we have a messaging service based on the publisher/subscriber pattern called Pub/Sub, where the publisher makes a publication of the message to be sent to a topic, and the subscriber subscribes to a topic in order to obtain this message. There are also two types of subscriptions: pull-type subscriptions, in which the subscriber must constantly consult the topic if there is a new message, and push-type subscriptions, where the subscriber receives a notification with the message through an HTTP call:

Figure 1.8 – Loosely coupled microservices

In a loosely coupled ecosystem of microservices, each microservice should publish its calls using messages on a topic, and the microservices that are interested in receiving these calls as messages must subscribe. In Pub/Sub, upon receiving a message, the subscriber must inform the topic with an ACK, or acknowledge, code to inform the topic that the message was received correctly. If the subscriber cannot receive or process the message correctly, the topic will resend the message to the subscriber for up to 7 days in order to ensure delivery.

Each microservice that acts as a subscriber must have the ability to filter duplicate messages and execute tasks idempotently, which means having the ability to identify whether a message has already been delivered previously, so as not to process it in a duplicate way.

The use of messaging services in microservices ecosystems allows us to have robust solution designs, loosely coupled and tolerant of communication errors.

Don't waste your time – use cloud management services and securely run your applications

In this section, we review some of the existing services to which we can delegate specific responsibilities so that we can focus development efforts on what really matters, which is business logic.

We will also review how we can communicate with these services in a secure way through service accounts, and how to ensure communication between services.

Don't reinvent the wheel

GCP offers multiple services that allow us to perform tasks just by invoking the available APIs without having to code a solution completely from scratch.

In this section, we will review 10 of the most used services in GCP. In later chapters, we will review each of these services in detail. The services are as follows:

Cloud storagePub/SubCloud SQLFirestoreMemorystoreCompute EngineGoogle Kubernetes EngineGoogle Secret ManagerCloud loggingCloud monitoring

Cloud Storage is an object store that allows unlimited storage of information in different regions, storage classes (the possibility of reducing costs, changing the pricing model, and the availability of the objects required by the solution), and life cycle policy configuration for existing files. It also allows access to information in a granular way and the hosting of static data web applications.

Pub/Sub is a serverless messaging service under the publisher/subscriber pattern, used for communication between two services in a reliable and decoupled way. It has automatic autoscaling and allows messages to be stored for up to 7 days in case of subscriber failures.

Cloud SQL is a self-managed solution for Online Transactional Processing (OLTP) databases such as MySQL, PostgreSQL, and SQL Server. It allows you to configure the type of machine on which the databases will run, create read-only replicas, and generate backups on a scheduled basis.

Firestore is a serverless NoSQL database solution that allows you to store information in the form of documents based on keys and values, allowing you to access the information very quickly.

Memorystore is an in-memory database solution