Efficient Cloud FinOps E-Book

Alfonso San Miguel Sánchez

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 654

Veröffentlichungsjahr: 2024

Beschreibung

In response to the escalating challenges of cloud adoption, where balancing costs and maximizing cloud values is paramount, FinOps practices have emerged as the cornerstone of fi nancial optimization. This book serves as your comprehensive guide to understanding how FinOps is implemented in organizations worldwide through team collaboration and proper cloud governance.
Presenting FinOps from a practical point of view, covering the three phases—inform, optimize, and operate—this book demonstrates an end-to-end methodology for optimizing costs and performing financial management in the cloud. You’ll learn how to design KPIs and dashboards for judicious cost allocation, covering key features of cloud services such as reserved instances, rightsizing, scaling, and automation for cost optimization. This book further simplifi es architectural concepts and best practices, enabling you to design superior and more optimized solutions. Finally, you’ll discover potential synergies and future challenges, from the integration of artifi cial intelligence to cloud sustainability considerations, to prepare for the future of cloud FinOps.
By the end of this book, you’ll have built the expertise to seamlessly implement FinOps practices across major public clouds, armed with insights and ideas to propel your organization toward business growth.

Leseprobe

Efficient Cloud FinOps

A practical guide to cloud financial management and optimization with AWS, Azure, and GCP

Alfonso San Miguel Sánchez

Danny Obando García

Efficient Cloud FinOps

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Preet Ahuja

Publishing Product Manager: Surbhi Suman

Senior Editor: Sujata Tripathi

Technical Editor: Yash Bhanushali

Copy Editor: Safis Editing

Project Coordinator: Ashwini Gowda

Proofreader: Safis Editing

Indexer: Hemangini Bari

Production Designer: Prashant Ghare

Marketing Coordinator: Rohan Dobhal

First published: February 2023

Production reference: 1310124

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB

ISBN 978-1-80512-257-9

www.packtpub.com

To my parents, who planted the seed of writing in me, and my brothers, for their continuous support. To Beatriz, my wife and life companion, whose support and love have been essential to completing this project. To all my colleagues and team at Santander and Avanade/Accenture, from whom I learned so much, without you this wouldn’t be possible. To Danny, who has endured this adventure with me for a whole year without faltering. And last but not least, to Edgar Bahilo and my brother Ignacio San Miguel, who have kindly shared their expertise and wisdom in the last chapter of this book. And, of course, to the whole Packt team, for trusting us with this project and supporting us throughout its completion.

– Alfonso San Miguel Sánchez

To my sons, Gonzalo and Clara, whose innocence and eagerness to learn are my biggest inspirations. I want to also thank my parents, who have always supported me and shown me what respect and commitment are. To all my work colleagues who have shared this journey with me. Special mention to my friend Alfonso San Miguel, for letting me experience this adventure. And last but not least, to Eva, my wife and friend, who makes anything possible with her love and support.

– Danny Obando García

Contributors

About the authors

Alfonso San Miguel Sánchez is a multi-cloud architect, with a deep experience both on premises and in the cloud. He has always enjoyed being close to development teams, implementing coding, DevOps, and other methodologies into his way of working, with a strong focus on automation. Alfonso has a degree in computer science from Universidad Complutense de Madrid and an M.Sc. degree in machine learning. After his studies, he worked as a cloud architect for Tecnicas Reunidas, Avanade, and B2Impact, where he works now as a lead cloud architect. Though passionate about cloud governance, in the past few years, he has specialized in FinOps, aiming to develop an entire methodology around the practice.

Danny Obando García is a multi-cloud data architect, who has worked in various roles during his professional career, always aiming to create reliable and scalable data and infrastructure solutions by applying different frameworks and methodologies. Danny has a degree in computer science from Universitat Oberta Catalunya (UOC), which he complemented with an M.Sc. in artificial intelligence for financial markets. With a rich IT experience of about 15 years, he is currently leading data strategy for Holaluz, one of the biggest players in Spain’s energy market. Before this, he had experience working and implementing FinOps for the biggest banking group in Spain.

About the reviewers

Israel Pérez Jiménez has more than 20 years of experience in IT. He has worked in multiple sectors such as transportation, banking, and the engineering industry, in varied roles such as project management, IT processes consultant, infrastructure, and digital transformation. He currently works for Tecnicas Reunidas as a lead systems architect, fully focused on cloud governance, modernization, and cost optimization, where he has been for more than 9 years. Architecting solutions is part of his DNA, with a strong focus on cost, security, automation, and reliability. From traditional on-premises environments to cloud solutions, his wide experience grants him a complete vision of IT challenges.

Ismael Doblas Bermudo began his journey as a cloud engineer. Through constant learning and training, he honed his skills as an architect to build more robust and scalable architectures. His focus on multi-cloud environments allows him a panoramic view, as well as a complete vision of the cloud’s ever-evolving landscape, enabling him to navigate seamlessly across various public clouds. His knowledge of automation and IaC has also become a cornerstone of his approach, to ensure efficiency, consistency, and scalability. During the last years of his experience, during a pivotal juncture in his evolution, he chose to specialize in FinOps. He currently works as a global FinOps lead for a multinational company in the banking sector.

Eric Duquesnoy is a seasoned professional, currently the head of Cloud and DevOps consulting at ELCA Cloud Services. Based in Geneva, Eric leads strategic initiatives using his extensive expertise in cloud architecture (Azure, AWS) and FinOps. In addition to his professional activities, Eric is the founder of the Silicon Chalet meetup in Switzerland, a dynamic community shaping the future of technology. From 2019 to 2023, Eric held the position of head of Cloud at Eurovisions, where he contributed significantly to the advancement of the organization’s cloud technology. Eric holds the FinOps Practitioner certification, which highlights his commitment to excellence in optimizing cloud spend and aligning financial strategies with business objectives.

Table of Contents

Preface

Part 1: Get Started with FinOps

1

Introduction to FinOps Principles

What is FinOps, and why do we need another buzzword?

Why FinOps?

Before the cloud

The cloud comes into play

A paradigm shift

Hidden on-premises costs

Back to the present

The FinOps Foundation

The three pillars of FinOps

Inform

Example (the Inform pillar)

Optimize

Example (the Optimize pillar)

Operate

Example (the Operate pillar)

Summary

2

Understanding How FinOps Fits into Cloud Governance

The Well-Architected Framework – an introduction

FinOps as part of bigger governance

FinOps + Agile methodologies

FinOps, Infrastructure as Code, CI/CD, and DevOps

FinOps and change management

Tailoring a FinOps approach for each organization

Scenario 1 – companies not yet in the cloud or beginning their journey to it

Scenario 2 – companies already in the cloud but not mature enough or that have non-optimized workloads

Scenario 3 – big companies with strong cloud maturity

Scenario 4 – companies focused on generating cloud cost savings

Selecting the right tools for the job

Base tools

Market tools

Other interesting tools

Summary

Part 2: Inform – How to Increase Cost Visibility

3

Designing and Executing the Tagging and Naming Convention Strategies

The importance of naming conventions and tagging in FinOps

Why are naming conventions significant?

Why are tagging strategies significant?

Naming conventions versus tagging

Naming convention and tagging enforcement

Naming conventions for cloud resources

Style and format

Separators

Key fields to include

Parent and child resources

Creating a name generator

Building a tagging strategy

Style and format

Simple and compound tags

Creating a tagging strategy

Automated tagging

Cost allocation

Summary

4

Estimating Cloud Solution Costs and Initiative Saving

Technical requirements

How to calculate the TCO for cloud solutions

TCO introduction

Cloud pricing calculators

Pricing APIs from cloud providers and how to work with them

Pricing APIs overview

Estimating potential savings of cost optimization initiatives

How to automate cost estimation

Data sources selection

Data consolidation

Estimation calculation

Change notification

Data update

Summary

5

Improving Cost Visibility with Dashboards and Reports

Understanding cloud invoices and billing data

Dashboards and reports

The main differences between a report and a dashboard

Key benefits

Dashboards from another view – simulators

How to prepare cost evolution reports and dashboards and their importance

Financial basics

Tracking savings to initiatives and adding milestones

Unit economics

How to prepare FinOps dashboards and reports

Existing dashboards and reports

Custom dashboards and reports

Summary

Part 3: Optimize – How to Get the Most out of Cloud Resources

6

Implementing IaaS Compute Optimization

Compute optimization key concepts

Quick wins

Introduction to IaaS, PaaS, and serverless

Stateless versus stateful

IaaS optimization

Quick win – orphaned resources

Virtual machine version upgrades

Virtual machine rightsizing

Virtual machine family standardization

Virtual machine power scheduling

Virtual machine scaling

Reserved Instances and Saving Plans

Spot VMs

Summary

7

Implementing PaaS and Other Compute Optimization Initiatives

PaaS optimization

PaaS rightsizing and workload consolidation

Example – Azure App Service and App Service plans

Serverless versus provisioned compute

The benefits of Serverless

Example – Azure SQL Serverless

Managed Kubernetes cluster optimization

Data transfer costs optimization

Azure – Data transfer costs

AWS – Data transfer costs

GCP – Data transfer costs

Licensing optimization

Bring-your-own-license model

Cloud provider agreements and resource allocation

Azure – Enterprise Agreement versus CSP

AWS organizations, billing accounts, and OUs

GCP organization, folders, projects and resources

Summary

8

Implementing Database Optimization

Relational versus non-relational/NoSQL databases

Relational databases

Non-relational or NoSQL databases

Which one should you choose?

Which database management system?

Example – SQL Server versus Oracle pricing for AWS RDS

SQL Server

Oracle

PostgreSQL

MySQL

MongoDB

IaaS versus PaaS versus serverless

IaaS database optimization

Rational database use

Backup storage optimization

Shared Disks for database clusters

Shrinking relational databases

Database grouping in SQL Server

PaaS database optimization

Compute optimization and rightsizing

Database grouping

Database scaling

Serverless versus Provisioned Compute

Backup storage and redundancy

Reserved capacity

Azure

AWS

Google

Licensing optimization

Bring your own license (BYOL)

Development scenarios

Summary

9

Implementing Storage Optimization

Storage key concepts

Types of storage in the cloud

Thick versus thin provisioning in disks

Disk snapshots

Storage redundancy

Block storage

File storage

Object storage

Block storage optimization

Snapshot optimization

Ephemeral disks

Disk rightsizing

Offloading to file and object storage

Reserved capacity

File storage optimization

File storage rightsizing and data temperature

Reserved capacity

Object storage optimization

Object storage tiering

Life cycle policies

Limiting and tracking versioning, soft delete, and object snapshot usage

Object storage inventory

Reserved capacity

Other storage optimization initiatives

Log storage optimization

Backup storage optimization

Summary

Part 4: Operate – How to Set Up a Governance Model around Cloud Costs

10

Designing and Implementing FinOps KPIs

What is a KPI?

KPI creation process

Types of KPIs

Objectives and key results

Using KPIs for FinOps practices

Example of a FinOps KPI in Azure – region placement

More FinOps examples

Summary

11

Defining New FinOps Roles and Processes

Target operating model and FinOps

FinOps operational model

Organizational model

Rollout and execution plan

Functions, capabilities, and processes

Roles and responsibilities

Governance

Summary

Part 5: Hands-On Cost Optimization with Real-Life Use Cases and More

12

Case Studies for Cost Optimization

IaaS case study – multi-tiered application migrated to the cloud

Solution description

Initiatives covered

Summary of initiatives and final results

PaaS case study – storage, serverless, and database optimization

Solution description

Initiatives covered

Summary of initiatives and final results

Summary

13

Wrapping up and Looking ahead

FinOps summary and future challenges – how to keep up

Inform (Chapters 3, 4, and 5)

Optimize (Chapters 6, 7, 8, and 9)

Operate (Chapters 10 and 11)

Case studies

FinOps future challenges

Cloud sustainability and FinOps

How environmental sustainability policies work

Public cloud and sustainability – GreenOps

Machine learning, artificial intelligence, and FinOps

How ML works

FinOps applications

Self-assessment/knowledge check

Chapter 1

Chapter 2

Chapter 3

Chapter 4

Chapter 5

Chapter 6

Chapter 7

Chapter 8

Chapter 9

Chapter 10

Chapter 11

Chapter 13

Summary

Index

Other Books You May Enjoy

Preface

First and foremost, greetings and welcome to this book! Before we dive into it, we want to introduce you to the reasons why we wrote this book and set up the context.

The idea for this book was born after an intense experience of building up a FinOps practice together with a great team, which we worked on from scratch and created something that we felt was worth sharing with the FinOps community.

For almost two nonstop years, we were fully dedicated to FinOps, unlike other architects or engineers who divide their time between a lot of projects. We worked on FinOps governance and implementation in a really complex environment, where nothing was easy, but it was definitely satisfying to build it and see it grow.

It was two years full of research, learning every step of the way, thinking about what else to propose, coming out with new ideas and approaches, overcoming the different walls that were in front of us, solving problems, and adapting along the way.

Our goal is to share all of it with you, in the hope that it will aid you in your future experiences.

Who this book is for

This book is intended for cloud engineers, cloud and solutions architects, as well as DevOps and systems operations engineers interested in learning more about FinOps and cloud financial management for efficiently architecting, designing, and operating software solutions and infrastructure using public clouds. This book will also be useful for team leads, project managers, and financial teams interested in getting the most out of cloud resources.

Some prior knowledge of cloud computing and major public clouds will be needed to get the most out of this book, as in some sections, we will delve deeper into more technical work, terms, and examples.

What this book covers

Chapter 1, Introduction to FinOps Principles, provides an introduction to what FinOps is and why it is needed for organizations that are transitioning to or already in the cloud.

Chapter 2, Understanding How FinOps Fits into Cloud Governance, covers how FinOps interacts with different methodologies widely used in organizations, such as the Well-Architected Framework, infrastructure as code, Agile project management, and other key processes, such as change management. This chapter also covers how FinOps can adapt to organizations in different phases of their cloud journey, and the basic tools to perform cost analysis on Azure, AWS, and Google Cloud, as well as other market tools that are offered by other vendors outside of Microsoft, Amazon, and Google.

Chapter 3, Designing and Executing the Tagging and Naming Convention Strategies, provides a detailed explanation of why both tagging and naming convention strategies are essential for FinOps practices, as well as recommendations and tools that can be used to design, implement, and enforce your own strategies.

Chapter 4, Estimating Cloud Solutions Costs and Initiative Savings, provides a detailed description of all the migration models that can be used to migrate workloads to the cloud, as well as some key concepts about cloud costs that should be understood before going forward. It also covers how to leverage pricing calculators and REST APIs offered by cloud providers to create your own estimations, as well as how potential savings concepts can boost and drive your FinOps practices further.

Chapter 5, Improving Cost Visibility with Dashboards and Reports, provides an introduction to cloud billing data and the structure and fields of a cloud bill, as well as what dashboards and reports are and how they are different from each other. It also includes a lot of insights to improve the quality of your FinOps dashboards and reports using financial concepts and other key ideas, such as unit economics.

Chapter 6, Implementing IaaS Compute Optimization, provides an overview of FinOps initiatives that can be carried out on infrastructure-as-a-service compute services for cost optimization.

Chapter 7, Implementing PaaS and Other Compute Optimization Initiatives, provides an overview of FinOps initiatives that can be carried out in platform-as-a-service compute services for cost optimization, as well as other initiatives that are related to backup, licensing, and resource management best practices.

Chapter 8, Implementing Database Optimization, provides an overview of FinOps initiatives that can be carried out in database services for cost optimization. It also introduces a lot of key basic concepts around databases in general that are needed to fully understand the tools at our disposal for optimizing database services.

Chapter 9, Implementing Storage Optimization, provides an overview of FinOps initiatives that can be carried out in database services for cost optimization. It also explains in depth how the different storage paradigms work and some key concepts, such as redundancy, data temperature tiering, and the cost drivers of storage services.

Chapter 10, Designing and Implementing FinOps KPIs, covers what a KPI is and the different categories of KPIs that exist. Once the basic concepts have been introduced, it also provides a complete methodology to design and develop your own KPIs, with a lot of examples of FinOps KPIs that can be used as a starting point to create your own dashboards and reports.

Chapter 11, Defining New FinOps Roles and Processes, provides an overview of how to define and implement your own FinOps operating model, which includes the functions, capabilities, processes, and roles and responsibilities that enable FinOps practices to be part of the organization’s DNA, as well as other key governance initiatives to enforce FinOps policies.

Chapter 12, Case Studies for Cost Optimization, presents two examples of real-life architectures to be optimized. In a step-by-step manner, we provide examples of different initiatives that we can use to optimize these solutions, analyzing throughout the process the impact on costs that these initiatives generate.

Chapter 13, Wrapping Up and Looking Ahead, provides a summary of sorts, where we reflect on what we’ve covered in this book and some challenges that FinOps practitioners may still be facing in the future. This chapter also covers two emergent fields of study that are on the rise, which are machine learning and sustainability, as well as the synergies to be found in each one with FinOps practices. To close the circle, this chapter also provides a self-assessment for you to evaluate what you have learned throughout this book.

To get the most out of this book

There are no specific requirements to follow along with this book. However, we have used certain conventions in the book, which we’ve explained as follows. Reviewing them will help you understand the content structure better.

Throughout this book, we will add some hints and important notes, for which we will use the following format:

Important note

This is a note, a comment, or an example.

When we dive deeper into the technical aspects of FinOps, we will include examples from all the major public clouds, which are currently the following:

Microsoft AzureAmazon Web Services (AWS)Google Cloud Platform (GCP)

Note also that in the last chapter of the book, you will find a self-assessment/knowledge check, which you can use to evaluate what you have learned throughout each chapter of the book.

Across the book, in the more technical chapters, you will find references to production, preproduction, and development environments. They are defined as follows:

Production: In this environment, our services are published to final users or used in business processes. The services of this environment are live, so everything should work perfectly. In this environment, changes that may impact users or business are done out of hours or in maintenance windows that are previously set and agreed upon.Preproduction: Also called staging or User Acceptance Testing (UAT), this environment should be as similar as possible to production. This environment is where key users can test applications before they are promoted to production, and also where contingency tests are carried out.Development: Development environments are where the development processes take place. In this environment, data is usually fictional or consists of dummy data, and the resources are downsized to optimize the costs, as their computing needs are way lower. These environments are where new code is thoroughly tested by developers, to add new features or solve bugs.Sandbox: A sandbox environment is an environment which is usually isolated from the rest and whose purpose is to freely experiment with cloud services and software development. Company and security policies are usually not that strict in sandbox environment, and it is often used to conduct Proof of Concepts (PoCs) in a controlled environment.

The currency for all the cost references, estimations, and calculations used in the book is the American dollar ($).

Some other conventions we have used are:

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “From this point, let’s say we want to check the current pricing. We should use currentVersionUrl.”

When we show some examples of CLI commands to be used, the format used is as follows:

aws pricing describe-services --service-code AmazonEC2

A block of code is set as follows:

{ "companyname" : "imagineinc", "businessunit" : "finance", "city" : "madrid", "region" : "spain" }

Apart from these general notes, there are no requirements to navigate this book. A word of advice, though: Chapters 6 to 9 do get really technical, which may be challenging for readers coming from other non-cloud backgrounds.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Efficient Cloud FinOps, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/978-1-80512-257-9

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Get Started with FinOps

In this part, we will go through all the ideas behind FinOps practices, and we will also explain in depth why this buzzword is becoming so popular.

Once through the basics, we will deep dive into how FinOps practices can fit and benefit multiple organizations in different stages of their cloud journey, as well as the synergies to be found with widely used methodologies such as DevOps and Agile.

This part has the following chapters:

Chapter 1, Introduction to FinOps PrinciplesChapter 2, Understanding How FinOps Fits in Cloud Governance

1

Introduction to FinOps Principles

This chapter aims to introduce the topic we are going to cover throughout the book, which is FinOps, a methodology relating to cloud financial optimization. We will explain why it is so important, and why it has become one of the biggest current IT trends.

In this chapter, we are going to cover the following main topics:

What is FinOps, and why do we need another buzzword?Why FinOps?The three pillars of FinOps

What is FinOps, and why do we need another buzzword?

Let’s start by asking the question, what is this new methodology that everyone is talking about and, suddenly, all companies are in need of? It’s a new groundbreaking practice, or way of doing things, that everyone claims will allow you to take control of your cloud costs and revolutionize how we work with cloud resources!

We are, of course, referring to FinOps. Let’s take a look at how it is defined by the FinOps Foundation:

“FinOps is an evolving cloud financial management discipline and cultural practice that enables organizations to get maximum business value by helping engineering, finance, technology, and business teams to collaborate on data-driven spending decisions.”

This is an amazing definition, but we will also provide our own interpretation of the topic. For us, FinOps is about the following things:

First, very much like DevOps, it is about teams working together to remove the barriers between them. Once these barriers are down, companies can benefit from this teamwork greatly by people sharing knowledge and working together to reach common objectives.It is also about creating a specialized team that oversees cloud projects and operations from a new and different perspective that did not exist before, which can act as a common bond between financial, architecture/engineering, business, and technical teams. This new team is especially important in big companies with different business units across the globe, where keeping standardized best practices on cost optimization and strong governance, while maintaining the independence of each business unit, can be a demanding and daunting task.This specialized FinOps team can also become technical and help other technical teams define, design, and implement plans to understand, control, and reduce costs. Often, this exercise leads to FinOps-specific solutions being developed to improve automation and control costs. FinOps training is also one of the biggest focuses of FinOps teams, which should educate all interested parties on cost optimization and FinOps concepts.FinOps practices also aim to change organizations and improve the governance processes themselves, by adapting them to the new reality that is the cloud, where the established rules of on-premises computing don’t apply anymore and never will do.Finally, FinOps is a mechanism to pave the way for technical teams to be more accountable for cloud costs. If FinOps practice and the FinOps team help to shed some light on what the cost drivers are and how to control costs while workloads are optimized, it can be very beneficial for technical teams, which will take responsibility for costs after the FinOps ideas and initiatives are understood and incorporated.

All in all, it brings a new vision to the table that everyone will benefit from.

We will describe in more detail what the FinOps Foundation is and its purpose later in this chapter.

Side note

It is important to understand that FinOps is not about generating savings.

In a non-optimized environment, a lot of savings can be obtained by implementing FinOps initiatives and governance. However, the savings can be generated only to an extent, as eventually, when FinOps practices mature, there are no more savings to be made.

Due to this, the performance FinOps practices should not be entirely measured in savings KPIs, as these have a limited lifespan.

FinOps is not about saving money; it is about creating value. It is also about making the most out of a budget and organizations being able to invest even more money in the cloud, being confident that it will be done in an efficient and optimized way that leads to assured investment returns in the future.

Why FinOps?

FinOps is a cloud cost optimization/cloud financial management methodology, aimed at companies that already work in the cloud or are transitioning to it.

The need for FinOps becomes apparent when companies embark on their cloud journeys or strive for excellence once they are already settled in using it:

For newcomers, when the journey to the cloud begins, it becomes apparent that the well-known on-premises rules regarding cost management or capacity management don’t apply anymore. This requires a difficult adaptation as well as training. FinOps can help in this complex process, both optimizing and enabling cloud governance in many ways.If an organization has used the cloud for a few years already, its cloud spend will not be aligned with its initial budget for a number of reasons, such as non-optimized workloads that are migrated through lift-and-shift, constant struggles with inefficiencies, and difficulties in applying a standardized cloud governance and best practice baseline. FinOps can help mitigate the impact on costs by working to implement cost reduction plans, as well as help build tighter cloud governance to avoid past mistakes in the future.

The public cloud changed everything, and the result often was that you either spent too much or you spent too little on things that were not relevant to the cloud or beneficial to a company, offering a bad impression of cloud computing to the business and its stakeholders. FinOps aims to bring visibility, optimization, and governance to improve the cloud’s image from a business perspective.

To fully understand why we have ended up where we are, and why this new methodology is needed, let’s go back a few years, around five or six. The public cloud, after years of development and improvements, has become a more mature product, ready to change how IT services are purchased and delivered. The public cloud, therefore, made a big splash in IT market, and many companies were interested in this new paradigm.

Before the cloud

From a financial and capacity management perspective, IT equipment traditionally was acquired with the expectation that it would last a few years. It should not only be operative for at least five years but also be able to accommodate the estimated expected growth in capacity (mainly storage but also computing capacity) before reaching its end-of-life date.

This led to the underutilization of virtual machines when growth did not meet expectations and non-optimized workloads were not as impactful financially. Unneeded virtual machines were often left unused on servers for several months, and badly optimized workloads or underused servers did not hurt that much if there was spare capacity left.

Being on-premises made it really hard to scale down, as due to this infrastructure purchasing model, once you committed to acquiring new assets, you were expected to keep growing instead of downsizing.

The cloud comes into play

When the public cloud made its entrance, every company adapted as much as possible. Depending on each company’s situation, the process was as follows:

For newly established companies, it was way easier to create everything from ground zero without an on-premises legacy, following somewhat of a greenfield approach. However, cloud knowledge was not that mature and quick decisions were needed, which often led to non-ideal architectures or non-optimized solutions.Smaller companies that were already established, due to budget and resource limits, were not able to jump into the cloud as eagerly, so they mostly continued following the traditional approach, postponing their journey to the cloud.For big companies with big budgets and business cases to migrate to the cloud in place, this process was much harder. There were contracts with infrastructure vendors in place, clusters of recently renewed servers, and money spent on disaster recovery data centers, following the traditional infrastructure approach. The cloud then became something like a commodity for most organizations, where they put their non-critical services or development environments, but they didn’t fully commit to a cloud-only operation. Cloud vendors often pressured their clients into monetary commitments that were often partially wasted, as migrations were hard to implement due to high price tags, limited budgets, and lots of technological debt from on-premises vendors and software.

A paradigm shift

Let’s explain point by point how the cloud revolutionized everything to fully understand how the traditional on-premises approach was moved away from:

With the acquisition of assets and capacity management, a major change occurs at a company, as the financial teams lose control of infrastructure spend and the technical teams still don’t feel accountable for cloud costs. Pay as you go is the billing model the cloud uses, which is a huge benefit when you need to downsize, for example, but it is a heavy burden if your workloads are not optimized. This approach is totally opposed to traditional on-premises approach, where assets were bought all the time, and technical architecture was set up around these assets, adapting to the platform or products each company made use of.Business and financial teams transition from a standard model where most of the investments were capital costs (hardware, for example) that followed a certain life cycle, as well as a few operational costs (infrastructure operations support, for example) on top to keep the business running. In the cloud, we shift to a totally different model, where there are no big capital costs and almost all the expenditure is operational (we will cover these ideas later on in this chapter)Access to the company assets also evolves. In a traditional data center, the assets were often hosted locally, and technical teams set up all the technical architecture and security needed for them (e.g., internal networks, firewalls, physical security, remote access, and VPNs to connect from multiple offices), building walls and secured doors to protect the organization assets. Resources and workloads in the cloud are not hosted locally anymore, so access is managed in more advanced ways, such as point-to-site VPNs, allowing users to work remotely anywhere (the COVID pandemic changed everything in this regard, forcing companies to establish such mechanisms). Traffic travels through the internet, so security teams put their efforts into building endpoints, data, and identity bastions instead of physical and perimeter security.Regarding the technical distribution of assets, there is often only one tenant or enterprise domain that can span multiple offices. Identity management is often unified under one unique domain (AD or Samba, for example), and there are minimal trust relationships or connections with other tenants or environments. Once assets such as virtual machines are deployed, they usually keep the same configuration during their lifespan, with minimal configuration changes. When the cloud comes into play, resources are distributed across multiple tenants (for example, per business unit), with federation and B2B/B2C with other tenants. The workloads adapt to scale on demand, following elasticity and loosely coupled architecture principles, rapidly changing configuration and resources in a much more agile manner.The delivery of projects and IT services transitions from locally hosted delivery, where all responsibilities and management are in the remit of each organization, to delivery through cloud services, in which responsibility is shared due to IaaS, PaaS, and SaaS models. In the case of a traditional IT approach, when a company grew, new land, bigger offices, and data centers were acquired. Now, we can save up some money and just focus on technology, without the need to invest in real estate to host infrastructure. This makes the Return of Investment (ROI) for a business much faster, as you can spin up some cloud services in minutes and build a proof of concept, without any added complexity. Also, if you use fewer managed cloud services, such as PaaS or SaaS, you can even make some savings on architectural or technical resources, for example.Security also undergoes a major change. There are transitions from perimeter (e.g., firewalls and demilitarized or DMZ networks) and physical (e.g., protecting offices with alarms, security personnel, and cameras) security to identification (e.g., multifactor and conditional access), endpoint, and other methods to prevent unwanted access to company data or assets. Methodologies such as Zero Trust also make an appearance to better adapt security measures to this new situation.From a business continuity perspective, disaster recovery and contingency plans were built in-house, which took a lot of effort from a technical architecture perspective. Backup was also part of this exercise, where backup software from different vendors was used and stored in different ways. In the cloud, when using less managed services such as PaaS or SaaS, services are highly available by default, as information is kept in multiple data centers across the same region and SLA is granted from the cloud vendor. Backup functionality is built in to most services, making this process much easier, and high availability can be achieved by using multi-regional services and multiple redundancy options for storing our data accross multiple cloud regions. As a result of all this, having a disaster recovery data center does not make sense anymore for most companies, as you can attain highly reliable and fault-tolerant workloads and applications in different ways. Also, some savings can be generated by getting rid of the housing, power, electricity, and maintenance costs for secondary data centers.

All this information can be summarized using the following table:

Model

Traditional computing

Cloud computing

Acquisition

Buy assets

Build technical architecture

Buy cloud PAYG services which require continuous cost optimization

Build minimal technical architecture

Business

Pay for assets

Administrative overhead

CAPEX/capital expenditure

Technical teams not accountable for infrastructure costs

PAYG

Reduce admin functions by using automation and shifting to fully managed cloud services

OPEX/operational expenditure

Technical teams fully accountable for infrastructure costs

Access

Hosted locally

Internal networks

Corporate desktop PCs and laptops

Local logon for access control

Hosted by third-party service and cloud providers

Over the internet

Any device

Unified access control

Technical

Single tenant and non-shared

Static and hard to transform and scale up or down

Tightly coupled workloads

Multi-tenant with B2B/B2C

Scalable, dynamic, adaptable, and elastic

Loosely coupled workloads

Delivery

Delivery through local data centers

Costly and lengthy deployments

Land and staff expansion

Delivery through cloud services

Much faster deployment and new services testing

Faster ROI

Security

Physical and perimeter security to protect company assets

Identity, data, device, network, and infrastructure security to protect assets anywhere

Business continuity

Use of secondary data centers

Backup and disaster recovery strategies built by technical teams

Highly available cloud services with SLA availability granted from the vendors

Built-in backup and redundancy in less managed services such as PaaS and SaaS

Apart from these groundbreaking changes, there are also other factors to consider when setting up cloud operations. They are as follows:

Lack of knowledge about how the cloud works and cloud offerings and technologies often create different degrees of change resistance in technical teams, and the cloud world can be perceived somewhat as an enemy. To overcome this, the best tool is to invest in technical training and work on a cloud enablement pitch, as well as having as much sponsorship from the C-level executives as possible.Hiring people with high cloud knowledge also becomes a challenge, as talent is scarce and there is a high demand for technical roles such as cloud engineers, architects, software developers, and DevOps experts. In the cloud, a new service or offering that has new features is released almost every day, so technical teams need to be up to date on everything and invest time and resources in constant training. It gets harder and harder to keep up with technology and adapt to it.

To avoid these issues, it is usually wise to avoid fully committing to one platform or product, easing the process of adapting to changes that may come in the future. Also, it is wise to wait for a new trend to mature a bit before jumping in so that you can benefit from community experience and global knowledge.

Now we have covered the differences between cloud and on-premises, let’s address another key question to evaluate when considering a move to the cloud – how much does the on-premises setup cost?

Hidden on-premises costs

When companies begin their own Journey to the Cloud (J2C), a complete business case is always needed. A business case should always include at least the following:

The actual cost of running on-premises infrastructureThe projected costs of moving to the cloudThe features of the chosen cloud and how cloud operations will change how operations are runThe benefits of the cloud in terms of security, reliability, and performanceThe value from a business perspective due to the migration

Part of this business case elaboration exercise is to evaluate the current cost versus the estimated costs in the cloud, as well as all the benefits, improvements, and drawbacks that the cloud brings.

At some point, the following question will be raised – how much does it cost to run infrastructure on-premises?

Important note

Given our experience, we have often found out that no one really knows the exact costs of keeping on-premises running because it was never needed.

Part of the FinOps team’s responsibility is to aid in improving the visibility of these costs for a business.

Before getting deeper into the topic, let’s introduce three key IT concepts, which are Capital Expenditure (CAPEX) costs, Operational Expenditure (OPEX) costs, and Total Cost of Ownership (TCO). These important key concepts are defined as follows:

CAPEX: This refers to the aforementioned capital or direct costs, which are fixed, one-time expenses incurred on the purchase of land, buildings, construction, and equipment used in the production of goods or the rendering of services. In the IT realm, it’s the cost of the infrastructure hardware (e.g., servers, networking, and all IT equipment) and the software itself.OPEX: These costs include the operating expenses needed for the infrastructure to function, such as electricity, housing, and hosting.TCO: As stated in the Gartner Glossary (https://www.gartner.com/en/information-technology/glossary/total-cost-of-ownership-tco), TCO “is a comprehensive assessment of information technology (IT) or other costs across enterprise boundaries over time. For IT, TCO includes hardware and software acquisition, management and support, communications, end-user expenses, and the opportunity cost of downtime, training, and other productivity losses.” It should include all CAPEX and OPEX costs.

To begin the exercise of analyzing on-premises costs, let’s consider the different OPEX costs that come to mind when thinking about on-premises infrastructure:

Power, utilities, and generators: These ensure that an infrastructure has enough electricity to function. For bigger companies, this often includes even a second circuit for equipment to function if there is an outage. These secondary circuits are rarely used, but they need to be in place to ensure business continuity.Cooling: Keeping servers at the right temperature for optimal performance is never cheap, and there are always maintenance costs involved.Real estate: Owning or renting a property where the infrastructure is hosted costs money and requires some degree of planning.Physical security: This point needs to always be considered, as no company can afford their servers to be tampered with or accessed by malicious third parties (to steal data, for example). These costs cover security guards, cameras, access control systems, and other security measures.

These indirect costs are inherent to IT infrastructure and are paid even in private cloud and hosting services models, even though the vendors or service providers that offer these services may be third parties external to an organization.

Now, let’s move on to CAPEX. These costs are not easy to measure either, as some of them are paid in full before equipment is purchased, and since the life of a CAPEX generally extends beyond a fiscal year, amortization should be used to redistribute costs while keeping in mind the depreciation of assets over time.

In terms of CAPEX costs, we have the following:

Hardware: This is the cost of physical servers, storage or disk cabinets, and Network Attached Storage (NAS) devices. Usually, these assets are acquired for a price that includes a warranty for a limited time, around five years. When the warranty expires, it is possible to extend it further by paying additional fees to the vendor or other service providers. If your server or device is no longer supported, you won’t have any help from the vendor, and the replacement parts can be impossible to attain.Support: When purchasing either hardware or software to be used in enterprise applications or environments, it is often necessary to have some degree of support from the vendor or a partner. These support fees are paid using different models.Software and licenses: Software and license purchases have varied purchasing models. It is possible for some software to acquire perpetual licenses, which avoids the management overhead that subscriptions pose. The fact that each piece of software has a different licensing model makes the process of evaluating costs more complex.Backup and disaster recovery: In order to run successful IT operations, it is important to ensure that, in the event of an issue or a hardware or software failure, there are mechanisms to keep workloads functioning and running. Designing solutions that include these mechanisms always poses additional costs, such as having a secondary copy of the infrastructure or storing backups in additional storage.Networking: This is the purchase of network devices such as firewalls and routers. These devices also require periodic maintenance.IT maintenance: Having a team that operates the infrastructure is not free.Training: To train the IT team to perform their management of the infrastructure, some degree of training is needed, which also comes with some associated costs, even if it is just the time spent by the technical teams to prepare documentation and train newcomers.

Example

To exemplify these concepts we have just described, let’s use as an example an out-of-support old server.

This server, which has already passed its end-of-life date is still used to host some workloads. We have experienced from our discussions with clients and organizations that this kind of situation happens more often than we think.

The cost of this old server for financial teams is really good, as almost all the costs for this asset are OPEX now.

However, there are things that need to be taken into account:

The TCO for this asset is really low compared to a new server. This creates the false perception that savings are generated by having this outdated asset in operation.

Having an end-of-life server involves a high risk from any perspective, even if it is used as a passive node or as part of a secondary data center. The hardware can malfunction anytime, and spare parts may be impossible to get a hold of.

Old hardware is usually tied to old software as well. Old software tends to have security vulnerabilities that newer versions often address, which poses a potential risk that could impact business operations in many ways.

Having old hardware and software in place also means that our personnel needs to know how to properly manage this far from state-of-the-art systems, equipment and products, which can be no easy feat.

There is a visual concept that is often used to reflect these fundamental differences between on-premises and cloud costs, which is the iceberg of on-premises costs, as shown in the following figure:

Figure 1.1 – On-premises versus cloud costs

Now that the fundamental differences in both approaches have been covered, let’s go back in our time machine from past to present, to close the circle and fully understand the need for FinOps practices.

Back to the present

Let’s travel back to present times with our FinOps-powered time machine. A few years have passed since the cloud’s first appearance, and cloud platforms have evolved in a big way, increasing the portfolio and features of services offered, which enables you to use the cloud for enterprise-grade solutions with full confidence.

From an organizational point of view though, the financial result of transitioning to the cloud is that the initial expected budget for the cloud is nothing like the estimated costs. Most companies are still not fully committed to the cloud, facing the same challenges, and people with knowledge of the cloud are even harder to come by.

There is also one thing that is vital to understand when migrating to the cloud, which a lot of organizations fail to comprehend. If a workload is not optimized when hosted on-premises, it won’t be optimized if it’s migrated to the cloud either. Subsequently, if something does not work well before migration, all those issues and technical limitations due to badly architected solutions will also be transferred to the cloud.

In summary, ideally, all efforts should be put, in our opinion, into optimizing workloads before migration, ensuring that all the benefits of the cloud can be leveraged to its full extent once the applications and workloads are moved.

After some time has passed, non-optimized workloads will have become more common, both on-premises and in the cloud. It’s time to face the truth – someone will need to get their hands dirty and tackle all these issues and limitations once and for all. This is where FinOps practitioners come into play.

The FinOps Foundation

The FinOps Foundation is a program of the Linux Foundation (https://www.linuxfoundation.org/) and is dedicated to advancing people who practice the discipline of cloud financial management.

It was founded by J.R. Storment (https://www.linkedin.com/in/jrstorment/) in 2019 and joined the Linux Foundation in 2020. According to its web page, it currently has more than 12,000 community members, representing more than 3,500 companies, and this figure is likely to keep growing in the future. It is a strong community where FinOps practitioners can share experiences, best practices, tools, and documents and get access to a lot of other assets to increase their FinOps knowledge.

In addition to this, it offers multiple certification programs and courses for FinOps practitioners to complete and tailor their FinOps education.

The FinOps Foundation created a complete framework (https://www.finops.org/framework) that describes how FinOps practices can take place. Essentially, it breaks down the practice into three different phases – inform, operate, and optimize. Be mindful that, in this book, we offer our own take on FinOps practices that, although similar, differ slightly from the FinOps Foundation approach. Instead of phases that must be followed one after another, we follow different streams or pillars that group together all the different FinOps initiatives we will work on. With this pillar-based approach, we can adapt to the specific needs of each company from a FinOps perspective, tackling the most pressing matters first and then working on different domains simultaneously, instead of following sequential phases one after another.

Let’s move on to describe in detail what initiatives can be included in the Inform, Operate, and Optimize pillars.

The three pillars of FinOps

In this section, we will explain our view of the three fundamental pillars upon which a successful FinOps practice should be built.

Let’s describe one by one the three pillars, which are Inform, Optimize, and Operate, and the key points to work on and develop in each one, alongside some examples to illustrate how to get started.

Inform

One of the most important things every FinOps practice must do is to improve the visibility of cloud costs for all the technical, financial, and project management teams. Everyone needs to be initially aware of how costs are distributed between, for example, business units, projects, offices or locations, and cloud regions. This process is often called cost allocation, as it allows you to divide costs across different units.

In many organizations, this is key to enabling two important cloud cost visibility exercises, which are showback and chargeback:

Showback refers to the operation of dividing the costs of IT services across different IT departments and providing the obtained information to managementChargeback, conversely, goes one step beyond and bills the different departments or business units for IT assets, licenses, training, and any kind of relevant work or service

This may seem like a simple thing to do, but in reality, it is a complicated matter, to say the least, as it often implies working on naming conventions, tagging, and other Operate pillar initiatives (which we will describe in detail later on this section), which are not present or not consistently applied in a lot of organizations. There is also another issue that arises here, which is how complexity rises when applying chargeback and showback on a shared infrastructure or other shared service costs. We will cover this in later chapters.

Another part of the Inform pillar, and one of the first starting points of any FinOps implementation, is to undertake a FinOps maturity review, assessing the status of an organization in terms of cloud cost optimization and cloud financial management. This review will serve as a starting and reference point for the future, assessing and determining how far a company has gone in this cost optimization process.

Also, in our experience, once the FinOps practice begins and there is a team working on visualizing cloud costs, everyone working on the cloud suddenly begins to be a little more aware of how everything they do may have an impact on costs, and they start to think twice before upscaling a database for a particular service or requesting a bigger virtual machine for a small application.

Awareness across cloud technical teams is a really good thing when we deal with cloud costs, and it is often one of the quick wins that every FinOps practice can achieve in almost no time.

The result of the Inform exercise can be delivered by creating FinOps dashboards and reports, which can include the degree of application of the different FinOps initiatives that are proposed by the organization.

When we improve visibility on costs and track the application of different initiatives, there is no better driver for these initiatives than to calculate the projected savings of these initiatives when applied to their full extent.

In our view, FinOps work in the Inform pillar never ends, as we can continue to improve visibility and adapt it to each team’s needs.

This diagram summarizes the key points to work on in the Inform pillar:

Figure 1.2 – The Inform pillar key points

Let’s try to summarize all this work with a small example to better understand the key questions to ask and answer during this process.

Example (the Inform pillar)

Do you know how costs are distributed per environment in your organization? How much does each environment cost?

The first thought that comes to mind is that costs for production and preproduction should be similar, with development environment costs way behind, assuming stable workloads that have been deployed for a long time. For new projects, development can sometimes have higher costs if a project has not gone live yet.

Each organization and workload should have its own cost variables, so the first step is to make this cost distribution visible for everyone to see. Once it’s visible, it’s our job to make sense of it (working together with technical teams) and decide whether it’s aligned with the use case of each environment and the status of each workload.

If it’s not, it’s time to dig deeper and check for inefficiencies.

This is the train of thought we should follow when working in the Inform pillar, which is raising questions and searching for answers that make sense, while we keep investigating and searching for inefficiencies along the way.

This example only covers a really small part of the Inform pillar but does illustrate the methodology to be followed when working to improve cost visibility.

Optimize

The Optimize pillar is about making the most effective use of cloud resources. This pillar requires a huge amount of research and technical work on all the cost drivers of cloud services and offerings in the cloud.

Apart from understanding how costs work and what drives them up or down, it is important to have deep architectural knowledge, to discern what is needed and when, as well as to be able to reduce unnecessary compute, features, or configurations that raise costs without justification.

In this pillar, we learn how to make use of new purchasing models such as Reserved Instances and Saving Plans, Spot Virtual Machines, and all the possible ways to improve governance based on costs in the cloud.

We also learn other cost optimization strategies, such as licensing, compute, storage, and database optimization, as well as other general concepts such as autoscaling and power scheduling.

Part of this Optimize path involves training and working together with technical teams, as well as helping them apply the initiatives that are proposed by FinOps teams and providing them aid in the form of assets, such as scripts, automation, and dashboards, to help them streamline and standardize the implementation of different initiatives that everyone will benefit from.

This diagram summarizes the key points to work on in the Optimize pillar:

Figure 1.3 – The Optimize pillar key points

Example (the Optimize pillar)

The best way to begin the Optimize pillar is to propose a series of quick wins, which are initiatives that provide cost savings with little to no impact on running workloads, such as deleting orphaned resources or decommissioning unused services. We will delve deeper into this topic in upcoming chapters.

After the implementation of some quick wins, we can begin to implement cost optimization initiatives that are easier to implement in new projects, while we remediate non-optimized workloads, using potential cost savings as a way to prioritize one initiative over another.

Once the practice is established, we can tackle the most difficult initiatives that require additional planning and careful analysis, such as making use of Reserved Instances, Saving Plans, and Spot Virtual Machine purchase models.

While we optimize, we cannot forget to make the advances visible for all stakeholders and interested parties to see. By doing so, we effectively promote FinOps practices and create confidence in this methodology.

Operate

In our experience talking and working with different organizations and clients, we have seen a lot of one-time cost optimization exercises that reduce costs. These exercises require a lot of effort and resources, but they provide great value.

The problem with this approach is that, if the roots of a practice are not deep enough, the same errors will eventually resurface again, which takes organizations back to square one, rendering all the effort and resources that were invested useless in the long run.

In this pillar we will also work in designing and implementing new KPIs and measures, which will show objectively how well we are doing in objective terms to a business, financial teams, and stakeholders, and bringing to the table a fresh point of view on optimization, modernization and any other topic that can be valuable.

The Operate pillar is about governance. With this pillar, our proposal is to explore the roots that are needed to build a strong practice. You can begin this pillar by working on naming conventions or tagging, including defining new FinOps roles and processes, enabling organizations to stay one step ahead and not keep making the same errors. This makes FinOps an iterative exercise that improves on each iteration as the teams involved mature and acquire cost optimization expertise.

This diagram summarizes the key points to work on in the Operate pillar:

Figure 1.4 – The Operate key points

Let’s illustrate with a simple example how we can get started in the Operate pillar.

Example (the Operate pillar)

One of the first steps in the Operate pillar can be simply establishing a quarterly review for cloud cost optimization and the cloud costs themselves. Technical and architectural teams can liaise with financial and FinOps teams, discuss the cloud costs, and prepare a report for the meeting, such as a manually created cost report in the initial phases of FinOps adoption, which can be replaced later on with a report that is generated automatically through an automated process.

This recurring meeting is not that hard at all to set up, and it can be a great starting point to make cost reviews an iterative exercise while fostering collaboration between teams.

Summary

As we are sure you have already determined by what we have covered so far, FinOps is definitely not a one-size-fits-all methodology.

The practice needs to be tailored and adapted to each company’s needs; it’s about doing things in a different way, which is always a challenge in organizations.