Atlassian DevOps Toolchain Cookbook E-Book

Robert Wen

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 343

Veröffentlichungsjahr: 2024

Beschreibung

Implementing DevOps practices and toolchains for automated testing and deployment can accelerate product development with minimal errors in the production environment. However, creating DevOps toolchains by integrating tools from various vendors presents challenges for both administrators and developers. Written by four well-known experts from the Atlassian community, this book addresses the complexities of DevOps toolchain creation and integration by leveraging Atlassian’s Open DevOps solution.
Starting with a holistic overview of the DevOps and Atlassian Open DevOps solution, you’ll learn to integrate Jira with other tools. You’ll then find out how to create and integrate a CI/CD pipeline in Bitbucket for automated testing and deployment to Docker containers. With step-by-step guidance, you’ll connect Jira and Bitbucket with other tools, such as Snyk for security and SonarQube for testing, to form an extensive toolchain. You’ll also learn how Compass uses CheckOps for observability and how to use Confluence for documentation and reporting. Finally, you’ll leverage Opsgenie’s ChatOps functionality to enhance collaboration between developers and operations teams.
By the end of this book, you’ll be able to establish your DevOps toolchain by integrating Atlassian tools to automate and optimize the software development lifecycle and beyond.

Leseprobe

Atlassian DevOps Toolchain Cookbook

Recipes for building, automating, and managing applications with Jira, Bitbucket Pipelines, and more

Robert Wen

Alex Ortiz

Edward Gaile

Rodney Nissen

Atlassian DevOps Toolchain Cookbook

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Preet Ahuja

Publishing Product Manager: Surbhi Suman

Book Project Manager: Ashwin Kharwa

Senior Editor: Mohd Hammad

Technical Editor: Yash Bhanushali

Copy Editor: Safis Editing

Proofreader: Mohd Hammad

Indexer: Hemangini Bari

Production Designer: Jyoti Kadam

DevRel Marketing Coordinator: Rohan Dobhal

First published: July 2024

Production reference: 1200623

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83546-378-9

www.packtpub.com

To Jill and Charles, who support all my escapades and projects.

– Robert Wen

To Lina, Jeremy, and Isaac, whose unwavering support has made every adventure unforgettable.

– Alex Ortiz

To Meredith and Gracie – you are my co-pilots in this crazy and fantastic life. I wouldn’t be the person I am today without your constant support.

– Ed Gaile

To Jennifer, who is always up for another adventure together. You have helped me discover more about myself than I thought there ever could be.

– Rodney Nissen

Contributors

About the authors

Robert Wen is a DevOps consultant at ReleaseTEAM. He specializes in continuous integration, continuous deployment, value stream management, and agile development. He has over 20 years of experience in hardware and software development, working in industries such as defense, aerospace, healthcare, and telecommunications. During his career, he has been a coach as the scrum master for several Agile projects and a trainer for courses on the Scaled Agile Framework (SAFe). He holds the SAFe Program Consultant (SPC), Atlassian Certified Expert, Certified CloudBees Jenkins Platform Engineer, GitLab Professional Services Engineer, and Flow Framework Professional titles and certifications.

There are a lot of people in the organizations where I have worked that I would like to thank, without whom this book wouldn’t have been possible.

First, many thanks go to my colleagues at ReleaseTEAM, Shawn Doyle, Karl Todd, and the members of the Atlassian practice team.

Second, kudos goes to Atlassian, not only for creating the tools we detail in this book but for fostering a community of enthusiasts to collaborate with, including my co-authors.

Alex Ortiz is an Atlassian educator and administrator at Apetech. He specializes in the administration and utilization of the various Atlassian tools. He has over 10 years of experience in hardware and software development, working for industries such as aerospace and robotics. During his career, he has been an IT administrator for TFS, Jira, Confluence, and Bitbucket. He has also served as a technical program manager for unmanned vehicles and autonomous robots. He is the creator of Apetech Tech Tutorials, a YouTube channel focusing on the Atlassian tools. He is a Certified Scrum Master (CSM). He has a BS in computer engineering from California State University, Long Beach (CSULB) and an MS in systems engineering from the University of Southern California (USC).

I’d like to thank my wife, kids, mother, and father who have believed in me since day one.

A special thanks to Gummy and Carlos for supporting me when everyone else didn’t.

And finally, a special thanks to everyone who has ever watched one of my YouTube videos. Without your support, I would have never attempted to even write a book.

Edward Gaile is a principal solution architect at Appfire. He has over 20 years of experience in software development, primarily in the healthcare insurance industry. His specialties include enterprise application design, DevOps, and change enablement optimization. He has been involved in the Atlassian eco-sphere for the past 12 years, including leading the Atlanta Atlassian user community for the past 10 years. He holds multiple Atlassian certifications and a bachelor’s degree in industrial engineering from the Georgia Institute of Technology. In his spare time, he is a professional BBQ pit master.

If it takes a village to raise a kid, the same can be said for writing a book.

I would like to thank my co-authors, “King” Bob Wen, Alex Ortiz, and Rodney Nissen. Working together on this project has been extremely rewarding, inspirational, and just plain fun.

I would also like to express my appreciation for the entire Atlassian community, especially fellow community leaders. Truly, they are the most supportive community of amazing people.

Rodney Nissen, better known as the Jira Guy, is a blogger who writes regularly on Atlassian tools and cohosts the The Jira Life podcast. He is also a DevOps engineer with ReleaseTEAM and has been working on setting up DevOps tools for 10 years. He specializes in integrating various DevOps tools together, Atlassian tools setup and configuration, and using “infrastructure as code” configuration tools to aid in system administration. He is currently an Atlassian Community Leader, Atlassian Certified Expert, and an active participant in the Atlassian Creator Program, holding credentials in Atlassian Jira administration, Atlassian confluence administration, Atlassian Agile development with Jira, and Atlassian system administration.

I’d like to thank my wife, who always supported me and believed I could make this hair-brained scheme work.

I’d also like to thank my many readers, who found my random shouts into the void and gave them a deeper meaning than I could have ever envisioned when I started this journey.

Lastly, my co-authors and co-conspirators, Alex, Bob, and Ed. I’ve got bad news because, if I have any say in it, we are stuck together now!

Table of Contents

Preface

Part 1: Beginning the Cycle

1

An Introduction to DevOps and the Atlassian Ecosystem

An introduction to DevOps

The CALMS/CALMR approach

Technology considerations for DevOps

Creating an Open DevOps toolchain from scratch

How to do it...

Creating an Atlassian Cloud site with Jira only

How to do it...

There’s more...

Creating a Jira project

How to do it...

Connecting Confluence

Getting ready

How to do it...

Connecting Bitbucket

How to do it...

2

Discovering Customer Needs with Jira Product Discovery

Adopting JPD

Getting ready

How to do it...

Creating a JPD project

How to do it...

Viewing ideas

How to do it...

Creating ideas

Getting ready

How to do it...

Adding information to ideas

Getting ready

How to do it...

Delivering ideas for development in Jira

Getting ready

How to do it...

3

Planning and Documentation with Confluence

Technical requirements

Creating Confluence pages linked to Jira projects

How to do it…

Displaying Jira issues on Confluence pages

How to do it…

See also

Viewing Jira reports on a Confluence page

How to do it…

Viewing Jira roadmaps on a Confluence page

How to do it…

Viewing linked pages from other applications using Smart Links

Getting ready

How to do it…

Part 2: Development to Deployment

4

Enabling Connections for Design, Source Control, and Continuous Integration

Technical requirements

Connecting Jira to design tools

Getting ready

How to do it…

Connecting Jira to source control using a native integration

Getting ready

How to do it…

Connecting Jira to source control using a universal integration

Getting ready

How to do it…

Connecting Jira to CI tools

Getting ready

How to do it…

5

Understanding Bitbucket and Bitbucket Pipelines

Technical requirements

Creating a workspace, project, and repository in Bitbucket

How to do it…

There’s more…

Creating branches in Bitbucket

Getting ready

How to do it…

Understanding pull requests and merging best practices

How to do it…

Enabling Bitbucket Pipelines

How to do it…

There’s more…

Configuring runners in Bitbucket

How to do it…

6

Extending and Executing Bitbucket Pipelines

Technical requirements

Configuring pipeline options

How to do it…

See also

Conditional execution of pipelines

Getting ready

How to do it…

See also

Manual execution

Getting ready

How to do it…

There’s more...

Scheduled execution

How to do it…

Connecting to Bitbucket Pipes

How to do it...

Defining variables

How to do it...

There’s more…

See also

Defining a runner for a pipeline

How to do it...

Testing steps in Bitbucket Pipelines

How to do it…

There’s more…

See also

Security steps in Bitbucket Pipelines

How to do it…

See also

Reporting test results

Getting ready

How to do it…

7

Leveraging Test Case Management and Security Tools for DevSecOps

Technical requirements

Adding test case management to Jira

Getting ready

How to do it…

Connecting Jira to security tools

Getting ready

How to do it…

Managing vulnerabilities

Getting ready

How to do it...

8

Deploying with Bitbucket Pipelines

Technical requirements

Configuring deployments

Getting ready

How to do it…

Pushing artifacts into the Bitbucket repository

Getting ready

How to do it…

Pushing artifacts into artifact repository tools

Getting ready

How to do it…

See also

Deploying artifacts to Bitbucket Downloads

Getting ready

How to do it…

Deploying artifacts using SCP

Getting ready

How to do it...

Deploying artifacts into AWS S3 buckets

Getting ready

How to do it…

Deploying artifacts to Google Cloud

How to do it…

Deploying artifacts to Microsoft Azure

Getting ready

How to do it…

Using Ansible in the deployment stage

Getting ready

How to do it…

There’s more…

See also

Using Terraform in the deployment stage

Getting ready

How to do it…

See also

9

Leveraging Docker and Kubernetes for Advanced Configurations

Technical requirements

Introducing containers and Bitbucket Pipelines

Using a Docker image as a build environment

Getting ready

How to do it…

Using containerized services in Bitbucket Pipelines

Getting ready

How to do it…

Using Docker commands in Bitbucket Pipelines

Getting ready

How to do it…

Deploying a Docker image to Kubernetes using Bitbucket Pipelines

Getting ready

How to do it…

Setting up Docker-based runners on Linux

Getting ready

How to do it…

Part 3: Maintaining Operations

10

Collaborating with Operations through Continuous Deployment and Observability

Technical requirements

Connect Jira with continuous deployment tools

Getting ready

How to do it…

Connect Jira with observability tools

Getting ready

How to do it…

11

Monitoring Component Activity and Metrics Through CheckOps in Compass

Technical requirements

Configuring Compass

Getting ready

How to do it…

There’s more…

Importing distributed architecture components using a CSV file

Getting ready

How to do it…

Integrating Compass with Bitbucket Cloud

How to do it…

Understanding configuration as code in Compass

How to do it…

Creating a developer platform with Compass

How to do it…

There’s more…

See also

Measuring DevOps health with Compass

How to do it…

Utilizing templates in Compass

How to do it…

Implementing developer CheckOps in Compass

How to do it…

There’s more…

See also

12

Escalate Using Opsgenie Alerts

Technical requirements

Setting up Opsgenie teams

How to do it

Setting up Opsgenie with Jira

How to do it

Setting up on-call schedules

How to do it

Configuring escalation policies and rules

How to do it

There’s more

Escalation and notifications configuration

How to do it

Improving team communication and response with ChatOps

How to do it

Part 4: Putting It into Practice

13

Putting It All Together with a Real-World Example

Technical requirements

Creating an idea in JPD

Getting ready

How to do it…

Connecting an idea to an epic in Jira

Getting ready

How to do it…

Creating a story in Jira

Getting ready

How to do it…

Creating a code change in Bitbucket

Getting ready

How to do it…

Committing changes in Bitbucket/Start Bitbucket Pipeline Build

Getting ready

How to do it…

Execute Snyk scanning through Bitbucket Pipelines

Getting ready

How to do it…

Displaying Bitbucket Pipeline Build Status in Compass

Getting ready

How to do it…

Creating an Opsgenie alert for the Jira project

Getting ready

How to do it…

Creating a bugfix branch

Getting ready

How to do it…

Committing a bugfix and watching the pipeline’s execution

Getting ready

How to do it…

14

Appendix – Key Takeaways and the Future of Atlassian DevOps Tools

Atlassian Intelligence and other AI technologies

Best practices for setting up your toolchain

Summary

Index

Other Books You May Enjoy

Preface

DevOps as a movement requires people in teams to examine and change three aspects where they work and interact: people, processes, and tools. Changes for people may include changes in organizational makeup or culture. Process change may include the addition of practices seen in Lean Thinking to ensure continuous delivery. Tool changes allow for quicker turnarounds through automation.

The tools produced by Atlassian can certainly aid our teams with improving processes and fostering automation. Key tools such as Jira and Confluence allow teams to establish standard processes that promote agility and Lean Thinking. Bitbucket Cloud, with its introduction of Bitbucket Pipelines, not only fosters version control but also allows for automation to establish continuous integration and continuous deployment.

But the hallmark of the Atlassian tools isn’t the tools themselves; it’s the ease to which they can connect to each other and to other tools from different manufacturers. Atlassian’s Open DevOps platform provides the ability to create easy interconnections, regardless of vendor.

The focus of this book is not any singular Atlassian product, such as Jira or Confluence. Rather, this book looks to highlight the ease of interconnecting Atlassian tools with other Atlassian tools or other development tools such as GitLab, GitHub, Snyk, and LaunchDarkly. The hope is that by reading this book, you can combine your Atlassian investment with your other investments to create a robust toolchain.

We begin with the assumption that your organization’s current state of tools includes tools from Atlassian and other vendors. Because of this, we do not advise reading this book sequentially. We feel the best use of this book is to read the chapters that apply to connecting your tools, without worrying about making an Atlassian-only stack. These incremental changes follow a Lean approach that is key to adopting DevOps.

We conclude the recipes in this book with an example of what an ideal DevOps toolchain looks like in Chapter 13. While the sample uses mostly Atlassian tools, it can be made with any combination of tools.

Who this book is for

This book is intended for administrators of Atlassian tools and DevOps and DevSecOps practitioners such as developers and site reliability engineers. Administrators will need to know how to create team structures such as Jira projects, Confluence spaces, and Bitbucket workspaces.

What this book covers

Chapter 1, An Introduction to DevOps and the Atlassian Ecosystem, details a brief history of DevOps and showcases recipes for installing Open DevOps or other Atlassian tools from Jira.

Chapter 2, Discovering Customer Needs with Jira Product Discovery, introduces Jira Product Discovery for product managers to develop and prioritize the seeds of products and features until they are ready for teams to develop.

Chapter 3, Planning and Documentation with Confluence, shows the capabilities of connecting Jira with Confluence to enable up-to-date reporting and documentation.

Chapter 4, Enable Connections for Design, Source Control, and Continuous Integration, contains recipes to connect Jira with other source control and continuous integration tools.

Chapter 5, Understanding Bitbucket and Bitbucket Pipelines, introduces Bitbucket Cloud. We look at the source control features and conclude with an initial look at Bitbucket Pipelines.

Chapter 6, Extending and Executing Bitbucket Pipelines, continues our exploration of Bitbucket Pipelines by seeing how to create and edit its primary control file, bitbucket-pipelines.yml.

Chapter 7, Leveraging Test Case Management and Security Tools for DevSecOps, demonstrates recipes to connect to testing and security scan tools and introduces the concept of DevSecOps.

Chapter 8, Deploying with Bitbucket Pipelines, looks at using Bitbucket Pipelines to configure continuous deployment. We demonstrate several examples of deployment to various environments.

Chapter 9, Leveraging Docker and Kubernetes for Advanced Configurations, shows how Docker and Kubernetes can be leveraged by Bitbucket Pipelines to aid in build and deployment.

Chapter 10, Collaborating with Operations through Continuous Deployment and Observability, looks at connecting Jira to tools that provide continuous deployment and monitoring.

Chapter 11, Monitoring Component Activity and Metrics Through CheckOps in Compass, introduces a new Atlassian tool, Compass, to collect and monitor information on deployments and incidents to get a measure of health from project components in a discipline called CheckOps.

Chapter 12, Escalate Using Opsgenie Alerts, demonstrates the use of Opsgenie, which allows for teams to collaborate when an incident occurs.

Chapter 13, Putting It All Together with a Real-World Example, puts everything you have learned from previous chapters together to see how a DevOps toolchain operates.

Chapter 14, Appendix – Key Takeaways and the Future of Atlassian DevOps Tools, finishes with a look at the future and concludes with other tips to get you started on your DevOps journey.

To get the most out of this book

Tool administrators need to know how to create projects in Jira; spaces in Confluence; and workspaces, projects, and repositories in Bitbucket.

Jira users need to know how to create issues.

Confluence users need to know how to create pages.

Bitbucket and other Git server tools (e.g., GitLab and GitHub) users need to know how to perform commits in Git and pull requests/merge requests in the Git server tools of their choice.

Although the Atlassian tools we discuss are cloud-based, Software-as-a-Service (SaaS) applications, some actions or features require external resources. We outline these external resources in the following table:

Action/feature covered in the book

External resource requirements

Connect to Moqups (Chapter 4)

Moqups account

Connect to Figma (Chapter 4)

Figma account

Connect to GitLab (Chapters 4 and 10)

GitLab account

Connect to GitHub (Chapter 4)

GitHub account

Connect to Jenkins (Chapter 4)

Jenkins server

Connect to CircleCI (Chapter 4)

CircleCI account

Bitbucket runner – self-hosted (Chapter 6)

Windows, macOS, or Linux machine

Connect to Snyk (Chapters 7 and 13)

Snyk account

Deploy to JFrog using JFrog CLI

Account in the JFrog environment

Deploy to Sonatype Nexus

Account in the Nexus environment

Deploy to AWS S3 (Chapter 8)

AWS account with appropriate role

Deploy to Google Cloud (Chapter 8)

Google Cloud Platform account with appropriate permissions

Deploy to Microsoft Azure (Chapter 8)

Azure account with appropriate permissions

Connect with monitoring tools (Chapter 10)

Datadog account

If you are using the digital version of this book, we advise you to type the code yourself or access the code via the GitHub repository (link available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Atlassian-DevOps-Toolchain-Cookbook. If there’s an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “On the page, type Jira report in the search bar.”

A block of code is set as follows:

      pipelines:         default:           - step:             script:              - echo "Running a command"

Any command-line input or output is written as follows:

kubectl run <my.app> --labels="app=<my.app>" --image=<my.dockerhub.username>/<my.app>:latest --replicas=2 --port=8080

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: “Select Insights to view links to other sources or create such a link yourself by clicking the Create an insight button.”

Tips or important notes

Appear like this.

Sections

In this book, you will find several headings that appear frequently (Getting ready, How to do it..., How it works..., There’s more..., and See also).

To give clear instructions on how to complete a recipe, use these sections as follows.

Getting ready

This section tells you what to expect in the recipe and describes how to set up any software or any preliminary settings required for the recipe.

How to do it…

This section contains the steps required to follow the recipe.

How it works…

This section usually consists of a detailed explanation of what happened in the previous section.

There’s more…

This section consists of additional information about the recipe in order to make you more knowledgeable about the recipe.

See also

This section provides helpful links to other useful information for the recipe.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Atlassian DevOps Toolchain Cookbook, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/978-1-83546-378-9

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1:Beginning the Cycle

This book is an exploration of DevOps as a whole and where tools fit. Of particular interest are the connections that Atlassian tools can make with each other and with tools from other manufacturers.

We set the stage for our exploration of DevOps and toolchains by understanding the background and context of why this approach is needed. Once we have discovered the “why” of DevOps, we shift attention to Jira and the connections it can make through the Open DevOps platform. We discover how to configure Jira to connect with other tools using this platform.

Before any implementation begins, developers have to work with product managers, stakeholders, and perhaps even the customer to determine the need, viability, and feasibility of a new product. Jira Product Discovery aids this effort of determining which ideas should be acted on by providing a location for collecting, tracking, elaborating, and comparing different ideas for products and features. Those ideas that are ready can be linked to a Jira issue to monitor development.

As we start development, we need to plan our efforts and ensure our efforts are on track. We also want to ensure that the development efforts link to the overall big picture of the product and its promise of value. To that end, we will connect Jira to Confluence so that Confluence can provide the necessary project documentation.

This part has the following chapters:

Chapter 1, An Introduction to DevOps and the Atlassian EcosystemChapter 2, Discovering Customer Needs with Jira Product DiscoveryChapter 3, Planning and Documentation with Confluence

1

An Introduction to DevOps and the Atlassian Ecosystem

DevOps has been a driving force for improvement in product development since its inception in 2009. As technology moved to internet-based, off-premises cloud environments, DevOps allowed people working in development and operations a way to collaborate, enabling quicker and more stable design, packaging, deployment, and maintenance of products. A key component of this is the adoption of automation that ensures these processes run smoother.

In this chapter, we will look at the DevOps movement and the role that automation plays in its success. We will then look at the Open DevOps platform from Atlassian that allows easy connection between Jira, other Atlassian tools such as Confluence and Bitbucket, as well as third-party tools such as LaunchDarkly, which allows you to release products through feature flags, and Snyk, which performs security scanning.

To start our journey into Open DevOps, we will examine the steps needed to enroll in Atlassian’s cloud environment to obtain trial versions of Jira and connections through Open DevOps. To accomplish this, we will cover the following recipes:

Creating the Open DevOps toolchain from scratchCreating a new Atlassian Cloud site with Jira onlyCreating a Jira projectConnecting ConfluenceConnecting Bitbucket

An introduction to DevOps

By 2009, developers began to adopt Agile product development methods. They started with small deliveries in incremental cycles, gathering customer feedback and using it to inform future development cycles. They gradually created a product that their customers would want.

A bottleneck would soon form from delivering changes to the operations part of the organization. While rapid development of value is the priority of development teams, operations teams are charged with maintaining the stability of the environment. Anything that would diminish that stability would mean a potential loss of revenue. Typically, that meant seeing any new change as risky and allowing any changes to be released in specific maintenance windows. These windows only ended up increasing the risk of further downtime.

Changes would soon emerge. During the O’Reilly Velocity 2009 conference, John Allspaw and Paul Hammond of Flickr gave a talk titled 10+ Deploys a Day – Dev and Ops Cooperation at Flickr that described the different practices that allowed them to perform multiple deployments in a single day, in contrast to many organizations that were struggling to perform a deployment in a single year. Patrick Debois, after watching the aforementioned talk and other talks on the importance of collaboration between development and operations, and being intrigued by the concept of Agile system administration following a conversation with Andrew Schafer the year before, decided to organize a conference in Ghent, Belgium, to address the topic. To emphasize this need, Debois added the common abbreviations of development and operations and combined them in the name of his conference – DevOpsDays.

The momentum of the first DevOpsDays conference continued to social media. Discussion continued on Twitter (now known as X), where the topic was identified through the hashtag #DevOps. Additional DevOpsDays conferences were organized in various cities throughout the world. The movement caught the attention of Gartner, who, in 2011, predicted that DevOps would soon be adopted by enterprises. This ensured that DevOps moved from an underground movement to a mainstream idea to implement in business.

Now that the idea of collaboration between development and operations has shown benefits and proven to be popular, one key question is how successful DevOps implementations are started. A key model to that approach is identified in the next section.

The CALMS/CALMR approach

In preparation for the 2010 DevOpsDays conference, John Willis and Damon Edwards were asked the same question about the burgeoning DevOps movement – how do we implement DevOps? In other words, what were the important factors for a successful DevOps implementation?

They came up with an acronym, CAMS, where each letter identified a key component for a successful DevOps approach. C stood for Culture, A represented Automation. M was for Measurement, and S stood for Sharing.

When writing in The DevOps Handbook, Jez Humble elaborated on the CAMS model. He added an L, which stood for Lean. With this addition, CAMS became CALMS.

Scaled Agile Inc., when adopting DevOps into the Scaled Agile Framework (SAFe), modified the approach. Reasoning that the ideal DevOps culture was one of shared responsibility, Scaled Agile removed the S for Sharing and replaced it with an R, which stood for Recovery. CALMS became CALMR when practicing DevOps in SAFe.

Let’s take a look at each letter of the CALMS and CALMR approaches and see how they fit in with the adoption of Atlassian tools and the Open DevOps platform.

Culture

The management expert Peter Drucker is credited with saying, “Culture eats strategy for breakfast,” underlying the impact that culture, as a common thread, can have on uniting individuals in a group, from as small as a team, to as large as nations. So, if the correct culture can drive organizations to desired results, what is the correct culture? To find the answer, we turn to a sociologist named Ron Westrum.

In 1988, Ron Westrum organized a study to measure the safety of medical teams. He organized the teams into three different types of cultures:

PathologicalBureaucraticGenerative

A pathological culture is one that’s leader-driven. Motivation mainly comes through fear and threats from the leaders to accomplish the (leader’s) goals.

A bureaucratic culture has safety mechanisms through rules and standards, but these can then be used to protect the members of the group from outsiders.

A generative culture, in contrast, is focused on aligning with a common mission. Information is freely shared with whomever, whether they are a member of the group or not, irrespective of whether they can play a role in the success of the mission.

Westrum found in his initial study that medical teams that possessed a generative culture also possessed alignment to their mission, an awareness of things that impeded the mission, and gave empowerment to any individual to make changes to remove those impediments. This allowed organizations to easily make long-lasting improvements to the system to improve the teams even more.

These benefits are not limited to medical teams. In the book Accelerate – Building and Scaling High Performing Technology Organizations, Nicole Forsgren, Jez Humble, and Gene Kim investigated recommending DevOps practices to teams to see how effective they were. They surveyed development teams and found that those teams that had a generative culture produced higher levels of performance for software delivery and experienced higher levels of job satisfaction.

A change in culture, while possible, is typically the last change that occurs after structural and behavioral changes. Atlassian does provide tools that assist in the change of structure and behavior to move teams to a generative culture. These tools that assist cultural change are available for free as the Team Playbook (https://www.atlassian.com/team-playbook). However, going into detail about these tools is beyond the scope of this book.

Automation

When people think of DevOps, the first thing that comes to mind is automation. In the talk given by Allspaw and Hammond at the O’Reilly Velocity 2009 conference, one of the key factors of success mentioned included the use of automated infrastructure and a common version control system for both development and operations.

These days, key automation is done by successfully linking tools together to form a toolchain. Each activity in development and operations has at least one tool associated with it.

A diagram that shows development activity with associated Atlassian tools is shown in the following figure:

Figure 1.1 – DevOps phases with Atlassian tools

This book will demonstrate in its chapters how to establish a toolchain with the Atlassian tools illustrated in the preceding figure.

Lean

The system created by automation can only work well if its demand does not exceed its capacity. To ensure this balance, we will look at applying lean thinking practices, initially developed as part of the Toyota Production System. Practices include the following:

Make all work visibleLimit the Work-in-Progress (WIP)Keep batch sizes smallMonitor queues

We will see the relationships between the size of the work (the queue length), the time it takes to process the work (the cycle time), and how long until we see the results (the wait time) from queueing theory. A key formula in this is Little’s law, expressed here:

L signifies the queue length. The Greek letter lambda represents the throughput the team has in processing the work. W represents the wait time for finished work.

An additional equation called Kingman’s formula tells us that there’s a direct correlation between the cycle time, utilization, and variability of the items of work the team must complete. This formula (often called the VUT equation) is shown here:

In this equation, E(W) represents the wait time. It is shown as equivalent to the product of variability (V), utilization (U), and cycle time (T).

Jira has features that allow us to display the work to do through Kanban boards, also allowing us to limit the WIP through column constraints. Wait times and cycle times can also be calculated from the metrics collected.

Measurement

In judging the effectiveness of our efforts to develop, release, and maintain our products, we need to ask three key questions:

Are we on track to deliver the solution?What is the health of all of our operating environments (test, stage, and production)?Do our customers think we have developed the right solution?

We saw when looking at lean practices in the previous section that we needed to pay attention to the following metrics to evaluate whether we operate in a state of flow:

The cycle timeThe WIPThe throughputAny impediments or blockers to progress

In looking at the condition of the operating environments, we turn to the discipline of observability, where we collect all aspects of the environments from infrastructure to applications. The typical measurements for observability include the following:

LogsCall/execution tracesMetrics

Measurements that can indicate whether our customers receive the proposed value from new product features can be tricky to determine. Vanity metrics may exist that occur naturally and indicate good trends, but after thorough analysis, they do not produce actionable answers. Metrics that have proven reliable in measuring customer sentiment include the following:

Pirate metrics (Acquisition, Activation, Retention, Referral, and Revenue) devised by Dave McClure, a look at ideal customer behavior toward a product or feature, with measurements of that behaviorGoogle HEART Metrics, used by Google UI/UX to gauge user preferences in terms of Happiness, Engagement, Adoption, Retention, and Task SuccessFit for Purpose (F4P) devised by David J. Anderson and Alexei Zheglov, which measures whether a customer’s needs are met, as outlined in the book Fit for Purpose – How Modern Businesses Find, Satisfy, and Keep Customers.

Atlassian applications can collect the needed metrics or easily integrate with dedicated metric collection tools. Jira is a proven platform to collect the metrics needed for lean. Out-of-the-box reports, as well as the easy application of third-party marketplace apps, allow us to collect and analyze required metrics. Open DevOps allows easy integration with observability tools such as those offered by Datadog and New Relic.

Sharing

Once we have collected the measurements, we need a way of easily displaying the information to all who need it to foster a generative culture.

The Atlassian tools described in this book provide a good basis to foster information sharing and transparency. Jira can produce charts and reports that can be shared on a dashboard. Other displays related to application health can be collected and displayed using Compass.

Recovery

In any DevOps implementation, we want to devote time and attention to planning contingency steps to take if a release causes an outage in the production environment. DevOps changes the operating model where both development and operations, either acting together or with specialized site reliability engineers, must collaborate to answer the following questions:

How do we reduce the risk when we release?What mitigation steps should be designed to limit the outage time, should it occur?If an outage happens, what procedures should we follow?

Atlassian tools may form the answer to these problems. Feature flag tools such as those provided by LaunchDarkly, easily integrate with Jira through Open DevOps. Compass provides early warning capabilities if an outage appears imminent. Opsgenie allows development and operations to collaborate in an outage to bring about a swift resolution.

We have seen the key pillars of a DevOps approach and how the Atlassian tools help foster them. In the next section, let’s examine the possible technologies that impact DevOps implementation and where Atlassian tools can help.

Technology considerations for DevOps

The adoption of the DevOps approach was encouraged by emerging technologies that made it easier and faster to deploy, release, and maintain products. The changes in technology include the following:

Continuous Integration/Continuous Deployment (CI/CD) pipelinesInfrastructure-as-Code (IaC)Cloud environmentsContainers/Kubernetes

Let’s take a closer look at these factors.

CI/CD pipelines

At the time of Allspaw and Hammond’s talk at the O’Reilly Velocity 2009 conference, CI tools were often used to make a daily build or a build of software that gathered the commits of that day. After creating the build, the CI tools would run automated tests, and report the success or failure of that operation.

Allspaw and Hammond took their CI tool further. If a build passed all its tests, the CI tool would allow you to move the build artifacts to either a test or staging environment for further testing, or to the production environment to prepare for release or go-live.

The extension of CI came to be known as CD. Automating deployment by using a CI/CD tool allows for consistent deployments because steps aren’t forgotten or skipped. Testing of deployment further ensures proper function and that the behavior seen is the desired behavior.

As tooling moved from just automated building to incorporating all steps of CI and CD from a single trigger, version control tools began to offer their own pipeline capabilities, controlled by a text file in the YAML (YAML Ain’t Markup Language) format. This facilitated a mini-movement called GitOps where build, testing, integration, packaging, and deployment all started from a single Git commit as a trigger.

IaC

As CI expanded to include CD, deployment into environments was about to be made easier. New tools such as Ansible, Chef, Terraform, and Puppet allowed for the definition of an ideal infrastructure as specified by text files.

By running the infrastructure tools with the text files as input, there would be a consistent way of creating environments, whether they were used for testing, staging, or production. This consistency helped in keeping environments similar, ensuring the same testing results no matter which environment was used and preventing configuration drift, where problems were seen when environments weren’t similar.

Cloud environments

While CI/CD pipelines and IaC tools could be applied to many physical platforms, from cyber-physical systems that became known as the Internet of Things (IoT) to physical servers, DevOps success has become associated most closely with the rise of cloud environments.

Cloud environments are created with virtual machines available from a vendor and are accessible through the Internet. Creation and disposal of the virtual machines can happen within minutes, allowing for dynamic and flexible setups that could be provisioned on demand.

Containers and Kubernetes

The popularity of cloud environments and IaC has encouraged further thought as to how to package a software application and propagate that to multiple test, staging, and production environments.

Containers have a long history as an approach to isolate processes and their resources, first in Unix and then in Linux. In 2013, Docker was the first company to introduce not only its standard of containers but also a way of managing them. The standard provided by Docker is the most prevalent one used today in describing, creating, and managing containers.

What is a container? If we start with the application we create and how its resources are allocated on a physical computer server, we can see that its code, libraries, and data occupy some of the server’s memory, as illustrated in the following figure:

Figure 1.2 – A packaged application in a physical server

Virtual machines allow a physical server to host multiple instances of Virtual Machines (VMs