37,19 €
Help your organization join the DevOps revolution
This book is written for engineers and companies that want to learn the minimum set of required technologies and processes to be successful in the DevOps world.
This book also targets system administrators, developers, and IT professionals who would like to employ DevOps techniques and best practices to manage IT infrastructures or would like to acquire the necessary skills needed to work in DevOps teams.
This book follows a unique approach to modern DevOps using cutting-edge tools and technologies such as Ansible, Kubernetes, and Google Cloud Platform.
This book starts by explaining the organizational alignment that has to happen in every company that wants to implement DevOps in order to be effective, and the use of cloud datacenters in combination with the most advanced DevOps tools to get the best out of a small team of skilled engineers. It also delves into how to use Kubernetes to run your applications in Google Cloud Platform, minimizing the friction and hassle of maintaining a cluster but ensuring its high availability.
By the end of this book, you will be able to realign teams in your company and create a Continuous Delivery pipeline with Kubernetes and Docker. With strong monitoring in place, you will also be able to react to adverse events in your system, minimizing downtime and improving the overall up-time and stability of your system.
This book takes a step-by-step practical approach to the implementation of DevOps. This book will teach you how to enable IT organizations to deliver faster and smarter through a unique approach using Code-Build-Test-Release-Configure-Monitor (CBTRCM).
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 357
Veröffentlichungsjahr: 2017
BIRMINGHAM - MUMBAI
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: October 2017
Production reference: 1290917
ISBN 978-1-78646-687-7
www.packtpub.com
Author
David Gonzalez
Copy Editor
Stuti Srivastava
Reviewer
Joakim Verona
Project Coordinator
Virginia Dias
Commissioning Editor
Kartikey Pandey
Proofreader
Safis Editing
Acquisition Editor
Divya Poojari
Indexer
Pratik Shirodkar
Content Development Editor
Sharon Raj
Graphics
Kirk D'Penha
Technical Editor
Prashant Chaudhari
Production Coordinator
Deepika Naik
David Gonzalez is an enthusiastic engineer and author of a book called Developing Microservices with Node.js; as microservices don't work without platform automation, he wrote this book to deliver the best possible deal to the readers of both books.
He is a Google Developer Expert (a nomination from Google to certain experts in several areas) in Kubernetes (GKE), who enjoys being pushed out of his comfort zone in order to sharpen his skills. Java, Node.js, Python, and DevOps--as well as a holistic approach to security--are part of the skill set that has helped him deliver value across different start-ups and corporations.
Nowadays, he is a consultant at nearForm, enabling companies to deliver the best possible solution to their IT problems or proposals, as well as an avid speaker at conferences, such as RebelCon and Google I/O Extended, among others.
Joakim Verona is a consultant with expertise in continuous delivery and DevOps. He has worked with all aspects of systems development since 1994. He has actively contributed as the lead implementer of complex multilayered systems, such as web systems, multimedia systems, and mixed software/hardware systems. His wide-ranging technical interests led him to the emerging field of DevOps in 2004, where he has stayed ever since. Joakim completed his master's in computer science at Linköping Institute of Technology. He has also worked as a consultant in a wide range of assignments in various industries, such as banking and finance, telecom, industrial engineering, press and publishing, and game development. He is also interested in the Agile field and is a certified Scrum master, Scrum product owner, and Java professional.
He has also technically reviewed Practical DevOps.
For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.comand as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1786466872.
If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
DevOps in the Real World
What is DevOps?
DevOps origins
DevOps and corporations
Traditional release management
Chain of communication
The cost of fixing a bug
Releasing new versions
Modern release management
Agile development and communication
Releasing new versions
DevOps and microservices
DevOps: Organizational alignment
What to expect from this book
Summary
Cloud Data Centers - The New Reality
Amazon Web Services
EC2 - computing service
Launching an instance
Relational Database Service
Networking in AWS and EC2
Storage in AWS and EC2
Amazon S3
Amazon ECR and ECS
Creating a cluster
Other services
Route 53
CloudFront
Amazon ElasticCache
Amazon RDS
DynamoDB
Google Cloud Platform
Google Compute Engine
Standard machine types
High-memory machine types
High-CPU machine types
Shared-core machine types
Custom machines and GPU processing
Launching an instance
Networking
Google Container Engine
Setting up a cluster
Other Google Cloud Platform products
Google App Engine
Machine Learning APIs
Big data
Other cloud providers
Heroku
DigitalOcean
Azure
Summary
Docker
Docker architecture
Docker internals
Docker client
Building Docker images
Dockerfile reference
Docker registries
Public registries
Private registries
Docker volumes
Docker networking
User-defined networks
Docker Compose
Summary
Continuous Integration
Software development life cycle
Testing types
Traditional CI servers
Bamboo
Jenkins
Secrets Management
Modern CI servers
Drone CI
Installing Drone
Running builds
Executing pipelines
Other features
Summary
Infrastructure as Code
Google Cloud Platform SDK - gcloud
Creating resources with Google Cloud SDK
Terraform
Creating resources
Remote state management
Modifying your infrastructure
Terraform variables
Terraform outputs
Summary
Server Provisioning
Server provision software
Chef
Puppet
Ansible
Ansible
Ansible configuration
Ansible variables, remote facts and templates
Ansible variables
Ansible remote facts
Ansible templates
Flow control
Roles
Ansible Tower
Summary
Docker Swarm and Kubernetes - Clustering Infrastructure
Why clustering ?
Docker Swarm
Kubernetes
Kubernetes logical architecture
Setting up a cluster in GCP
Kubernetes building blocks
Pods
Deploying a pod
Replica Sets
Deployments
Services
Other Building Blocks
Daemon Sets
PetSets
Jobs
Secrets and configuration management
Kubernetes- moving on
Summary
Release Management – Continuous Delivery
Playing against the statistics
The test system
ISO date and UTC date services
Aggregator service
Pushing the images to Google Container Registry
Setting up a continuous delivery pipeline for images
Setting up Jenkins
Continuous delivery for your application
Regular release
Blue-green deployment
Canary deployment
Summary
Monitoring
Introduction
Types of monitoring
Blackbox monitoring
Whitebox monitoring
Monitoring third-party tools
Pingdom
Logentries
AppDynamics
Stackdriver
Monitoring applications
Monitoring Kubernetes clusters
Summary
What is next?
DevOps is the newest revolution in deploying software quickly and efficiently. With a set of automation tools, an orchestration platform, and a few processes, companies can speed up the release cycle of their IT systems by enabling the engineers to do more with fewer resources and become more engaged in the business process.
Chapter 1, DevOps in the Real World, shows the place of DevOps in the current engineering department of IT companies and how to align resources to maximize delivery potential.
Chapter 2, Cloud Data Centers, compares the different cloud solutions for managing resources (VMs, networks, disks, and so on) on the cloud and on demand.
Chapter 3, Docker, teaches about Docker and some of its internals in order to better understand how containerization technologies work.
Chapter 4, Continuous Integration, talks about continuous integration technologies that can be used to execute tests across your applications as well as many other actions, as we will see in Chapter 8, Release Management – Continuous Delivery.
Chapter 5, Infrastructure as Code, shows how to describe our infrastructure in a way that can be managed as code and apply the SDLC best practices to it in order to ensure its integrity.
Chapter 6, Server Provisioning, shows how to use Ansible to manage the configuration of remote servers in order to facilitate the maintenance of a large number of servers that, even though we are going to focus on Kubernetes, are good to know.
Chapter 7, Docker Swarm and Kubernetes - Clustering Infrastructure, briefly visits Docker Swarm and then points your attention toward Kubernetes, the most modern container orchestration technology, which is used across the biggest corporations in the world, such as Google.
Chapter 8, Release Management – Continuous Delivery, shows how to set up a continuous delivery pipeline on Google Cloud Platform with Kubernetes and Jenkins.
Chapter 9, Monitoring, shows how to monitor our software and servers to be the first ones to know about a potential outage very quickly and fix it (potentially) before impacting our customers.
In order to follow this book and its contents, you will need a trial account on Google Cloud Platform and an editor (I used Atom but any other editor will work) as well as Node.js installed on your local machine. You will also need to install Docker locally in order to test the different examples. We will use Google Container Engine (GKE) for the Kubernetes examples but if you want to play locally with Kubernetes, Minikube can also be used, although you need a fairly powerful computer.
This book is for engineers who want to step up in the DevOps ladder, particularly if they want to master Kubernetes and containers. People of mid-level skills are ideal readers.
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply email [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files emailed directly to you. You can download the code files by following these steps:
Log in or register to our website using your email address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Implementing-Modern-DevOps. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/ImplementingModernDevOps_ColorImages.pdf.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
In the past few years, the software world has evolved at a very high pace. One of my favorite examples of evolution is FinTech, a new field whose name comes from the fusion of finance and technology. In this field, companies tend to build financial products in a disruptive way up to a point that they are threatening the big traditional banks and putting them in jeopardy.
This happens mainly due to the fact that big companies lose the ability to be cost-effective in their IT systems and banks are fairly big companies. It is not strange that banks still run their systems in an IBM mainframe and are reluctant to move to the cloud, and it is also not strange that the core components of the banks are still COBOL applications that haven't been renewed since the 90s. This wouldn't be bad if it wasn't because a small number of talented engineers with an AWS or Google Cloud Platform account can actually build a service that could virtually replace some bank products such as currency exchange or even a broker.
This has become a norm in the last few years, and one of the keys for the success of small companies in FinTech is partially due to DevOps and partially due to its scale. Usually, big companies commoditize the IT systems over time, outsourcing them to third parties that work on price, pushing the quality aside. This is a very effective cost-cutting measure, but it has a downside: you lose the ability to deliver value quickly.
In this chapter, we are going to put DevOps into perspective and see how it can help us create cost-effective work units that can deliver a lot of value in a very short period of time.
There is a famous quote by Henry Ford, the creator of Ford (the popular car-maker brand):
This is what happened with the traditional system administrator role: people were trying to solve the wrong problem.
By the wrong problem, I mean the lack of proper tools to automate the intervention in production systems, avoiding the human error (which is more common than you may think) and leading to a lack of communication continuity in the processes of your company.
Initially, DevOps was the intersection of development and operations as well as QA. The DevOps engineer is supposed to do everything and be totally involved in the SDLC (software development life cycle), solving the communication problems that are present in the traditional release management. This is ideal and, in my opinion, is what a full stack engineer should do: end-to-end software development, from requirement capture to deployments and maintenance.
Nowadays, this definition has been bent up to a point where a DevOps engineer is basically a systems engineer using a set of tools to automate the infrastructure of any company. There is nothing wrong with this definition of DevOps, but keep in mind that we are losing a very competitive advantage: the end-to-end view of the system. In general, I would not call this actor a DevOps engineer but an Site reliability engineering (SRE). This was a term introduced by Google few years back, as sometimes (prominently in big companies), is not possible to provide a single engineer with the level of access required to execute DevOps. We will talk more about this role in the next section, SRE model.
In my opinion, DevOps is a philosophy more than a set of tools or a procedure: having your engineers exposed to the full life cycle of your product requires a lot of discipline but gives you an enormous amount of control over what is being built. If the engineers understand the problem, they will solve it; it is what they are good at.
In the last few years, we have gone through a revolution in IT: it sparkled from pure IT companies to all the sectors: retail, banking, finance, and so on. This has led to a number of small companies called start-ups, which are basically a number of individuals who had an idea, executed it, and went to the market in order to sell the product or the service to a global market (usually). Companies such as Amazon or Alibaba, not to mention Google, Apple, Stripe or even Spotify, have gone from the garage of one of the owners to big companies employing thousands of people.
One thing in common in the initial spark with these companies has always been corporate inefficiency: the bigger the company, the longer it takes to complete simple tasks.
This phenomenon creates a market on its own, with a demand that cannot be satisfied with traditional products. In order to provide a more agile service, these start-ups need to be cost-effective. It is okay for a big bank to spend millions on its currency exchange platform, but if you are a small company making your way through, your only possibility against a big bank is to cut costs by automation and better processes. This is a big drive for small companies to adopt better ways of doing things, as every day that passes is one day closer to running out of cash, but there is a bigger drive for adopting DevOps tools: failure.
Failure is a natural factor for the development of any system. No matter how much effort we put in, failure is always there, and at some point, it is going to happen.
Usually, companies are quite focused on removing failure, but there is a unwritten rule that is keeping them from succeeding: the 80-20 rule:
It takes 20% of time to achieve 80% of your goals. The remaining 20% will take 80% of your time.
Spending a huge amount of time on avoiding failure is bound to fail, but luckily, there is another solution: quick recovery.
Up until now, in my work experience, I have only seen one company asking "what can we do if this fails at 4 A.M. in the morning?" instead of "what else can we do to avoid this system from failing?", and believe me, it is a lot easier (especially with the modern tools) to create a recovery system than to make sure that our systems won't go down.
All these events (automation and failure management) led to the development of modern automation tools that enabled our engineers to:
Automate infrastructure and software
Recover from errors quickly
DevOps fits perfectly into the small company world (start-ups): some individuals that can access everything and execute the commands that they need to make the changes in the system quickly. Within these ecosystems is where DevOps shines.
This level of access in traditional development models in big companies is a no-go. It can be an impediment even at a legal level if your system is dealing with highly confidential data, where you need to get your employees security clearance from the government in order to grant them access to the data.
It can also be convenient for the company to keep a traditional development team that delivers products to a group of engineers that runs it but works closely with the developers so that the communication is not an issue.
SREs also use DevOps tools, but usually, they focus more on building and running a middleware cluster (Kubernetes, Docker Swarm, and so on) that provides uniformity and a common language for the developers to be abstracted from the infrastructure: they don't even need to know in which hardware the cluster is deployed; they just need to create the descriptors for the applications that they will deploy (the developers) in the cluster in an access-controlled and automated manner in a way that the security policies are followed up.
SRE is a discipline on its own, and Google has published a free ebook about it, which can be found at https://landing.google.com/sre/book.html.
I would recommend that you read it as it is a fairly interesting point of view.
Through the years, companies have pushed the development of their IT systems out of their business core processes: retail shop business was retail and not software but reality has kicked in very quickly with companies such as Amazon or Alibaba, which can partially attribute their success to keeping their IT systems in the core of the business.
A few years ago, companies used to outsource their entire IT systems, trying to push the complexity aside from the main business in the same way that companies outsource the maintenance of the offices where they are. This has been successful for quite a long time as the release cycles of the same applications or systems were long enough (a couple of times a year) to be able to articulate a complex chain of change management as a release was a big bang style event where everything was measured to the millimeter with little to no tolerance for failure.
Usually, the life cycle for such projects is very similar to what is shown in the following diagram:
This model is traditionally known as waterfall (you can see its shape), and it is borrowed from traditional industrial pipelines where things happen in very well-defined order and stages. In the very beginning of the software industry, engineers tried to retrofit the practices from the traditional industry to software, which, while a good idea, has some drawbacks:
Old problems are brought to a new field
The advantages of software being intangible are negated
With waterfall, we have a big problem: nothing moves quickly. No matter how much effort is put into the process, it is designed for enormous software components that are released few times a year or even once a year. If you try to apply this model to smaller software components, it is going to fail due to the number of actors involved in it. It is more than likely that the person who captures the requirements won't be involved in the development of the application and, for sure, won't know anything about the deployment.
I remember that when I was a kid, we used to play a game called the crazy phone. Someone would make up a story with plenty of details and write it down on paper. This person read the story to another person, who had to capture as much as possible and do the same to the next person, up until we reached the end of the number of people playing this game. After four people, it was almost guaranteed that the story wouldn't look anywhere close to the initial one, but there was a more worrying detail: after the first person, the story would never be the same. Details would be removed and invented, but things would surely be different.
This exact game is what we are trying to replicate in the waterfall model: people who are working on the requirements are creating a story that is going to be told to developers, who are creating another story that is going to be told to QA so that they can test that the software product delivered matches with a story that was in two hands (at the very least) before reaching them.
As you can see, this is bound to be a disaster but hold on, what can we do to fix it? If we look at the traditional industry, we'll see that they never get their designs wrong or, at least, the error rate is very small. The reason for that (in my opinion) is that they are building tangible things, such as a car or a nuclear reactor, which can easily be inspected and believe me or not, they are usually simpler than a software project. If you drive a car, after a few minutes, you will be able to spot problems with the engine, but if you start using a new version of some software, it might take a few years to spot security problems or even functional problems.
In software, we tried to ease this problem by creating very concise and complex diagrams using Unified Modeling Language(UML) so that we capture the single source of truth and we can always go back to it to solve problems or validate our artifacts. Even though this is a better approach, it is not exempt from problems:
Some details are hard to capture in diagrams
People in the business stakeholders do not understand UML
Creating diagrams requires time
Particularly, the fact that the business stakeholders do not understand UML is the big problem here. After the capture of requirements, changing them or even raising questions on lower levels (development, operations, and so on) requires involving some people, and at least one of them (the business stakeholder) does not understand the language of where the requirements were captured. This wouldn't be a problem if the project requirements were spot on since the first iteration, but in how many projects have you been involved where the requirements were static? The answer is none.
Once we have made it clear that we have a communication problem, bugs are expected to arise during our process. Either a misalignment with the requirements or even the requirements being wrong usually leads to a defect that could prevent us from deploying the application to production and delay everything.
In waterfall, fixing a bug is increasingly possible in every step we take. For example, fixing a bug in the requirements phase is very straightforward: just update the diagrams/documentation, and we are done. If the same bug is captured by a QA engineer in the verification phase, we need to:
Update the documents/diagrams
Create a new version of the application
Deploy the new version to the QA environment
If the bug is caught in production, you can imagine how many steps are involved in fixing it, not to mention the stress, particularly if the bug compromises the revenue of your company.
A few years ago, when I was in the middle of a manual deployment at 4 A.M., I remember asking myself "there has to be a better way". Tools were not mature enough, and the majority of the companies did not consider IT the core of their business. Then, a change happened: DevOps tools started to do well in the open source community and companies started to create continuous delivery pipelines. Some of them were successful, but a big majority of them failed for two reasons:
Release management process
Failure in the organizational alignment
We will talk about organizational alignment later on in this chapter. For now, we are going to focus on the release management process as it needs to be completely different from the traditional release management in order to facilitate the software life cycle.
In the preceding section, we talked about different phases:
Requirements
Design
Implementation
Verification
Maintenance
We also explained how it works well with gigantic software where we group features into big releases that get executed in a big bang style with all or nothing deployments.
The first try to fit this process into smaller software components was what everyone calls agile, but no one really knew what it was.
In the traditional release management, one of the big problems was the communication: chains of people passing on messages and information, as we've seen, never ends well.
Agile encourages shorter communication strings: the stakeholders are supposed to be involved in the software development management, from the definition of requirements to the verification (testing) of the same software. This has an enormous advantage: teams never build features that are not required. If deadlines need to be met, the engineering team sizes down the final product sacrificing functionality but not quality.
Deliver early and deliver often is the mantra of agile, which basically means defining an Minimum Viable Product (MVP) and delivering it as soon as it is ready in order to deliver value to the customers of your application and then delivering new features as required. With this method, we are delivering value since the first release and getting feedback very early on in the product life.
In order to articulate this way of working, a new concept was introduced: the sprint. A sprint is a period of time (usually 2 weeks) with a set of functionalities that are supposed to be delivered at the end of it into production so that we achieve different effects:
Customers are able to get value very often
Feedback reaches the development team every 2 weeks so that corrective actions can be carried on
The team becomes predictable and savvy with estimates
This last point is very important: if our estimates are off by 10% in a quarter release, it means that we are off by two weeks, whereas in a two weeks sprint, we are off only by 1 day, which, over time, with the knowledge gained sprint after sprint, means the team will be able to adjust due to the fact that the team builds a database of features and time spent on them so that we are able to compare new features against the already developed ones.
These features aren't called features. They are called stories. A story is, by definition, a well-defined functionality with all the info for the development team captured before the sprint starts, so once we start the development of the sprint, developers can focus on technical activities instead of focusing on resolving unknowns in these features.
Not all the stories have the same size, so we need a measurement unit: the story points. Usually, story points do not relate to a time-frame but to the complexity of it. This allows the team to calculate how many story points can be delivered at the end of the sprint, so with time, they get better at the estimates and everybody gets their expectations satisfied.
At the end of every sprint, the team is supposed to release the features developed, tested, and integrated into production in order to move to the next sprint.
The content of the sprints are selected from a backlog that the team is also maintaining and preparing as they go.
The main goal is to meet everyone's expectations by keeping the communication open and be able to predict what is being delivered and when and what is needed for it.
There are several ways of implementing the agile methodologies in our software product. The one explained earlier is called Scrum, but if you look into other development methodologies, you'll see that they all focus on the same concept: improving the communication across different actors of the same team.
If you are interested in Scrum, there is more info at https://en.wikipedia.org/wiki/Scrum_(software_development).