33,59 €
Learn Chef Provisioning like a boss and finally own your infrastructure
This book is for Software Engineers, System Administrators, or DevOps Engineers who need to quickly deliver reliably consistent infrastructure at scale. You are expected to have intermediate experience with Chef and Ruby and will be reading this book to advance your knowledge and take your skillset to the next level.
This book will show you the best practices to describe your entire infrastructure as code.
With the help of this book you can expand your knowledge of Chef because and implement robust and scalable automation solutions. You can automate and document every aspect of your network, from the hardware to software, middleware, and all your containers. You will become familiar with the Chef's Chef Provisioning tool. You will be able to make a perfect model system where everything is represented as code beneath your fingertips.
Make the best possible use of your resources, and deliver infrastructure as code, making it as versionable, testable and repeatable as application software
By dedicating a whole book solely to the question of provisioning, this book will teach administrators to use Chef as a birds-eye lens for their entire system. It will moves you away from the specifics of each machine and its automations and instead will teach you them how to approach the entire cluster as something different than the sum of its parts. By focusing on infrastructure as code as its own project, the book offers elegant, time-saving solutions for a perfectly described and automated network.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 339
Veröffentlichungsjahr: 2016
Copyright © 2016 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: June 2016
Production reference: 1230616
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78588-891-5
www.packtpub.com
Author
Earl Waud
Reviewer
Federico Gimenez
Commissioning Editor
Sarah Crofton
Acquisition Editor
Rahul Nair
Content Development Editor
Mamata Walker
Technical Editor
Pramod Kumavat
Copy Editor
Dipti Mankame
Project Coordinator
Kinjal Bari
Proofreader
Safis Editing
Indexer
Monica Ajmera Mehta
Graphics
Kirk D'Penha
Production Coordinator
Shantanu N. Zagade
Cover Work
Shantanu N. Zagade
First and foremost, if you have this book in your hands, congratulations! You are part of the top talent pool in the IT industry, focusing on systems management and provisioning at scale using powerful automation tools.
I have been in the industry for more than 12 years, implementing and leading next generation platform architecture and engineering efforts in world-class enterprise data centers hosting tens of thousands of servers. Throughout my professional journey, I've had a chance to be part of many paradigm shifts, including virtualization, the cloud era, software-defined data centers, and the latest but not the least, infrastructure as a code.
It was during one of these paradigm shifts when I met the author of this book, Earl Waud. Shortly after meeting, and after spilling a cup of coffee on his notebook, we became very close both professionally and as friends.
Earl has always impressed me with how he cares about people, how he values relationships, while still being very professional, methodical, and self-driven to achieve any goal. Additionally, Earl is not short of innovations, driving them all the way from the back of a napkin into highly available multi-data center production deployments that successfully address today's biggest IT challenges. When you combine these traits with his tremendous experience, it becomes a very effective recipe for success both personally and professionally. Mastering Chef Provisioning is the latest example of that success.
In today's IT world, we are experiencing another paradigm shift. In order to support ever-evolving business needs, traditional IT must adopt critical skillsets around infrastructure automation and orchestration. Although there are numerous automation tools available, many professionals have chosen to use Chef because it is one of the most effective platforms to successfully implement provisioning system automation and orchestration.
One of the fundamentally critical and most repeated functions of IT in mission-critical environments is provisioning compute systems. Because of this, provisioning is one of the major targets of automation in IT. In Mastering Chef Provisioning by Earl Waud, the author clearly depicts how you can strategize, design, and implement provisioning automation with Chef across all major hosting platforms anywhere from your own data center all the way out to Amazon Web Services Public Cloud. With the skillset that you will gain from this book, you will be able to implement your own custom provisioning automation to deliver systems rapidly and accurately for all your business needs.
Considering that we are in the middle of another paradigm shift, Mastering Chef Provisioning is one of the most important references that you can have in your library. As a matter of fact, I have already begun using it as a guideline to aid our provisioning automation efforts in delivering thousands of systems in a very short period of time.
I am very confident that you will enjoy this book, and by mastering the techniques within its pages, you will learn how to provision systems effectively with Chef, saving you tremendous amounts of time in your day-to-day operations and reaffirming your position as part of top talent pool in the IT industry.
I would like to conclude by saying thank you to the author of this book, Earl Waud, for leading the way by setting an example for the rest of the world, both at a professional level and a personal level. "We always win!" my friend.
Mert Cubukcuoglu
Senior Manager, Intuit Inc.
Earl Waud is a virtualization development professional with more than 10 years of focused industry experience, creating innovative solutions for hypervisor provisioning, management, and automation. He is an expert in aligning engineering strategy with organizational vision and goals, and delivering highly scalable and user friendly virtualization environments.
With more than 20 years of experience developing customer-facing and corporate IT software solutions, he has a proven track record of delivering high-caliber and on-time technology solutions that significantly impact business results.
Earl lives in San Diego, California. He is blessed with a beautiful wife, Patti, and three amazing daughters, Alexis, Daniella, and Madison.
Currently, Earl is a senior software engineer with Intuit Inc., a company that creates business and financial management solutions that simplify the business of life for small businesses, consumers, and accounting professionals.
Earl can be found online at http://sandiegoearl.com.
Creating this book was a much larger undertaking than I imagined, and as a result, I owe a debt of gratitude to many.
Thank you to my Heavenly Father. Thank you for my days and for my many blessings. Thank you for this book. Thank you for the opportunity to learn and grow as I created its content. Thank you for allowing me to get it right. And thank you for blessing all who read it so that they can find the solutions they need. I love Thee very much. Thank you.
Thank you to my amazing wife Patti. Thank you, honey, for your patience with me and for being okay with all the time I had to invest to create this book. You carried a lot of the weight of our household, so I could research, write, and revise the information within these pages, and without your support, this book would not exist. Thank you for your confidence in me, which helped me get through some moments of self-doubt. And thank you for loving me. I love you very much. Thank you.
Thank you to my daughter Madison. Thank you, Madison, for your light and for your faith in me. Thank you for seeing me the way that you do. Thank you for being willing to give up a lot of our father-daughter time over the past few months. I know how precious that time is, and I hope we can catch up on some of it before you go off to college. I love you very much. Thank you.
Thank you to my daughter Alexis. Thank you, Lexi, for believing in me. And thank you for the confidence and pride you have for me. Thank you for having faith in my ability to be a writer. I love you very much. Thank you.
Thank you to my daughter Daniella. Thank you, Dani, for your energy and unique perspective. Thank you for your support and your belief in me and my ability to write this book. I love you very much. Thank you.
Thank you to my friend Mert Cubukcuoglu. Thank you, Mert, for your friendship and trust. Thank you for your confidence in me and my ability to write a book about Chef provisioning. And thank you for writing the book's Foreword. Thank you.
Thank you to my friends at work. Thank you Ray Eivaz, Barry Ruffner, Wayne Chatham, Achal Shah, Amy Tam, Angela Bouchard, Mai Lubega, Thy Guintivano, Mike Sharp, and the many others at Intuit that I work with every day. Thank you for your excitement about and interest in my book. Thank you for expressing your belief that I could do it. And thank you for your eagerness to read it when it comes out. Thank you Jay Grissom for being one for the first to preorder your copy. Thank you Michel Cole for being the first to request that I write a second book. Thank you.
Thank you to my friend Tom "Big Al" Schreiter. Thank you, Tom, for suggesting that I start writing books and for being such a great example by taking your own advice and creating a fantastic set of training books. And thank you for your excitement about my book. Thank you.
Thank you to my reviewer Federico Gimenez. Thank you, Federico, for poring over the chapters of this book and making sure I've presented the most accurate content and examples possible. Thank you.
Thank you to my Packt Publishing team. Thank you Mamata Walkar, Rahul Nair, Pramod Kumavat, Sachin Karnani, and Kinjal Bari. Without you, this book would never have happened. Thanks for allowing me to officially become an author and for making sure that my book is a good as it can be. Thank you.
"I can no other answer make, but, thanks, and thanks."
--William ShakespeareFederico Gimenez works currently as a software quality engineer for Canonical in the Snappy team. He has an extensive background supporting agile teams in all the product delivery steps, focusing on applying software engineering solutions to multidisciplinary problems. For the past 7 years, he has also contributed to several open source projects, including Docker, Kubernetes, OpenStack, Ceph, and Debian.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.
Today, the DevOps Engineer is responsible for delivering infrastructure that is rock solid and consistently configured every time. What's more, these superheroes need to supply the infrastructure rapidly and at scale, often to the tune of dozens or even hundreds of identically configured systems.
The days of doing this superhero work manually are long gone. Now the only question is "which superpower will best serve our heroes in their time of greatest need?" The answer is the power that can transform their infrastructure into code so that it can be managed the same way as any other source code-based project. Chef is that superpower.
And Mastering Chef Provisioning is your secret decoder ring for learning how to use that power to provision containers, servers, and networking devices across the universe of hosting targets, including your local workstation, the traditional data center, and in the clouds.
In this book, you will learn the secrets of how to automate and document every aspect of your infrastructure. You will find the best practices for describing your infrastructure as code. You will learn to automate the provisioning of everything from the smallest container deployed to your local workstation to the biggest distributed application clusters deployed to the cloud, all in a single bound.
This book will help you, the DevOps superhero, to create a perfect model system where everything is represented as code beneath your fingertips. You will be able to make the best possible use of your resources, avoid redundancy, and always be ready to scale—faster than a speeding bullet. And you don't have to leave your home planet, get bitten by a radioactive spider, or fall in a vat of toxic waste to do it.
But as the great Stan Lee wrote, "With great power …".
Chapter 1, Setting Up a Development Environment on Your Workstation, reviews Chef basics and covers the installation of our Chef workstation and an on-premise Chef Server.
Chapter 2, Knife Tooling and Plugins, shows how to get all the help you need with knife, explores using both common and uncommon knife subcommands, and teaches you how to create your own custom knife plugins.
Chapter 3, Leveraging Roles, Environments, and Policies, examines Chef roles, environments, and organizations, covers the attribute precedence hierarchy, and investigates the exciting new Chef policy feature.
Chapter 4, Custom Resources, introduces the custom resource, explores how the custom resource has improved upon LWRPs and HWRPs, and shows how to create and use your own custom resources.
Chapter 5, Provisioning in the Traditional Data Center, presents provisioning in the traditional data center, covering provisioning to VMware vSphere, OpenStack, VMware's desktop hypervisors, and common network devices, and teaches how to deploy your own development OpenStack environment.
Chapter 6, Provisioning in the Cloud, presents provisioning in the cloud, covering provisioning to Amazon AWS, Microsoft Azure, Google Compute Platform, and Linode, and shows how to provision containers in Docker.
Chapter 7, Test-Driven Development, teaches how to use several of the tools available to implement a test-driven development cycle for infrastructure code development, including RuboCop, Foodcritic, ChefSpec, and Test Kitchen.
Chapter 8, Using Chef Provisioning, reveals how to use the ChefDK feature known as Chef provisioning, providing examples for Vagrant, AWS, and Docker.
The examples in this book were written with the ChefDK version 0.12.0. Examples were tested primarily on Mac OS X and Ubuntu workstations, although all examples should work equally well on other flavors of Linux as well as on Microsoft Windows.
This book is for software engineers, system administrators, and DevOps Engineers who need to quickly deliver reliably consistent infrastructure at scale. You are expected to have intermediate experience with Chef and Ruby and will be reading this book to advance your knowledge and take your skillset to the next level.
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.
Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, and user input are shown as follows: "What this error message is probably indicating is that there is no valid host entry in the /etc/hosts file."
A block of code is set as follows:
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
Any command-line input or output is written as follows:
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Click on the Continue button to advance the installation."
Warnings or important notes appear in a box like this.
Tips and tricks appear like this.
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Chef-Provisioning. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.
In today's fast-paced IT world, it is a requirement to deliver infrastructure at warp speed. There's really only one way to achieve this requirement, and that is through automation.
One of the best paths to infrastructure automation is via Chef. Using Chef, you can turn your infrastructure into code. With infrastructure as code, you can automate the way you build, deploy, and manage all of it. Using Chef, your infrastructure becomes very consistent, very duplicable, and version-controlled. Using Chef, you can easily test your infrastructure setup and configuration. With Chef, you can become an IT superhero!
In this chapter, we're going to detail the setup and configuration of a complete development system or workstation. The main focus will be centered on deploying the Chef Development Kit (ChefDK) and preparing the various components of a development environment. The chapter also includes suggestions for additional tools to round out the DevOps toolbox. However, before we dive into ChefDK, we will fly through a high-level review of the what-and-how of Chef so that we are all on the same page.
"If you wish to make an apple pie truly from scratch, you must first invent the universe." – Carl Sagan
Here is what you will find in this chapter:
If you are reading this book, then you've probably been working with Chef for some time now, and you know the many benefits it brings. You may also know that, in the past, it was a somewhat daunting task to set up a new Chef workstation. You had to download and install Chef, then download and install a variety of community tools, and make sure that all the versions were compatible and configured correctly.
Today, a lot of the work is done for you via the ChefDK. Once you install it, you have a basic workstation setup and are ready to create, modify, and test Chef code.
To make it easy to follow along with the contents, let's go over the versions of the tools that will be used throughout this book.
In March 2016, chef.io announced the release of ChefDK 0.12.0, which includes Chef client 12.8.1. As this is the latest version at the time of writing, I will be using it as the version for this book. All of the examples shown will be based on this version of the ChefDK and Chef client. This is very exciting because a lot of really exciting changes are in this release, including Policies and the transition from Resource Providers to Custom Resources.
Currently, there are ChefDK installers available for Mac OS, Windows OS, and Linux OS (RHEL, Debian, and Ubuntu). In the examples within this book, the workstation used will be Mac OS X, so the ChefDK version will be the Mac OS version.
With the many choices available to use for your Chef Server mode, including Hosted Chef Server, Private Chef Server, Open Source Chef Server, and Chef-Solo, it would be difficult to show examples for each mode. Therefore, the majority of examples you'll see in this book will be based on using the Hosted Chef Server mode. Later in this chapter, I will briefly review the installation and setup of a Private Chef Server onto an Ubuntu server, for readers who want to use Chef Server on-premise.
Which OS do I use for my workstation? Everyone has their own, nearly religious, choice for the best workstation platform. However, it would make this book way too long to provide examples for the major OSes alone. Therefore, to keep the focus on Chef content and not on the differences between workstation implementations, I will be using a Mac OS X (Yosemite version 10.10) workstation for the examples. I may at times show additional examples on a Windows or Ubuntu workstation where the differences are significant and worth the extra detail. And for the nodes used in the examples, a variety of OSes will be represented in the hope of having some overlap with the reader's real environment.
First things first. Let's have a quick, high-level review of Chef. A Chef 101 class, if you will. The plan here is to make sure that all readers are on the same page with me so that the rest of the book will be easier to follow and benefit from. If you are already a Chef ninja, you can probably skip right on past this section and start exploring the ChefDK in the next section. Otherwise, let's audit our 101 class now.
Chef is a ruby framework, and its main purpose is to facilitate the automation, reuse, and documentation of server configurations. Chef allows you to treat server configuration just like any other type of software code.
There are three major components of this framework for almost all corporate-level Chef deployments—the workstation, the Chef Server, and the nodes. Let's talk a little about each of these components:
The Chef client is the tool that is deployed to all nodes and used to configure or reconfigure the node to its desired state. The Chef client is what executes the automation. It is the magician that, through the magic of Chef, transforms an ordinary server into the desired state node that the developer defined back on his workstation.
Ohai is the tool that gathers information about a node. Information such as platform details, operating system data, and processor information is made available to the Chef client so that the latter can have the know-how to bring the node to the desired state. Ohai is executed at the beginning of a Chef client run to gather the state of the node. At the end of a Chef client run, all of the data gathered by Ohai (usually) is shared as node data with the Chef server, and the shared data is available for searches done against the Chef server. Ohai can be extended via plugins, and we'll take a look at that in a later chapter.
Recipes are the building blocks used to define the desired states. Recipes are files of ruby code that define the commands to be run on nodes. They are like blueprints used to "build" a node. Recipes are collections of Chef resources. We will explain more about Chef resources shortly.
Cookbooks are collections of related recipes, templates, files, and custom resources. They provide organization and versioning for recipes. Each unique version of a cookbook represents unique sets of functionality, such as bug fixes or added features.
Cookbooks define a scenario, such as everything needed to install and configure apt-docker or Sublime text, and they contain all the elements needed to support the defined scenario.
Recipes and cookbooks provide modularity and let you easily reuse code.
A "run list" is, as the name would suggest, a list of, and the sequence for, the recipes, cookbooks, and policies (spoiler alert!) to be applied to a node. A run list contains all of the information required to configure a node to a desired state. That is, a Chef run list describes the desired final state of the node. It is important to note that, if the node's state already matches what the resources in the run list describe, then no action will be taken to change the node's state.
Roles are functional groupings of recipes and cookbooks used to describe the full blueprint needed for a node to become everything it is intended to be. Roles are reusable configurations, and they can be applied to multiple nodes to make functionally identical servers, such as a farm of web servers.
Chef resources are statements of configuration policy. They are defined in recipes and take actions through the Chef client to put the node into the desired state. Chef resources have some types: Package, Template, Service, and so on. They have a name and parameters. Also, Chef resources can send notifications to other resources.
Chef resources define what we want to happen to a node. They don't say how to do it. The how to do it is left to the providers which are platform-specific. That is to say, the way you install a package will be different depending on the OS, and the provider determines the correct way to do it—the "how". The Chef resource simply defines the "what," such as "install ntp".
Apart from the three types of Chef resources we saw earlier, let's look at some other important aspects of recipes and cookbooks.
Chef resources are idempotent. That is, applying them more than once results in the same outcome every time. If no inputs related to the resource have changed, then applying that resource won't change anything after the first application. In fact, if none of the inputs have changed, the corresponding commands don't even get run.
This is actually one of the most important concepts and features of Chef.
Data bags are containers for information that is not tied to a specific node. Data bags are the global variables of a Chef server. They can be used in recipes and can be searched like node data via the Chef server. One common use case for data bags is for user and group information. Items in a data bag can be encrypted. This allows secret information to be stored in them, for example, passwords.
Environments allow you to define specific cookbook versions that are applied to a given set of nodes. They permit you to model the stages of your infrastructure workflow, that is, Development, Test, Stage, and Production. By identifying specific nodes as Development, and other nodes as Production, for example, you can apply different versions of your cookbooks to the nodes based on their environment membership.
The supermarket is a site that provides shared cookbooks. There is a public supermarket that contains community-created and -maintained cookbooks. This site is hosted by Chef and is available at https://supermarket.chef.io. In addition to the public supermarket, anyone can create and manage a private supermarket that can host cookbooks intended for the private consumption of you and your organization.
It is always best practice to do a thorough code review of any cookbooks obtained from the public community supermarket before using them in your production environments.
The ChefDK contains everything you need to start working with Chef on a workstation. It provides all the tools that a developer needs to create and modify cookbooks and upload them to a Chef server. We are going to go into a lot of detail regarding the ChefDK later in this chapter.
You can jump ahead to the ChefDK sections now, or you can read on and learn how to set up your very own private Chef server.
Many companies will have strong reasons to keep their infrastructure configuration data within the firewalls of their datacenters, and Chef server has a mode for that. It's called on-premise or private Chef server.
There are some real benefits to using an in-house solution, including control. With a private installation, you have full control of your system, and for some that reason is enough. But there are additional considerations such as performance. Since a private Chef server will likely be physically closer to the node's network, Chef client runs will be faster. For example, if Chef is used in a load-based on-demand server deployment solution, then the extra speed the private Chef server can provide when converging your new Nodes can make all the difference in meeting the load demand in time.
With all the reasons to use an on-premise Chef server, there is a downside: you are responsible for the deployment, configuration, and maintenance of your Chef servers.
To get you started, let's take a look at the
