Mastering Chef Provisioning - Earl Waud - E-Book

Mastering Chef Provisioning E-Book

Earl Waud

0,0
33,59 €

-100%
Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.

Mehr erfahren.
Beschreibung

Learn Chef Provisioning like a boss and finally own your infrastructure

About This Book

  • This is the first Chef book focused on provisioning infrastructure as its sole task. The book offers a clear solution to a specific pain point: learn to make your system work faster.
  • Learning better approaches to load balancing and parallelization with this book will save you time
  • By mastering the techniques in this book, you will know how to run an entire fleet of machines without breaking a sweat
  • This book is more helpful than the documentation ( https://docs.chef.io/provisioning.html), with a stronger guiding voice and clearer explanations and use cases

Who This Book Is For

This book is for Software Engineers, System Administrators, or DevOps Engineers who need to quickly deliver reliably consistent infrastructure at scale. You are expected to have intermediate experience with Chef and Ruby and will be reading this book to advance your knowledge and take your skillset to the next level.

What You Will Learn

  • Use best practices to describe your entire infrastructure as code
  • Automate and document every aspect of your network, from the hardware of individual nodes to software, middleware, and all containers and clouds
  • Create a perfect model system
  • Make the best possible use of your resources and avoid redundancy
  • Deliver on the promise of Infrastructure as Code
  • Scale with ease by properly provisioning their infrastructure
  • Use the best Test Driven Development methodologies

In Detail

This book will show you the best practices to describe your entire infrastructure as code.

With the help of this book you can expand your knowledge of Chef because and implement robust and scalable automation solutions. You can automate and document every aspect of your network, from the hardware to software, middleware, and all your containers. You will become familiar with the Chef's Chef Provisioning tool. You will be able to make a perfect model system where everything is represented as code beneath your fingertips.

Make the best possible use of your resources, and deliver infrastructure as code, making it as versionable, testable and repeatable as application software

Style and approach

By dedicating a whole book solely to the question of provisioning, this book will teach administrators to use Chef as a birds-eye lens for their entire system. It will moves you away from the specifics of each machine and its automations and instead will teach you them how to approach the entire cluster as something different than the sum of its parts. By focusing on infrastructure as code as its own project, the book offers elegant, time-saving solutions for a perfectly described and automated network.

Sie lesen das E-Book in den Legimi-Apps auf:

Android
iOS
von Legimi
zertifizierten E-Readern

Seitenzahl: 339

Veröffentlichungsjahr: 2016

Bewertungen
0,0
0
0
0
0
0
Mehr Informationen
Mehr Informationen
Legimi prüft nicht, ob Rezensionen von Nutzern stammen, die den betreffenden Titel tatsächlich gekauft oder gelesen/gehört haben. Wir entfernen aber gefälschte Rezensionen.



Table of Contents

Mastering Chef Provisioning
Credits
Foreword
About the Author
Acknowledgments
About the Reviewer
www.PacktPub.com
eBooks, discount offers, and more
Why subscribe?
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
1. Setting Up a Development Environment on Your Workstation
Re-introducing Chef
What version of the Chef tools do I use?
References
Filling in the gaps in your Chef knowledge base
Major pieces of a Chef puzzle
The Chef client
The Ohai Chef tool
Recipes and cookbooks
Chef run lists
Chef roles
Chef resources
More things to consider
Data bags
Environments
The Chef supermarket
Chef Development Kit
Setting up an on-premise (private) Chef Server
References
Exploring the Chef Development Kit
The Chef Knife
Community tools
References
Installing Chef Development Kit
Downloading ChefDK
Installing ChefDK
Verifying ChefDK
References
Setting up a standard Chef repo
Using the Chef tool for Chef repo
The repo starter kit
Files for your .chef folder – a safer way
Growing your own tree
Self-signed certificates
References
What else do you need?
Using your favorite editor
Version control systems
Virtualization hypervisor
Vagrant
References
What is chef-apply?
Why is that useful?
References
Summary
2. Knife Tooling and Plugins
Knife help
References
Knife subcommands
Knife Bootstrap
Knife Node
Knife Diff
Knife Exec
Knife SSH
Knife Search
References
Knife to editor integration
Using the "wait" option
Does my editor require the wait option?
References
Knife Windows plugin
Installing the Knife Windows plugin
Bootstrapping a Windows node
Searching for Windows node data
References
Knife EC2 plugin
Installing the Knife EC2 plugin
Creating an EC2 instance
Bootstrapping an existing EC2 instance
Deleting EC2 instances
References
Creating new Knife plugins
The basic plugin file format
Example – A simpler EC2 Create Server plugin
References
Summary
3. Leveraging Roles, Environments, and Policies
Using Chef roles
How to create a role
Using per-environment run-lists in roles
Using nested roles
There is a downside to roles
References
Using Chef environments
The _default environment
Creating environments
Specifying cookbook versions in environments
References
Using Chef organizations
Creating organizations
Access control
References
Learning the attribute precedence hierarchy
Understanding attribute precedence factors
Resulting precedence hierarchy
Exploring policies
What exactly is a policy?
Why would I use a policy?
How do I use a policy?
Another example
How to specify the source options for cookbooks
What are the "Gotchas"?
References
Summary
4. Custom Resources
Out with the old (LWRPs), in with the new (Custom Resources)
What is the difference between the old way and the new way of defining resources?
What is a Custom Resource?
References
Creating an example use case
What is the goal?
Creating the cookbook
Creating the default recipe
The recipe worked, but what is missing?
References
Creating and using a Custom Resource
What makes up a Custom Resource?
Creating the "website" Custom Resource
Using our Custom Resource
References
Making a Custom Resource more reusable
Making a Custom Resource-only cookbook
Reuse the code from the "mywebapp" example
Using the reusable Custom Resource-only cookbook
Handling idempotence
References
Subclassing a Custom Resource
Convert our resource into a library module
Subclassing the library module Custom Resource
References
Summary
5. Provisioning in the Traditional Data Center
VMware provisioning
Installing the knife-vsphere plugin
Provisioning VMs using the knife-vsphere plugin
Cleaning up our vSphere and Chef environments
Using Chef with VMware and Vagrant
References
OpenStack provisioning
Setting up a DevStack OpenStack all-in-one environment
Deploying the DevStack Host VM
Downloading and installing DevStack
Downloading the demo project rc file and applying it
Adding the security rules to allow ping and ssh traffic
Installing the knife OpenStack plugin
Setting up the DevStack public key
Allocating some floating IP addresses to the project
Configuring the knife-openstack plugin knife settings
Adding a CentOS instance image to provision
Provisioning a new instance
References
Network automation
Automating juniper devices
Automating Cisco systems
Automating F5 LTMs
References
Summary
6. Provisioning in the Cloud
Provisioning AWS EC2 instances
Understanding EC2 authentication
Creating key pairs
Creating security groups
Installing Chef Server into an EC2 instance
References
Provisioning Microsoft Azure instances
Installing the knife-azure plugin
Configuring the plugin
Verifying the configuration
Generating public/private key pairs
Provisioning virtual machines
Provisioning a new Chef server into Azure
References
Provisioning in the Google Cloud platform
Installing and configuring the knife-google plugin
Creating and registering your key pairs
Provisioning virtual machines with the GCE
Deploying Chef server into a GCE instance
References
Looking at an honorable mention – Linode
References
Provisioning containers in Docker
Setting up a Docker server to host our containers
Installing and using Docker on our Chef Workstation
Using the knife-docker plugin
References
Summary
7. Test-Driven Development
Analyzing cookbooks with RuboCop
Using RuboCop
Examining the RuboCop output
Customizing RuboCop
References
Analyzing cookbooks with Foodcritic
Using Foodcritic
Examining the Foodcritic output
Customizing Foodcritic
References
Unit testing with ChefSpec
Using ChefSpec
Expanding our tests
References
Integration testing with Kitchen and ServerSpec
Preparing our tests
Kitchen create
Kitchen converge
Kitchen verify
References
Summary
8. Using Chef Provisioning
What is Chef provisioning?
It's highly idempotent
A hypothetical example
References
How to use Chef Provisioning
Chef Provisioning resources
A Vagrant example
A cloud example with AWS
A Docker example
References
Summary
Index

Mastering Chef Provisioning

Mastering Chef Provisioning

Copyright © 2016 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: June 2016

Production reference: 1230616

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-78588-891-5

www.packtpub.com

Credits

Author

Earl Waud

Reviewer

Federico Gimenez

Commissioning Editor

Sarah Crofton

Acquisition Editor

Rahul Nair

Content Development Editor

Mamata Walker

Technical Editor

Pramod Kumavat

Copy Editor

Dipti Mankame

Project Coordinator

Kinjal Bari

Proofreader

Safis Editing

Indexer

Monica Ajmera Mehta

Graphics

Kirk D'Penha

Production Coordinator

Shantanu N. Zagade

Cover Work

Shantanu N. Zagade

Foreword

First and foremost, if you have this book in your hands, congratulations! You are part of the top talent pool in the IT industry, focusing on systems management and provisioning at scale using powerful automation tools.

I have been in the industry for more than 12 years, implementing and leading next generation platform architecture and engineering efforts in world-class enterprise data centers hosting tens of thousands of servers. Throughout my professional journey, I've had a chance to be part of many paradigm shifts, including virtualization, the cloud era, software-defined data centers, and the latest but not the least, infrastructure as a code.

It was during one of these paradigm shifts when I met the author of this book, Earl Waud. Shortly after meeting, and after spilling a cup of coffee on his notebook, we became very close both professionally and as friends.

Earl has always impressed me with how he cares about people, how he values relationships, while still being very professional, methodical, and self-driven to achieve any goal. Additionally, Earl is not short of innovations, driving them all the way from the back of a napkin into highly available multi-data center production deployments that successfully address today's biggest IT challenges. When you combine these traits with his tremendous experience, it becomes a very effective recipe for success both personally and professionally. Mastering Chef Provisioning is the latest example of that success.

In today's IT world, we are experiencing another paradigm shift. In order to support ever-evolving business needs, traditional IT must adopt critical skillsets around infrastructure automation and orchestration. Although there are numerous automation tools available, many professionals have chosen to use Chef because it is one of the most effective platforms to successfully implement provisioning system automation and orchestration.

One of the fundamentally critical and most repeated functions of IT in mission-critical environments is provisioning compute systems. Because of this, provisioning is one of the major targets of automation in IT. In Mastering Chef Provisioning by Earl Waud, the author clearly depicts how you can strategize, design, and implement provisioning automation with Chef across all major hosting platforms anywhere from your own data center all the way out to Amazon Web Services Public Cloud. With the skillset that you will gain from this book, you will be able to implement your own custom provisioning automation to deliver systems rapidly and accurately for all your business needs.

Considering that we are in the middle of another paradigm shift, Mastering Chef Provisioning is one of the most important references that you can have in your library. As a matter of fact, I have already begun using it as a guideline to aid our provisioning automation efforts in delivering thousands of systems in a very short period of time.

I am very confident that you will enjoy this book, and by mastering the techniques within its pages, you will learn how to provision systems effectively with Chef, saving you tremendous amounts of time in your day-to-day operations and reaffirming your position as part of top talent pool in the IT industry.

I would like to conclude by saying thank you to the author of this book, Earl Waud, for leading the way by setting an example for the rest of the world, both at a professional level and a personal level. "We always win!" my friend.

Mert Cubukcuoglu

Senior Manager, Intuit Inc.

About the Author

Earl Waud is a virtualization development professional with more than 10 years of focused industry experience, creating innovative solutions for hypervisor provisioning, management, and automation. He is an expert in aligning engineering strategy with organizational vision and goals, and delivering highly scalable and user friendly virtualization environments.

With more than 20 years of experience developing customer-facing and corporate IT software solutions, he has a proven track record of delivering high-caliber and on-time technology solutions that significantly impact business results.

Earl lives in San Diego, California. He is blessed with a beautiful wife, Patti, and three amazing daughters, Alexis, Daniella, and Madison.

Currently, Earl is a senior software engineer with Intuit Inc., a company that creates business and financial management solutions that simplify the business of life for small businesses, consumers, and accounting professionals.

Earl can be found online at http://sandiegoearl.com.

Acknowledgments

Creating this book was a much larger undertaking than I imagined, and as a result, I owe a debt of gratitude to many.

Thank you to my Heavenly Father. Thank you for my days and for my many blessings. Thank you for this book. Thank you for the opportunity to learn and grow as I created its content. Thank you for allowing me to get it right. And thank you for blessing all who read it so that they can find the solutions they need. I love Thee very much. Thank you.

Thank you to my amazing wife Patti. Thank you, honey, for your patience with me and for being okay with all the time I had to invest to create this book. You carried a lot of the weight of our household, so I could research, write, and revise the information within these pages, and without your support, this book would not exist. Thank you for your confidence in me, which helped me get through some moments of self-doubt. And thank you for loving me. I love you very much. Thank you.

Thank you to my daughter Madison. Thank you, Madison, for your light and for your faith in me. Thank you for seeing me the way that you do. Thank you for being willing to give up a lot of our father-daughter time over the past few months. I know how precious that time is, and I hope we can catch up on some of it before you go off to college. I love you very much. Thank you.

Thank you to my daughter Alexis. Thank you, Lexi, for believing in me. And thank you for the confidence and pride you have for me. Thank you for having faith in my ability to be a writer. I love you very much. Thank you.

Thank you to my daughter Daniella. Thank you, Dani, for your energy and unique perspective. Thank you for your support and your belief in me and my ability to write this book. I love you very much. Thank you.

Thank you to my friend Mert Cubukcuoglu. Thank you, Mert, for your friendship and trust. Thank you for your confidence in me and my ability to write a book about Chef provisioning. And thank you for writing the book's Foreword. Thank you.

Thank you to my friends at work. Thank you Ray Eivaz, Barry Ruffner, Wayne Chatham, Achal Shah, Amy Tam, Angela Bouchard, Mai Lubega, Thy Guintivano, Mike Sharp, and the many others at Intuit that I work with every day. Thank you for your excitement about and interest in my book. Thank you for expressing your belief that I could do it. And thank you for your eagerness to read it when it comes out. Thank you Jay Grissom for being one for the first to preorder your copy. Thank you Michel Cole for being the first to request that I write a second book. Thank you.

Thank you to my friend Tom "Big Al" Schreiter. Thank you, Tom, for suggesting that I start writing books and for being such a great example by taking your own advice and creating a fantastic set of training books. And thank you for your excitement about my book. Thank you.

Thank you to my reviewer Federico Gimenez. Thank you, Federico, for poring over the chapters of this book and making sure I've presented the most accurate content and examples possible. Thank you.

Thank you to my Packt Publishing team. Thank you Mamata Walkar, Rahul Nair, Pramod Kumavat, Sachin Karnani, and Kinjal Bari. Without you, this book would never have happened. Thanks for allowing me to officially become an author and for making sure that my book is a good as it can be. Thank you.

 

"I can no other answer make, but, thanks, and thanks."

  --William Shakespeare

About the Reviewer

Federico Gimenez works currently as a software quality engineer for Canonical in the Snappy team. He has an extensive background supporting agile teams in all the product delivery steps, focusing on applying software engineering solutions to multidisciplinary problems. For the past 7 years, he has also contributed to several open source projects, including Docker, Kubernetes, OpenStack, Ceph, and Debian.

www.PacktPub.com

eBooks, discount offers, and more

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www2.packtpub.com/books/subscription/packtlib

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can search, access, and read Packt's entire library of books.

Why subscribe?

Fully searchable across every book published by PacktCopy and paste, print, and bookmark contentOn demand and accessible via a web browser

Preface

Today, the DevOps Engineer is responsible for delivering infrastructure that is rock solid and consistently configured every time. What's more, these superheroes need to supply the infrastructure rapidly and at scale, often to the tune of dozens or even hundreds of identically configured systems.

The days of doing this superhero work manually are long gone. Now the only question is "which superpower will best serve our heroes in their time of greatest need?" The answer is the power that can transform their infrastructure into code so that it can be managed the same way as any other source code-based project. Chef is that superpower.

And Mastering Chef Provisioning is your secret decoder ring for learning how to use that power to provision containers, servers, and networking devices across the universe of hosting targets, including your local workstation, the traditional data center, and in the clouds.

In this book, you will learn the secrets of how to automate and document every aspect of your infrastructure. You will find the best practices for describing your infrastructure as code. You will learn to automate the provisioning of everything from the smallest container deployed to your local workstation to the biggest distributed application clusters deployed to the cloud, all in a single bound.

This book will help you, the DevOps superhero, to create a perfect model system where everything is represented as code beneath your fingertips. You will be able to make the best possible use of your resources, avoid redundancy, and always be ready to scale—faster than a speeding bullet. And you don't have to leave your home planet, get bitten by a radioactive spider, or fall in a vat of toxic waste to do it.

But as the great Stan Lee wrote, "With great power …".

What this book covers

Chapter 1, Setting Up a Development Environment on Your Workstation, reviews Chef basics and covers the installation of our Chef workstation and an on-premise Chef Server.

Chapter 2, Knife Tooling and Plugins, shows how to get all the help you need with knife, explores using both common and uncommon knife subcommands, and teaches you how to create your own custom knife plugins.

Chapter 3, Leveraging Roles, Environments, and Policies, examines Chef roles, environments, and organizations, covers the attribute precedence hierarchy, and investigates the exciting new Chef policy feature.

Chapter 4, Custom Resources, introduces the custom resource, explores how the custom resource has improved upon LWRPs and HWRPs, and shows how to create and use your own custom resources.

Chapter 5, Provisioning in the Traditional Data Center, presents provisioning in the traditional data center, covering provisioning to VMware vSphere, OpenStack, VMware's desktop hypervisors, and common network devices, and teaches how to deploy your own development OpenStack environment.

Chapter 6, Provisioning in the Cloud, presents provisioning in the cloud, covering provisioning to Amazon AWS, Microsoft Azure, Google Compute Platform, and Linode, and shows how to provision containers in Docker.

Chapter 7, Test-Driven Development, teaches how to use several of the tools available to implement a test-driven development cycle for infrastructure code development, including RuboCop, Foodcritic, ChefSpec, and Test Kitchen.

Chapter 8, Using Chef Provisioning, reveals how to use the ChefDK feature known as Chef provisioning, providing examples for Vagrant, AWS, and Docker.

What you need for this book

The examples in this book were written with the ChefDK version 0.12.0. Examples were tested primarily on Mac OS X and Ubuntu workstations, although all examples should work equally well on other flavors of Linux as well as on Microsoft Windows.

Who this book is for

This book is for software engineers, system administrators, and DevOps Engineers who need to quickly deliver reliably consistent infrastructure at scale. You are expected to have intermediate experience with Chef and Ruby and will be reading this book to advance your knowledge and take your skillset to the next level.

Conventions

In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning.

Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, and user input are shown as follows: "What this error message is probably indicating is that there is no valid host entry in the /etc/hosts file."

A block of code is set as follows:

# add a rule for ssh neutron security-group-rule-create $ID \ --direction ingress --ethertype IPv4 \ --protocol tcp --port-range-min 22 --port-range-max 22 \ --remote-ip-prefix 0.0.0.0/0

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

# add a rule for ssh neutron security-group-rule-create $ID \ --direction ingress --ethertype IPv4 \ --protocol tcp --port-range-min 22 --port-range-max 22 \ --remote-ip-prefix 0.0.0.0/0

Any command-line input or output is written as follows:

$ sudo knife openstack image listName ID Snapshotcirros-0.3.4-x86_64-uec 84dff654-7f0d-45c8-a20b-b08c0ef39fd1 no

New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Click on the Continue button to advance the installation."

Note

Warnings or important notes appear in a box like this.

Tip

Tips and tricks appear like this.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail <[email protected]>, and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.Hover the mouse pointer on the SUPPORT tab at the top.Click on Code Downloads & Errata.Enter the name of the book in the Search box.Select the book for which you're looking to download the code files.Choose from the drop-down menu where you purchased this book from.Click on Code Download.

You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for WindowsZipeg / iZip / UnRarX for Mac7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Chef-Provisioning. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[email protected]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at <[email protected]>, and we will do our best to address the problem.

Chapter 1. Setting Up a Development Environment on Your Workstation

In today's fast-paced IT world, it is a requirement to deliver infrastructure at warp speed. There's really only one way to achieve this requirement, and that is through automation.

One of the best paths to infrastructure automation is via Chef. Using Chef, you can turn your infrastructure into code. With infrastructure as code, you can automate the way you build, deploy, and manage all of it. Using Chef, your infrastructure becomes very consistent, very duplicable, and version-controlled. Using Chef, you can easily test your infrastructure setup and configuration. With Chef, you can become an IT superhero!

In this chapter, we're going to detail the setup and configuration of a complete development system or workstation. The main focus will be centered on deploying the Chef Development Kit (ChefDK) and preparing the various components of a development environment. The chapter also includes suggestions for additional tools to round out the DevOps toolbox. However, before we dive into ChefDK, we will fly through a high-level review of the what-and-how of Chef so that we are all on the same page.

"If you wish to make an apple pie truly from scratch, you must first invent the universe." – Carl Sagan

Here is what you will find in this chapter:

Filling in the gaps in your Chef knowledge baseExploring ChefDKInstalling ChefDKSetting up a standard Chef repoWhat else do you need?Using chef-apply

Re-introducing Chef

If you are reading this book, then you've probably been working with Chef for some time now, and you know the many benefits it brings. You may also know that, in the past, it was a somewhat daunting task to set up a new Chef workstation. You had to download and install Chef, then download and install a variety of community tools, and make sure that all the versions were compatible and configured correctly.

Today, a lot of the work is done for you via the ChefDK. Once you install it, you have a basic workstation setup and are ready to create, modify, and test Chef code.

What version of the Chef tools do I use?

To make it easy to follow along with the contents, let's go over the versions of the tools that will be used throughout this book.

In March 2016, chef.io announced the release of ChefDK 0.12.0, which includes Chef client 12.8.1. As this is the latest version at the time of writing, I will be using it as the version for this book. All of the examples shown will be based on this version of the ChefDK and Chef client. This is very exciting because a lot of really exciting changes are in this release, including Policies and the transition from Resource Providers to Custom Resources.

Currently, there are ChefDK installers available for Mac OS, Windows OS, and Linux OS (RHEL, Debian, and Ubuntu). In the examples within this book, the workstation used will be Mac OS X, so the ChefDK version will be the Mac OS version.

With the many choices available to use for your Chef Server mode, including Hosted Chef Server, Private Chef Server, Open Source Chef Server, and Chef-Solo, it would be difficult to show examples for each mode. Therefore, the majority of examples you'll see in this book will be based on using the Hosted Chef Server mode. Later in this chapter, I will briefly review the installation and setup of a Private Chef Server onto an Ubuntu server, for readers who want to use Chef Server on-premise.

Which OS do I use for my workstation? Everyone has their own, nearly religious, choice for the best workstation platform. However, it would make this book way too long to provide examples for the major OSes alone. Therefore, to keep the focus on Chef content and not on the differences between workstation implementations, I will be using a Mac OS X (Yosemite version 10.10) workstation for the examples. I may at times show additional examples on a Windows or Ubuntu workstation where the differences are significant and worth the extra detail. And for the nodes used in the examples, a variety of OSes will be represented in the hope of having some overlap with the reader's real environment.

References

IT infrastructure automation begins at https://www.chef.io/Additional opportunities to learn Chef can be found at https://learn.chef.io/

Filling in the gaps in your Chef knowledge base

First things first. Let's have a quick, high-level review of Chef. A Chef 101 class, if you will. The plan here is to make sure that all readers are on the same page with me so that the rest of the book will be easier to follow and benefit from. If you are already a Chef ninja, you can probably skip right on past this section and start exploring the ChefDK in the next section. Otherwise, let's audit our 101 class now.

Major pieces of a Chef puzzle

Chef is a ruby framework, and its main purpose is to facilitate the automation, reuse, and documentation of server configurations. Chef allows you to treat server configuration just like any other type of software code.

There are three major components of this framework for almost all corporate-level Chef deployments—the workstation, the Chef Server, and the nodes. Let's talk a little about each of these components:

Workstations: The Chef workstation is, as the name would suggest, where the real "work" is done. This is where developers will create their infrastructure code. It's also where they will test their creations. The workstation is where developers will interact with their source code control systems. It is where genius is born. It is where the "desired state" of nodes is defined via code.Chef servers: The Chef server is a database-backed web API server with a browsable user interface. It is like the matrix housing the hive mind of the Chef universe. The "work" created in the workstation is uploaded into the Chef server so that it can be used to automate the configuration of the many nodes. It stands ready to provide the desired state information to the Chef client on a node so that it can bring that node into alignment or convergence with the desired state of configuration.Nodes: Nodes are the ultimate targets of the "work" that's been created on the workstation and uploaded to the Chef server. A node is where the automation occurs via the Chef client. Nodes are transformed into the desired state configuration. Commonly, nodes are servers, either virtual or physical, but a node can be anything that needs to be configured, such as a Docker container, or a network device such as a switch or router. Every node has a unique name, such as the FQDN of a server.

The Chef client

The Chef client is the tool that is deployed to all nodes and used to configure or reconfigure the node to its desired state. The Chef client is what executes the automation. It is the magician that, through the magic of Chef, transforms an ordinary server into the desired state node that the developer defined back on his workstation.

The Ohai Chef tool

Ohai is the tool that gathers information about a node. Information such as platform details, operating system data, and processor information is made available to the Chef client so that the latter can have the know-how to bring the node to the desired state. Ohai is executed at the beginning of a Chef client run to gather the state of the node. At the end of a Chef client run, all of the data gathered by Ohai (usually) is shared as node data with the Chef server, and the shared data is available for searches done against the Chef server. Ohai can be extended via plugins, and we'll take a look at that in a later chapter.

Recipes and cookbooks

Recipes are the building blocks used to define the desired states. Recipes are files of ruby code that define the commands to be run on nodes. They are like blueprints used to "build" a node. Recipes are collections of Chef resources. We will explain more about Chef resources shortly.

Cookbooks are collections of related recipes, templates, files, and custom resources. They provide organization and versioning for recipes. Each unique version of a cookbook represents unique sets of functionality, such as bug fixes or added features.

Cookbooks define a scenario, such as everything needed to install and configure apt-docker or Sublime text, and they contain all the elements needed to support the defined scenario.

Recipes and cookbooks provide modularity and let you easily reuse code.

Chef run lists

A "run list" is, as the name would suggest, a list of, and the sequence for, the recipes, cookbooks, and policies (spoiler alert!) to be applied to a node. A run list contains all of the information required to configure a node to a desired state. That is, a Chef run list describes the desired final state of the node. It is important to note that, if the node's state already matches what the resources in the run list describe, then no action will be taken to change the node's state.

Chef roles

Roles are functional groupings of recipes and cookbooks used to describe the full blueprint needed for a node to become everything it is intended to be. Roles are reusable configurations, and they can be applied to multiple nodes to make functionally identical servers, such as a farm of web servers.

Chef resources

Chef resources are statements of configuration policy. They are defined in recipes and take actions through the Chef client to put the node into the desired state. Chef resources have some types: Package, Template, Service, and so on. They have a name and parameters. Also, Chef resources can send notifications to other resources.

Chef resources define what we want to happen to a node. They don't say how to do it. The how to do it is left to the providers which are platform-specific. That is to say, the way you install a package will be different depending on the OS, and the provider determines the correct way to do it—the "how". The Chef resource simply defines the "what," such as "install ntp".

Package: This contains software or applications, such as apache, ntp, and cron, and the action to be performed on that software or application, such as "install".Template: These are files with place-holders for attributes that are transformed into configuration files for package installation and execution.Service: This is the installed executable of the package and the actions that the executable can perform, such as start, stop, or restart. Service also defines whether the software or application is launched at node startup.

More things to consider

Apart from the three types of Chef resources we saw earlier, let's look at some other important aspects of recipes and cookbooks.

Attributes: These are variables used in recipes and templates. Generally speaking, a recipe will represent the pattern used in configuration. The attributes provide the specific details for the recipes. For example, the recipe will say "configure the port" and the attribute will say "which port?". Attributes can be provided in a large variety of places, such as cookbooks, roles, and environments. As such, there is a necessity for an order of precedence. There is a complete description of attribute precedence on the chef.io site. You can find it at https://docs.chef.io/attributes.html.Order matters: When creating recipes, the resources need to be listed in order. First, the package, then the template, and finally the service; when creating run lists, the order of policies, roles, cookbooks, and recipes is the order in which they are evaluated. Any recipe that appears in a run list (or as a dependency of a policy, role, or cookbook) more than once will only be evaluated the first time.Convergence and idempotence: A Chef client run converges the node into the desired state. What this means is that only things that do not equal the desired state on the node are modified during the Chef client run. If, for example, the recipe says that the desired state of the node is to have the ntp demon running, the ntp package is already installed on the node, and the ntp demon is already running, then Chef client will take no action to install or start ntp.

Chef resources are idempotent. That is, applying them more than once results in the same outcome every time. If no inputs related to the resource have changed, then applying that resource won't change anything after the first application. In fact, if none of the inputs have changed, the corresponding commands don't even get run.

This is actually one of the most important concepts and features of Chef.

Data bags

Data bags are containers for information that is not tied to a specific node. Data bags are the global variables of a Chef server. They can be used in recipes and can be searched like node data via the Chef server. One common use case for data bags is for user and group information. Items in a data bag can be encrypted. This allows secret information to be stored in them, for example, passwords.

Environments

Environments allow you to define specific cookbook versions that are applied to a given set of nodes. They permit you to model the stages of your infrastructure workflow, that is, Development, Test, Stage, and Production. By identifying specific nodes as Development, and other nodes as Production, for example, you can apply different versions of your cookbooks to the nodes based on their environment membership.

The Chef supermarket

The supermarket is a site that provides shared cookbooks. There is a public supermarket that contains community-created and -maintained cookbooks. This site is hosted by Chef and is available at https://supermarket.chef.io. In addition to the public supermarket, anyone can create and manage a private supermarket that can host cookbooks intended for the private consumption of you and your organization.

Tip

It is always best practice to do a thorough code review of any cookbooks obtained from the public community supermarket before using them in your production environments.

Chef Development Kit

The ChefDK contains everything you need to start working with Chef on a workstation. It provides all the tools that a developer needs to create and modify cookbooks and upload them to a Chef server. We are going to go into a lot of detail regarding the ChefDK later in this chapter.

You can jump ahead to the ChefDK sections now, or you can read on and learn how to set up your very own private Chef server.

Setting up an on-premise (private) Chef Server

Many companies will have strong reasons to keep their infrastructure configuration data within the firewalls of their datacenters, and Chef server has a mode for that. It's called on-premise or private Chef server.

There are some real benefits to using an in-house solution, including control. With a private installation, you have full control of your system, and for some that reason is enough. But there are additional considerations such as performance. Since a private Chef server will likely be physically closer to the node's network, Chef client runs will be faster. For example, if Chef is used in a load-based on-demand server deployment solution, then the extra speed the private Chef server can provide when converging your new Nodes can make all the difference in meeting the load demand in time.

With all the reasons to use an on-premise Chef server, there is a downside: you are responsible for the deployment, configuration, and maintenance of your Chef servers.

To get you started, let's take a look at the