40,81 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Leverage Puppet 5 for medium to large scale enterprise deployment.
Key Features
- Use and deploy Puppet 5, irrespective the size of your organization
- Scaling, performance improvements, and managing multiple developer requests
- Troubleshooting techniques, tips and tricks to make the most of Puppet 5
Book Description
Puppet is a configuration management system and a language written for and by system administrators to manage a large number of systems efficiently and prevent configuration drift.
The core topics this book addresses are Puppet's latest features and mastering Puppet Enterprise. You will begin by writing a new Puppet module, gaining an understanding of the guidelines and style of the Puppet community. Following on from this, you will take advantage of the roles and profiles pattern, and you will learn how to structure your code. Next, you will learn how to extend Puppet and write custom facts, functions, types, and providers in Ruby, and also use the new features of Hiera 5. You will also learn how to configure the new Code Manager component, and how to ensure code is automatically deployed to (multiple) Puppet servers. Next, you will learn how to integrate Puppet with Jenkins and Git to build an effective workflow for multiple teams, and use the new Puppet Tasks feature and the latest Puppet Orchestrator language extensions. Finally, you will learn how to scale and troubleshoot Puppet.
By the end of the book, you will be able to deal with problems of scale and exceptions in your code, automate workflows, and support multiple developers working simultaneously.
What you will learn
- Solve problems using modules and the roles and profiles pattern
- Extend Puppet with custom facts, functions, types, and providers
- Use Hiera 5 and Code Manager/r10k to separate code from data
- Continuously integrate your code using Jenkins, Git and automated testing
- Use exported resources and the new Puppet Orchestration features
- Explore Puppet Discovery features and their use
- Troubleshoot various parts of the Puppet Enterprise infrastructure
- Scale up and scale out Puppet infrastructure using various techniques
Who this book is for
If you are a system administrator or developer who has used Puppet in production and are looking for ways to easily use Puppet in an enterprise environment, this book is for you. Some knowledge of writing simple configuration management modules would be necessary.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 329
Veröffentlichungsjahr: 2018
Ähnliche
Mastering Puppet 5
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Vijin BorichaAcquisition Editor: Meeta RajaniContent Development Editor: Nithin George VargheseTechnical Editor: Komal KarneCopy Editor: Safis EditingProject Coordinator: Drashti PanchalProofreader: Safis EditingIndexer: Tejal Daruwale SoniGraphics: Tom ScariaProduction Coordinator: Shantanu Zagade
First published: September 2018
Production reference: 1280918
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78883-186-4
www.packtpub.com
mapt.io
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.comand as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Ryan Russell-Yates is a technical consultant in the fields of automation, DevOps, and infrastructure architecture. He has helped numerous IT practitioners at companies of various shapes and sizes across a range of industries to implement automation best practices at scale. Ryan's true passion in the technology industry is teaching practitioners new tools, technologies, and strategies for dealing with today's complicated digital landscape.
Jason Southgate has been working in the IT industry for more than 15 years, has been using Puppet for more than 6 years, and has tackled some very large projects in Europe, most recently creating an IaaS/PaaS cloud for KPN, the Netherlands' premier telecommunications company, using Puppet Enterprise at a very large scale. Jason was certified in Puppet in 2014, and also has AWS and Azure certification.
About the reviewer
Deniz Parlak has worked with Linux/UNIX technologies for more than five years. He is a [Dev-Sys-Sec]Ops enthusiast. Deniz is currently working with [Dev-Sys-Sec]Ops applications, especially Docker, Ansible, Kubernetes, and cloud providers, and continues to share information about new technologies with people at many technical conferences and lectures. He has also published a book entitled CentOS System and Server Management, and is still writing Docker and Bash scripting books. His presentations on the Zeus tool and AWS hardening were selected for events such as Black Hat Asia 2018, DevOpsDays, and NuitDuHack.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Mastering Puppet 5
Dedication
Packt Upsell
Why subscribe?
packt.com
Contributors
About the authors
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Authoring Modules
Using a decent IDE and plugins
Vim
TextMate
Atom
Visual Studio
Using good module and class structure
Following the class-naming conventions
Having a single point of entry to the module
Using high cohesion and loose coupling principles
Using the encapsulation principle
Providing sensible, well-thought-out parameter defaults
Strongly typing your module variables
Using the new Puppet Development Kit commands
Validating your module
Unit testing your module
Staying on the lookout for code smells
Working with dead code
Using Puppet Forge
Working with the community
Writing great documentation
Grabbing yourself a Markdown editor
Vim
TextMate
Atom
Visual Studio
Standalone Markdown editors
Remarkable
MacDown
Adding module dependencies
Adding compatibility data for your modules
Operating systems support
Puppet and PE version support
Using the new Hiera 5 module level data
Summary
Roles and Profiles
Summary of the pattern
Profiles
Profiles best practices summary
Designing for use of the include keyword
Using subdirectories for sensible, readable profile class groups
Hiding complexity with parameters, defaults, and abstraction
Deciding how to set the parameters for component classes
Deciding to use either automatic class parameter lookup or the lookup function
Roles
Roles best practices summary
Constructing roles only with the include keyword
Naming roles in your business's conversational name
Deciding on the granularity of roles for your nodes
Summary
Extending Puppet
Custom facts
Debugging facts
Custom functions
Types and providers
Types
Creating and distributing the type
Adding the namevar special attribute
Adding additional type properties
Adding the optional ensure property
Adding type parameters
Setting property and parameter defaults
Checking the input value with a validate block
Checking the input value against a newvalues array
Checking datatype compatibility with munge
Using autorequire for implicit relationships
Using arrays to assign a list of values to an attribute
Using the desc method to add inline documentation
Providers
Creating and distributing the provider
Indicating the suitability of the provider to the type
Using the confine method
Using the defaultfor method
Using the commands method
Implementing the ensure property
Using the exists? method
Using the create and destroy methods
Using the GET and SET methods to manage type properties
Implementing the self.instances method
Summary
Hiera 5
Separation of concerns between code and data
Introducing a frame for the environment
A more complete hierarchy
Hiera 5 summary
Global, environment, and module layers
Encrypted YAML backend
Installing hiera-eyaml
Creating the encryption keys
Securely storing away the encryption keys
Changing hiera.yaml
Lookup function
The lookup function syntax
Lookup function arguments
Lookup function examples
Lookup strategies
Deep merge lookup settings explained
knockout_prefix setting
sort_merge_arrays setting
merge_hash_arrays setting
unpack_arrays setting
Debugging Hiera
Old debugging techniques
Equivalent debugging technique
Beyond Hiera using Jerakia
Jerakia advanced use cases
Installing Jerakia
Configuring Jerakia
Creating your default Jerakia policy
Using Vault as an encryption backend
Installing and configuring Vault
Unsealing Vault
Enabling the transit backend
Creating an encryption key
Creating a policy for encrypting and decrypting
Checking the encryption is working correctly
Allowing Jerakia to authenticate with our Vault
Configuring Jerakia for encryption
Encryption-enabling our Jerakia lookups
Summary
Managing Code
Efficiently managing code
Code Manager
Git
r10k
Control repository
production-like environments
non-production-like environments
Puppetfile
hiera.yaml
site.pp
environment.conf
Roles and profiles
Control repository example
Installing and using r10k
Code Manager
Enabling Code Manager
Code Manager RBAC
PE client tools
Multitenant control repository
Summary
Workflow
Puppet workflow
Ease of use
Rapid feedback
Ease of onboarding
Quality control
Designing a Puppet workflow
Components of the Puppet workflow
Repositories
Control repository
Module repository
Tasks
Clone and edit the component repositories
Cloning the control repository
Editing the control repository
Deploying the new environment on the Puppet Master
Testing the changes
Merging branches
Git tags and versioning
Using the PDK
PDK
Creating new Puppet artifacts
The pdk new command
The pdk validate command
The pdk test unit command
Summary
Continuous Integration
Continuous Integration systems
Puppet Pipelines
Jenkins
Managing Jenkins with Puppet
rtyler/jenkins
Managing our plugins
Creating our first build
Building our profile module
Building our Jenkinsfile
Connecting Jenkins to our repository
Integrating the PDK
Unit testing with Puppet RSpec
Relevant RSpec files
.fixtures.yml
jenkins_spec.rb
Extending our Jenkinsfile
Extending our test
Acceptance testing with Test Kitchen
Beaker
Test Kitchen and kitchen-puppet
Preparing Test Kitchen on our Jenkins node
Jenkins Profile
.kitchen.yml
Puppetfile
Jenkinsfile
acceptance.sh
Test
Performing the test
Summary
Extending Puppet with Tasks and Discovery
Puppet Tasks
Bolt
Installing Bolt
Managing nodes
Ad hoc commands
Bolt tasks
task.json
Task
Bolt plans
Puppet Enterprise Task Management
Puppet Discovery
Installing Discovery
Preparing Puppet Discovery
Managing sources
Adding sources by IP address
Managing credentials
SSH key file
Discovering
Viewing the Discovery
Discovering hosts
Discovering packages
Acting
Installing agents
Managing services
Uses for Discovery
Summary
Exported Resources
Virtual and exported resources
Virtual resources
Tags
Exported resources
Use cases
Hosts file
Load balancing
Database connections
Concat, file lines, and you!
Concat – the hammer
file_line – the scalpel
Summary
Application Orchestration
Application definition
Application components
Service resources
Modeling applications
Application and database
Dependencies
Build
Node declaration
Application declaration
DB service resource
Application components
Deploy
Adding a load balancer and providing horizontal scaling
Dependencies
Build
Deploy
Summary
Scaling Puppet
Inspection
Puppetserver
PuppetDB dashboard
Tuning
Puppetserver tuning
Puppet Enterprise implementation
Open source implementation
PuppetDB tuning
Deactivating and purging nodes
Managing the heap size
Tuning CPU threads
Automatically determining settings
Puppet Enterprise
PuppetDB – PostgreSQL with PGTune
Horizontal scaling
Puppetserver
Estimating the number of agents a Puppetserver supports
Adding new compile masters
Load balancing
Simple setup – direct connection
Load balancing
Certificate authority
PuppetDB
Summary
Troubleshooting and Profiling
Common component errors
Puppet agents and Puppetserver
Waiting on certificate signing
Certificate reuse
Wrong Puppet user
Network connectivity
DNS alt name
Date and time
PE console service is down
Catalog errors
Syntax errors
Duplicate resource declaration
Missing resources
Autoload format
Circular dependencies
Debug mode – catalog
Logging
The logback.xml file
Main configuration
Appender
Loggers
Root logger
Puppet agent
PuppetDB
Puppetserver
Puppet Enterprise console
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Puppet 5 remains the software configuration management software of choice, especially for larger-scale configurations.
Here are some examples of current real-world use cases of Puppet:
Twitter uses Puppet for what is currently one of the larger social networking infrastructures (
https://blog.twitter.com/engineering/en_us/topics/infrastructure/2017/the-infrastructure-behind-twitter-scale.html
). Facebook uses Opscode Chef, the competitive product in the software configuration management category.
Uber uses Puppet for its standard configuration management (
https://eng.uber.com/uchat/
).
Walmart is also a very large Puppet user (
https://puppet.com/blog/how-walmart-scaled-puppet-55K-nodes-and-beyond
).
Although there are newer products, such as Ansible and Salt, Puppet remains – I believe – the premier tool, especially for such larger infrastructures (10,000+ servers). It is worth mentioning that Ansible has also become very popular, possibly due to its shallower learning curve and adoption by Red Hat.
Dealing with this level of scale and complexity is non-trivial. With this book, Mastering Puppet 5, we want to put the know-how at your disposal to tackle your own large-scale challenges, at mastery level.
Version 5 of Puppet, which is the version covered by this book, was announced with considerable fanfare at last year's PuppetConf (2017) by the new CEO, Sanjay Mirchandani, as "Puppet's largest set of product innovations. Ever." In this book, we've gone through these new technologies, including Puppet Discovery, Puppet Tasks, and Puppet Pipelines, to give you the know-how you need to use Puppet 5 in the real world with confidence.
Who this book is for
If you are a system administrator or developer who has already used Puppet and are looking for mastery-level skills and best practices with a view to using Puppet in an enterprise environment, at large scale, this book is for you. Some beginner-level knowledge of using Puppet would be necessary.
If you are looking for a gentle introduction to Puppet, before starting to use this book, please take a look at Puppet's own free self-paced training courses (https://learn.puppet.com/category/self-paced-training) or attend some instructor-led or private training (https://puppet.com/support-services/training).
What this book covers
Chapter 1, Authoring Modules, will really get you on the right path to higher quality Puppet modules and manifests, introducing 12 best practices for module writing.
Chapter 2, Roles and Profiles, introduces two additional layers of abstraction and improved interfaces for making your hierarchical business data easier to integrate, making system configurations easier to read, and making refactoring easier.
Chapter 3, Extending Puppet, covers three parts of the ecosystem that can still be accessed at the Ruby level for the purposes of extending Puppet to suit more advanced use cases; namely, custom facts, custom functions, and types and providers.
Chapter 4, Hiera 5, covers the latest incarnation of Hiera, which allows us to keep all site-specific and business-specific data out of our manifests, making our Puppet modules vastly more portable. We also take a quick look in this chapter at the three layers of configuration and data: global, environment, and module. We also cover how to set up an encrypted YAML backend. Lastly, we take a cursory look at using Jerakia to extend Heira.
Chapter 5, Managing Code, covers the use of r10k and Code Manager, allowing us to store all Puppet code in Git repositories and providing version control and rollback capabilities. We discuss directory environments, which give us multiple versions of code on a single master, and how they're supported by r10k and Code Manager. We build a Puppetfile and actively deploy our code to our Puppet Master.
Chapter 6, Workflow, covers a basic Puppet workflow. We'll be incorporating the PDK into our basic workflow more heavily, allowing us to write code more efficiently.
Chapter 7, Continuous Integration, covers tying Puppet into Jenkins as a Continuous Integration (CI) system. We'll discuss the components of a CI/Continue Deployment (CI/CD) pipeline, what it takes to achieve some of the milestones to get there, and actively improve our Puppet code in a CI system.
Chapter 8, Extending Puppet with Tasks and Discovery, covers Puppet Tasks and Puppet Discovery. Puppet Tasks allows us to run ad hoc commands and use them as building blocks for imperative scripts. We'll be building a task to inspect log files and planning to build an aggregated log file for our Puppet Master. Puppet Discovery allows us to inspect our existing infrastructure and determine ground truth on packages, services, users, and various other components of a virtual machine or container.
Chapter 9, Exported Resources, covers virtual and exported resources in Puppet. We'll explore exporting and collecting resources in our manifests, and the common use cases for exported and collected resources to include the following: a dynamic /etc/hosts file, load balancing, and automatic database connections. We'll also explore the file_line and concat resources to allow us to build dynamic configuration files based on these exported resources.
Chapter 10,Application Orchestration, covers the ordering of multiple node runs. We'll build application orchestration manifests, which allow us to tie nodes together and provide configuration values across multiple nodes, ensuring that our multi-node application runs in the order necessary, with the information it needs.
Chapter 11,Scaling Puppet, covers the horizontal and vertical scaling of Puppet. We'll explore some common settings for tuning, and inspect ways to horizontally scale Puppet services.
Chapter 12,Troubleshooting and Profiling, covers some common troubleshooting cases we see with Puppet. We'll focus on both Puppet service errors and catalog compilation errors, and inspect tuning and configuring our log files.
To get the most out of this book
Users with some prior Puppet experience will get the most out of this book, but every lesson is intended to be helpful to someone at any stage of learning about Puppet. To follow along in the book, users should install a trial version of Puppet Enterprise and attach some nodes to the Puppet Master. Each chapter will help set up the Puppet Master in a way to utilize the existing infrastructure.
Directions for installing Puppet Enterprise can be found at https://puppet.com/docs/pe/latest/installing_pe.html.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Mastering-Puppet-5. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781788831864_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "The earlier incarnations of Hiera (version 3 or earlier) used a single, entirely global hiera.yaml."
A block of code is set as follows:
lookup({ 'name' => 'classification', 'merge' => { 'strategy' => 'deep', 'knockout_prefix' => '--', }, })
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
lookup({ 'name' => 'classification', 'merge' => {
'strategy' => 'deep',
'knockout_prefix' => '--', }, })
Any command-line input or output is written as follows:
$ sudo /opt/puppetlabs/puppet/bin/gem install hiera-eyaml
Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "We can reach the plugin page by clicking on Manage Jenkins on the left-hand side of the screen."
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Authoring Modules
Authoring Puppet modules and manifests is the real heart of the work for your Puppet ecosystem.
So, you've perhaps already written at least a few modules for software components in your infrastructure, and there's already a great guide to getting started writing modules in the Puppet documentation at https://puppet.com/docs/pe/2017.3/quick_start_guides/writing_modules_nix_getting_started_guide.html, so I won't waste any time going over that material again. But I'm sure that, in pursuit of mastering Puppet v5, what you would really like to do is to write those modules correctly.
Let's take that step together toward better quality modules in this chapter. I've spent a lot of time in the trenches over the last few years, gathering together best practices from some of the best projects across Europe and applying practices and software principles I've learned from both my university education and 15+ years in the industry. I hope I can introduce you to some shortcuts and make your life easier!
The following are a set of recommendations that I feel will really get you on the right path to higher quality Puppet modules and manifests:
Using a decent IDE and plugins
Using a good module class structure:
Following the class-naming conventions
Having a single point of entry to the module
Using high cohesion and loose coupling principles
Using the encapsulation principle
Strongly typing your module variables
Using the new Puppet Development Kit commands:
Creating the module framework and metadata
Creating the
init.pp
Creating further classes
Validating your module
Unit testing your module
Staying on the lookout for code smells
Making sure you are not working with dead code
Working with the community
Using Puppet Forge
Writing great documentation
Adding module dependencies
Adding compatibility data for your modules
Operating systems support
Puppet and PE version support
Using the new Hiera 5 module level data
Upgrading your templates from ERB to ERP syntax
Let's examine each of these best practices now in turn.
Using a decent IDE and plugins
Using a decent text editor with the plugins that equip you to write well for Puppet is a really good step toward better quality. There are quite a few options out there, and it's best to use whatever suits your own unique writing style. Personally, I have used Atom (https://atom.io) most successfully, and recently installed it locally on my workstation. I used Eclipse many years ago (this has also been known previously as Geppetto), which I in fact felt was unwieldy due to a large memory footprint. It's also nice to remain fairly handy with Vim, especially for working on the command line server-side, or if you use a Linux OS on your workstation. There's also TextMate, for an macOS X only editor that has all of Apple's look and feel.
Let's take a look at some of the various options for an Integrated Development Machine (IDE) available to us as Puppet developers.
Vim
Vim (http://www.vim.org) is, of course, still a mainstay for text file editing. It has a very long history in the Unix world, and it's a very lightweight command-line text editor. Vim is just about as raw a text editor as you can get. It can be used as a lightning fast and efficient IDE if you have the memory and patience to learn the myriad keyboard commands. My advice is to start out with a few basic commands, and make an effort to pick up a few more each time you use Vim.
You can pimp your Vim and make it better suited for editing Puppet manifests. Let's take a look at that, assuming you've just grabbed a fresh Vim installation, and you have Git installed.
Move to your home directory and clone the given repository with the following commands:
cd ~
git clone https://github.com/ricciocri/vimrc .vim
cd .vim
git pull && git submodule init && git submodule update && git submodule status
cd ~
ln -s .vim/.vimrc
Cloning the repository into your home directory's .vim directory will configure your Vim settings for you. The repository contains several submodules containing the following:
Pathogen
(
https://github.com/tpope/vim-pathogen
) is Vim guru Tim Pope's general-purpose add-on that allows you to manage your Vim
runtimepath
with ease and install Vim plugins and runtime files each in their own private directories, rather than having file collisions.
Vim-puppet
(
https://github.com/rodjek/vim-puppet
) is the original Vim plugin written by Tim Sharpe, making Vim much more Puppet-friendly.
snipmate.vim
(
https://github.com/msanders/snipmate.vim
) is a Vim script that implements some of TextMate's snippet features for Vim.
Syntastic
(
https://github.com/vim-syntastic/syntastic
) is a syntax-checking plugin that runs files through external syntax checkers and displays any resulting errors. This can be done from the command line with the
pdk validate
command, or automatically as files are saved.
Tabular
(
https://github.com/godlygeek/tabular
) is used to line up your fat arrows (=>) according to the
Puppet Style Guide
, so that it will pass running the
pdk validate
command. (We will cover the
pdk validate
command in full later.)
vim-fugitive
(
https://github.com/tpope/vim-fugitive
) provides deep Git integration for Vim.
I can't promise this will be a perfect Vim setup for your own personal Vim style, but it will certainly get you on the right path, and you will have Pathogen installed, so you can further tweak your Vim settings until you have it just how you like it.
You might also want to fork this repository in GitHub, so you can keep all your settings and share them with your team.
TextMate
TextMate (http://macromates.com) is an macOS X only editor, and there's a TextMate bundle available (https://github.com/masterzen/puppet-textmate-bundle) for editing Puppet manifests. First, install TextMate and Git (available with the command-line developer tools), and follow these commands to set up the Puppet bundle:
$ mkdir ~/temp
$ cd ~/temp
$ git clone https://github.com/masterzen/puppet-textmate-bundle.git Puppet.tmbundle
$ mv ~/temp/Puppet.tmbundle ~/Library/Application\ Support/TextMate/Bundles/
$ rm -fr ~/temp
Now select a manifest and open it with TextMate. In the TextMate dialog, select Puppet and Install Bundle, and you are all ready to rock.
Atom
Here's the IDE that I would recommend based on my own personal style, using my MacBook as the host OS. Atom (https://atom.io) is a fully featured IDE described as, A hackable text editor for the 21st Century and contains all the functionality you'd expect: cross-platform, package (that is, plugin) manager, auto-completion, file browser, multiple panes, find and replace, and so on.
GitHub has developed Atom, and they have built it with the goal of combining the convenience of a fully fledged IDE with the deep configurability of a classic but complex editor such as Vim.
There are literally thousands of open source packages that add new features and functionality to Atom, and here are the ones I recommend specifically for Puppet development:
language-puppet
(adds syntax highlighting and snippets to Puppet files)
linter-puppet-lint
(provides linter support to your Puppet manifests)
aligner-puppet
(aligns the fat arrows according to the Puppet Style Guide)
erb-snippets
(snippets and hotkeys for writing Puppet ERB templates)
linter-js-yaml
(parses your YAML files with JS-YAML)
tree-view-git-status
(displays the Git status of files in the tree view)
Visual Studio
If you're a developer in the Windows and .NET world, then look no further than the Puppet language support for Visual Studio Code extension (https://marketplace.visualstudio.com/items?itemName=jpogran.puppet-vscode).
It contains all the features you would expect for Puppet development in the Visual Studio IDE: syntax highlighting, code snippets, file validation, linting according to the Puppet Style Guide, IntelliSense for resources and parameters, importing from the puppet resource command, node graph previewing, and now, Puppet Development Kit (PDK) integration.
Using good module and class structure
This section contains a set of recommendations surrounding good module and class design. Bear in mind that Puppet development is, in principle, just like any other type of software development, and we've learned over many years in software development, and especially at O&O software, that certain modular and class design principles make our development better. I also feel that part of our journey toward infrastructure as code is making our Puppet code just as well-designed, structured, and tested as any other application code.
Following the class-naming conventions
There's a certain class-naming convention that has developed over time within the Puppet community, and it's really worth taking these into account when structuring your classes:
init.pp
:
init.pp
contains the class named the same as the module, and is the main entry point for the module.
params.pp
: The
params.pp
pattern (more on this later in the chapter) is an elegant little hack, taking advantage of Puppet's class inheritance behavior. Any of the other classes in the module inherit from the
params
class, so have their parameters set appropriately.
install.pp
: The resources related to installing the software should be placed in an
install
class. The install class must be named
<modulename>::install
and must be located in the
install.pp
file.
config.pp
: The resources related to configuring the installed software should be placed in a
config
class. The
config
class must be named
<modulename>::config
and must be located in the
config.pp
file.
service.pp
: The resources related to managing the service for the software should be placed in a
service
class. The service class must be named
<modulename>::service
and must be located in the
service.pp
file.
For software that is configured in a client/server style, see the following:
<modulename>::client::install
and
<modulename>::server::install
would be the class names for the
install.pp
file placed in the
client
and
server
directories accordingly
<modulename>::client::config
and
<modulename>::server::install
would be the class names for the
config.pp
file placed in the
client
and
server
directories accordingly
<modulename>::client::service
and
<modulename>::server::service
would be the class names for the
service.pp
files placed in the
client
and
server
directories accordingly
Having a single point of entry to the module
init.pp should be the single entry point for the module. In this way, someone reviewing the documentation in particular, as well as the code in init.pp, can have a complete overview of the module's behavior.
If you've used encapsulation effectively and used descriptive class names, you can get a very good sense just by looking at init.pp of how the module actually manages the software.
Ideally, you can use your module with a simple include statement, as follows:
include mymodule
You can also use it with the use of a class declaration, as follows:
class {'mymodule': myparam => false,}
The Apache virtual directory style of configuring a number of defined types would be the third way to use your new module:
mymodule::mydefine {‘define1': myotherparam => false,}
The anti-pattern to this recommendation would be to have a number of classes other than init.pp and your defined types with parameters expecting to be set.
Using high cohesion and loose coupling principles
As far as possible, Puppet modules should be made up of classes with a single responsibility. In software engineering, we call this high, functional cohesion. Cohesion in software engineering is the degree to which the elements of a certain module belong together. Try to make each class have a single responsibility, and don't arbitrarily mix together unrelated functionalities in your classes.
Using the encapsulation principle
As far as possible, these classes should use encapsulation to hide the implementation details from the user; for example, users of your module don't need to be aware of individual resource names. In software engineering, we call this encapsulation. For example, in a config class, we can use several resources, but the user doesn't need to know all about them. Rather, they just simply know that they should use the config class for the configuration of the software to work correctly.
Having classes contain other classes can be very useful, especially in larger modules where you want to improve code readability. You can move chunks of functionality into separate files, and then use the contain keyword to refer to these separated chunks of functionality.
Providing sensible, well-thought-out parameter defaults
If the vast majority of the people using your module will use the module with a certain parameter set, then of course it makes sense to set that parameter with a default.
Carefully think through how your module is used, and put yourself in the position of a nonexpert user of your own module.
Present the available module parameters in a sensible order, with more often accessed settings before least accessed settings, as opposed to some arbitrary order, such as alphabetical order.
Strongly typing your module variables
In versions of Puppet proper to the new language features which came out in version 4, we would create class parameters with undefined data types, and then, if we were being very nice, we would use the stdlib validate_<datatype> functions to check appropriate values for those variables:
class vhost ( $servername, $serveraliases, $port){ ...
Puppet 4 and 5 have an in-built way of defining the data type that a parameterized class accepts. See the following example:
class vhost ( String $servername, Array $serveraliases, Integer $port){ ...
Using the new Puppet Development Kit commands
Some features to improve quality in your Puppet development, such as puppet-lint, puppet-rspec, and commands such as puppet module create have been around for some time, but previously, you had to discover these tools out there in the wild, install them, and figure out how to use them effectively yourself.
Puppet decided back in August 2017 to bring these things all together on the client side and make them a breeze to use with the new Puppet Development Kit version 1.0. I can certainly recall puppet-rspec always took some time to set up and get working correctly. Now it's all really easy.
Let's take a whistle-stop tour of the module development process using the new PDK 1.0.
Creating the module framework and metadata
: The
pdk new module
command runs in the same way as the old
puppet module create
command, as follows:
$ pdk new module zope –-skip-interview
Creating the
init.pp
: There is now a set of creation commands for manifests inside modules, as follows:
pdk new class
(
https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-class-command
)
pdk new defined_type
(
https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-definedtype-command
)
pdk new task
(
https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-new-task-command
)—see
Chapter 6
,
Workflow
, for more details on the new Puppet task functionality.
So, just use the name of the module to create init.pp:
$ pdk new class zope
These commands now negate any need for snippets in your text editor to create the comments, declarations, and other boilerplate code.
Creating further classes
: Create any further classes using the same command. See the following example:
$ pdk new class params
Validating your module
As you are working, you can use the new pdk validate command (https://puppet.com/docs/pdk/1.0/pdk_reference.html#pdk-validate-command) to assist with checking that the module compiles, conforms to the Puppet Style Guide, and has valid metadata:
$ pdk validate
Unit testing your module
The number one most important thing you can do to bring quality to your modules is to test them! Testing really is one of the most important aspects of software quality assurance in any field of software development. In the agile development community, we've been banging on the table about automated testing for more than 10 years!
Puppet RSpec (http://rspec-puppet.com/tutorial) has been allowing the Puppet community to unit test their modules for quite some time, but it's even easier now with the new PDK 1.0, as everything is set up ready, and you can just add your testing code and run the tests.
From a Puppet perspective, unit testing means checking the output from the compiler. Are the resources contained in the compiled relationship resource catalog, and is their order as expected, given the parameters passed and/or facts present?
