E-Book
36,59 €

Bash Cookbook E-Book

Ron Brash

0,0

36,59 €

Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.

Mehr erfahren.

Herausgeber: Packt Publishing
Kategorie: Wissenschaft und neue Technologien
Sprache: Englisch

Beschreibung

Create simple to advanced shell scripts and enhance your system functionality with effective recipes

Key Features

Automate tedious and repetitive tasks

Create several novel applications ranging from a simple IRC logger to a Web Scraper

Manage your system efficiently by becoming a seasoned Bash user

Book Description

In Linux, one of the most commonly used and most powerful tools is the Bash shell. With its collection of engaging recipes, Bash Cookbook takes you through a series of exercises designed to teach you how to effectively use the Bash shell in order to create and execute your own scripts.

The book starts by introducing you to the basics of using the Bash shell, also teaching you the fundamentals of generating any input from a command. With the help of a number of exercises, you will get to grips with the automation of daily tasks for sysadmins and power users. Once you have a hands-on understanding of the subject, you will move on to exploring more advanced projects that can solve real-world problems comprehensively on a Linux system. In addition to this, you will discover projects such as creating an application with a menu, beginning scripts on startup, parsing and displaying human-readable information, and executing remote commands with authentication using self-generated Secure Shell (SSH) keys.

By the end of this book, you will have gained significant experience of solving real-world problems, from automating routine tasks to managing your systems and creating your own scripts.

What you will learn

Understand the basics of Bash shell scripting on a Linux system

Gain working knowledge of how redirections and pipes interact

Retrieve and parse input or output of any command

Automate tasks such as data collection and creating and applying a patch

Create a script that acts like a program with different features

Customize your Bash shell and discover neat tricks to extend your programs

Compile and install shell and log commands on your system's console using Syslog

Who this book is for

The Bash Cookbook is for you if you are a power user or system administrator involved in writing Bash scripts in order to automate tasks. This book is also ideal if you are interested in learning how to automate complex daily tasks.

Details

Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:

EPUB

Seitenzahl: 297

Veröffentlichungsjahr: 2018

Bewertungen

0,0

Rezensionen(0 Rezensionen)

Leseprobe

Bash Cookbook

Leverage Bash scripting to automate daily tasks and improve productivity

Ron Brash

Ganesh Naik

BIRMINGHAM - MUMBAI

Bash Cookbook

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Commissioning Editor: Vijin BorichaAcquisition Editor: Rahul NairContent Development Editor: Sharon RajTechnical Editor: Mohit HassijaCopy Editor: Safis EditingProject Coordinator: Drashti PanchalProofreader: Safis EditingIndexer: Tejal Daruwale SoniGraphics: Tom ScariaProduction Coordinator: Shantanu Zagade

First published: July 2018

Production reference: 1300718

Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.

ISBN 978-1-78862-936-2

www.packtpub.com

mapt.io

Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.

Why subscribe?

Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals

Improve your learning with Skill Plans built especially for you

Get a free eBook or video every month

Mapt is fully searchable

Copy and paste, print, and bookmark content

PacktPub.com

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.

Contributors

About the authors

Ron Brash was the CTO and co-founder of a successful technology consultancy company that provides services in a multitude of domains, but primarily in FOSS and Linux. For over 7 years, he has worked on embedded systems, which provide security and network connectivity in industrial control systems and SCADA networks, all running an optimized embedded Linux. He participates regularly at FOSS and community events, providing feedback and mini-seminars where appropriate. He loves to share knowledge.

Ganesh Naik is an author, consultant, and corporate trainer for embedded Android, embedded Linux, IoT, and ML-related product development. He has over 20 years of experience and project accomplishment in IT. He has been a corporate trainer for Indian Space Research Organization, Intel, GE, Samsung, Motorola, Penang Skills Development Center, and various firms in Singapore and India. He has started a company called Levana Technologies, which works with organizations for consulting and training activities.

About the reviewer

Fabio Alessandro Locati – commonly known as Fale – is a director at Otelia, public speaker, author, and open source contributor. His main areas of expertise are Linux, automation, security, and cloud technologies. Fale has more than 12 years of working experience in IT, with many of them spent consulting for many companies including dozens of Fortune 500 companies. This has allowed him to consider technologies from different points of view, and to develop critical thinking about them.

Packt is searching for authors like you

If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.

Title Page

Bash Cookbook

Packt Upsell

Why subscribe?

PacktPub.com

Contributors

About the authors

About the reviewer

Packt is searching for authors like you

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Conventions used

Sections

Getting ready

How to do it...

How it works...

There's more...

Preface

In this book, we are writing a variety of scripts using Bash, or the Bourne Again Shell. They range from simple to more complex and handy utilities or programs. Currently, Bash is the default shell used by most GNU/Linux distributions and is used ubiquitously within the Linux Terminal. It can be used for any number of tasks and is flexible across the Linux/Unix ecosystem. In other words, a user familiar with Bash and the Linux CLI can install it themselves on almost any other Linux system and perform similar tasks with a negligible amount of alterations (if any are required at all). Bash scripts can also still work with few dependencies on other software installed, and on a very lean system (minimal installation), a user can still write a powerful script to automate tasks or assist with repeated task execution.

This cookbook focuses entirely on Bash usage in an Ubuntu environment, a very common Linux distribution, but it should be portable to other distributions with relative ease. This book is not written for direct usage on Apple or Windows OSes even though it is possible to port elements over to them.

Who this book is for

Bash Cookbook is for power users or system administrators who are involved in writing Bash scripts in order to automate tasks or are aiming to enhance their productivity on the command line. For example, instead of remembering a series of commands to perform a specific operation, all of those commands can be put into a single script dedicated to that task, and they can perform input validation and format output. Why not save time and reduce errors through a powerful tool?

This book is also ideal if you are interested in learning how to automate complex daily system tasks that can be executed by various system infrastructure such as starting a script on system boot, or through scheduled cron jobs.

What this book covers

Chapter 1, Crash Course in Bash, covers the Linux shell/Bash to get you up and running, and the remainder of the book will just fall into place.

Chapter 2, Acting Like a Typewriter and File Explorer, introduces several bolt-on technologies to make Bash even more extensive when searching for items and text, or automating file explorer/filesystem operations.

Chapter 3, Understanding and Gaining File System Mastery, will help you view files from various angles: head, tail, less, searching for files by name and/or extension, creating a diff of two files, patching, creating symbolic links and using them effectively, crawling filesystem directories, printing a tree, and more.

Chapter 4, Making a Script Behave Like a Daemon, is about creating components that mimic application functionality, such as menus or a daemon.

Chapter 5, Scripts for System Administration Tasks, introduces logs, archiving them, job/task management, network connectivity, securing systems using a firewall (iptables), monitoring directories for changes, and creating users.

Chapter 6, Scripts for Power Users, is about creating syslog entries using the logger command, taking backups, creating graphics and presentations on the CLI, checking file integrity and tampering, mounting network filesystems, retrieving files, browsing the Web, capturing network traffic, finding binary dependencies, and encrypting and decrypting a file.

Chapter 7, Writing Bash to Win and Profit, will help you learn how to use commands and scripts for many tasks. You will get an idea about writing bash scripts for monitoring certain tasks.

Chapter 8, Advanced Scripting Techniques, will help you learn about advanced scripting techniques as well as how to customize their shell.

To get the most out of this book

As the authors, we wrote this book to be accessible and teach you, the reader many different approaches to programming with Bash through several recipes. However, to get the most out of them, we encourage you to:

Have a Linux system setup and configured (ideally Ubuntu) to complete the recipes

Work through the recipes

Keep in mind what the components of each recipe and even the recipes themselves to see how they could be reused or combined in new ways

However, this book assumes a certain level of knowledge to begin your journey and these skills will not be covered in this book. These skills include the following:

How to setup and configure a Linux system

How to install, access, and configure a specific text editor (although several are already included in most Linux distributions)

Some basics about computing and programming (although we will do our best to provide a crash course)

To gain proficiency in those areas, we recommend that you either give it your best shot if you are an adept problem solver and quick learner, or checkout the following resources first:

Linux or distribution communities

Open source forums or groups

YouTube or similar media

Download the example code files

You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.

You can download the code files by following these steps:

www.packtpub.com

Select the

SUPPORT

tab.

Click on

Code Downloads & Errata

Enter the name of the book in the

box and follow the onscreen instructions.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR/7-Zip for Windows

Zipeg/iZip/UnRarX for Mac

7-Zip/PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Bash-Cookbook. In case, there's an update to the code, it will be updated on the existing GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "The full path is more concrete and hardcoded; the interpreter will try to use the complete path. For example, /bin/ls or /usr/local/bin/myBinary.."

A block of code is set as follows:

#!/bin/bashAGE=17if [ ${AGE} -lt 18 ]; then echo "You must be 18 or older to see this movie"fi

Any command-line input or output is written as follows:

rbrash@moon:~$ history

1002 ls

1003 cd ../

1004 pwd

1005 whoami

1006 history

Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this.

Warnings or important notes appear like this.

Tips and tricks appear like this.

Sections

In this book, you will find several headings that appear frequently (Getting ready, How to do it..., How it works..., There's more..., and See also).

To give clear instructions on how to complete a recipe, use these sections as follows:

Getting ready

This section tells you what to expect in the recipe and describes how to set up any software or any preliminary settings required for the recipe.

How to do it...

This section contains the steps required to follow the recipe.

How it works...

This section usually consists of a detailed explanation of what happened in the previous section.

There's more...

This section consists of additional information about the recipe in order to make you more knowledgeable about the recipe.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Reviews

Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!

For more information about Packt, please visit packtpub.com.

Crash Course in Bash

The primary purpose of this chapter is to give you enough knowledge about the Linux shell/Bash to get you up and running, as that the remainder of the book will just fall into place.

In this chapter, we will cover the following topics:

Getting started with Bash and CLI fundamentals

Creating and using basic variables

Hidden Bash variables and reserved words

Conditional logic using if, else, and elseif

Case/switch statements and loop constructs

Using functions and parameters

Including source files

Parsing program input parameters

Standard in, standard out, and standard error

Linking commands using pipes

Finding more information about the commands used within Bash

This chapter will set you up with the basic knowledge needed to complete the recipes in the remaining chapters of the book.

Getting started with Bash and CLI fundamentals

First, we need to open a Linux terminal or shell. Depending on your flavor (distribution) of Linux, this will be done in one of several ways, but in Ubuntu, the easiest way is to navigate to the Applications menu and find one labeled terminal. The terminal or shell is the place where commands are entered by a user and executed in the same shell. Simply put, results (if any) are displayed, and the terminal will remain open, waiting for new commands to be entered. Once a shell has been opened, a prompt will appear, looking similar to the following:

rbrash@moon:~$

The prompt will be in the format of your username@YourComputersHostName followed by a delimiter. Throughout this cookbook, you will see commands with the user rbrash; this is short for the author's name (Ron Brash) and in you case, it will match your username.

It may also look similar to:

root@hostname #

The $ refers to a regular user and the # refers to root. In the Linux and Unix worlds, root refers to the root user, which is similar to the Windows Administrator user. It can be used to perform any manner of tasks, so caution should be used when using a user with root privileges. For example, the root user can access all files on the OS, and can also be used to delete any or all critical files used by the OS, which could render the system unusable or broken.

When a terminal or shell is run, the Bash shell is executed with a set of parameters and commands specific to the user's bash profile. This profile is often called the .bashrc and can be used to contain command aliases, shortcuts, environment variables, and other user enhancements, such as prompt colors. It is located at ~/.bashrc or ~/.bash_profile.

~ or ~/ is a shortcut for your user’s home directory. It is synonymous with /home/yourUserName/ , and for root, it is /root.

Your user's Bash shell also contains a history of all of the commands run by the user (located in ~/.bash_history), which can be accessed using the history command, shown as follows:

rbrash@moon:~$ history

1002 ls

1003 cd ../

1004 pwd

1005 whoami

1006 history

For example, your first command might be to use ls to determine the contents of the directory. The command cd is used to change the directory, to one directory in above the parent directory. The pwd command is used to return the complete path to the working directory (for example, where the terminal is currently navigated to).

Another command you may execute on the shell might be the whoami command, which will return the user currently logged in to the shell:

rbrash@moon:/$ whoami

rbrash

rbrash@moon:/$

Using the concept of entering commands, we can put those (or any) commands into a shell script. In its most simplistic representation, a shell script looks like the following:

#!/bin/bash

# Pound or hash sign signifies a comment (a line that is not executed)

whoami #Command returning the current username

pwd #Command returning the current working directory on the filesystem

ls # Command returning the results (file listing) of the current working directory

echo “Echo one 1”; echo “Echo two 2” # Notice the semicolon used to delimit multiple commands in the same execution.

The first line contains the path to the interpreter and tells the shell which interpreter to use when interpreting this script. The first line will always contain the shebang (#!) and the prefix to the path appended without a space:

#!/bin/bash

A script cannot execute by itself; it needs to be executed by a user or to be called by another program, the system, or another script. The execution of a script also requires it to have executable permissions, which can be granted by a user so that it can become executable; this can be done with the chmod command.

To add or grant basic executable permissions, use the following command:

$ chmod a+x script.sh

To execute the script, one of the following methods can be used:

$ bash script.sh # if the user is currently in the same directory as the script

$ bash /path/to/script.sh # Full path

If the correct permissions are applied, and the shebang and Bash interpreter path is correct, you may alternatively use the following two commands to execute script.sh:

$ ./script.sh # if the user is currently in the same directory as the script

$ /path/to/script.sh # Full path

From the preceding command snippets, you might notice a few things regarding paths. The path to a script, file, or executable can be referred to using a relative address and a full path. Relative addressing effectively tells the interpreter to execute whatever may exist in the current directory or using the user's global shell $PATH variables. For example, the system knows that binaries or executable binaries are stored in /usr/bin, /bin/ and /sbin and will look there first. The full path is more concrete and hardcoded; the interpreter will try to use the complete path. For example, /bin/ls or /usr/local/bin/myBinary.

When you are looking to run a binary in the directory you are currently working in, you can use either ./script.sh, bash script.sh, or even the full path. Obviously, there are advantages and disadvantages to each approach.

Hardcoded or full paths can be useful when you know exactly where a binary may reside on a specific system and you cannot rely on $PATH variables for potential security or system configuration reasons. Relative paths are useful when flexibility is required. For example, program ABC could be in location /usr/bin or in /bin, but it could be called simply with ABC instead of /pathTo/ABC.

So far, we have covered what a basic Bash script looks like, and briefly introduced a few very basic, but essential commands and paths. However, to create a script—you need an editor! In Ubuntu, usually by default, you have a few editors available to you for the creation of a Bash script: vi/vim, nano, and gedit. There are a number of other text editors or integrated development editors (IDEs) available, but this is a personal choice and up to the reader to find one they like. All of the examples and recipes in this book can be followed regardless of the text editor chosen.

Without using a full-blown editor such as the popular Eclipse, Emacs or Geany may also be useful as flexible IDEs within resource-constrained environments, for example, a Raspberry Pi.Knowledge of vi/vim and nano is very handy when you want to create or modify a script remotely over SSH and on the console. Vi/vim may seem a bit archaic, but it saves the day when your favorite editor is not installed or cannot be accessed.

Your first Bash script with Vim

Let's start by creating a script using improved version of vi (called vim). If Vim (VI-enhanced) is not installed, it can be installed with sudo or root using the following command (-y is short for yes):

For Ubuntu or Debian based distributions

sudo apt-get -y install vim

For CentOS or RHEL

$ s

udo yum install -y vimFor Fedora

sudo dnf install -y vim

Open a terminal and enter the following commands to first see where your terminal is currently navigated to, and to create the script using vim:

$ pwd

/home/yourUserName

$ vim my_first_script.sh

The terminal window will transform into the Vim application (similar to the following screenshot) and you will be just about ready to program your first script. Simultaneously press the Esc+ I keys to enter Insert mode; there will be an indicator in the bottom left and the cursor block will begin to flash:

To navigate Vim, you may use any number of keyboard shortcuts, but the arrow keys are the simplest to move the cursor up, down, left, and right. Move the cursor to the beginning of the first line and type the following:

#!/bin/bash

# Echo this is my first comment

echo "Hello world! This is my first Bash script!"

echo -n "I am executing the script with user: "

whoami

echo -n "I am currently running in the directory: "

pwd

exit 0

We have already introduced the concept of a comment and a few basic commands, but we have yet to introduce the flexible echo command. The echo command can be used to print text to the console or into files, and the -n flag prints text without the end line character (end line has the same effect as pressing Enter on the keyboard)—this allows the output from the whoami and pwd commands to appear on the same line.

The program also exits with a status of 0, which means that it exited with a normal status. This will be covered later as we move toward searching or checking command exit statuses for errors and other conditions.

When you've finished, press Esc to exit insert mode; going back to command mode and typing : will allow you to write the vim command w + q. In summary, type the following key sequence: Esc and then :wq. This will exit Vim by writing to disk (w) and quitting (q), and will return you to the console.

More information about Vim can be obtained by reviewing its documentation using the Linux manual pages or by referring to a sibling book available from Packt (https://www.packtpub.com/application-development/hacking-vim-72).

To execute your first script, enter the bash my_first_script.sh command and the console will return a similar output:

$ bash my_first_script.sh

Hello world! This is my first Bash script!

I am executing the script with user: rbrash

I am currently running in the directory: /home/rbrash

Congratulations—you have created and executed your first Bash script. With these skills, you can begin creating more complex scripts to automate and simplify just about any daily CLI routines.

Creating and using basic variables

The best way to think of variables is as placeholders for values. They can be permanent (static) or transient (dynamic), and they will have a concept called scope (more on this later). To get ready to use variables, we need to think about the script you just wrote: my_first_script.sh. In the script, we could have easily used variables to contain values that are static (there every time) or dynamic ones created by running commands every time the script is run. For example, if we would like to use a value such as the value of PI (3.14