Bash Shell Scripting for Pentesters E-Book

Steve Campbell

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 531

Veröffentlichungsjahr: 2024

Beschreibung

Bash shell scripting is essential for penetration testing because it’s versatile and efficient and integrates thoroughly with the Unix-based systems commonly used in cybersecurity assessments. In this book, the author leverages his decades of experience in IT and pentesting to help you automate repetitive tasks, rapidly analyze data, and craft sophisticated exploits, boosting your effectiveness and productivity.
You’ll get to grips with Bash basics, set up a hacking environment, and create basic scripts, before exploring file management, text processing, and manipulation. The chapters will guide you through advanced topics such as networking, parallel processing, and regular expressions. From there, the book will move on to practical applications, walking you through reconnaissance, web application and infrastructure pentesting, privilege escalation, persistence, and pivoting, all using Bash scripting. You’ll also get a solid understanding of advanced topics, including evasion and obfuscation tactics, integrating AI into penetration testing workflows, and implementing DevSecOps practices.
By the end of this book, you’ll be well-versed with Bash shell scripting techniques tailored to penetration testing scenarios.

Leseprobe

Bash Shell Scripting for Pentesters

Master the art of command-line exploitation and enhance your penetration testing workflows

Steve Campbell

Bash Shell Scripting for Pentesters

Copyright © 2024 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

The author acknowledges the utilization of an advanced AI assistant, specifically Claude.ai, with the sole objective of suggesting improvements to the author’s typically concise writing style and providing assistance with diagnosing code errors. It’s important to note that the content itself has been crafted by the author and edited by a professional publishing team.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Dhruv Jagdish Kataria

Publishing Product Manager: Prachi Sawant

Book Project Manager: Ashwin Kharwa

Senior Editor: Mohd Hammad

Technical Editor: Nithik Cheruvakodan

Copy Editor: Safis Editing

Proofreader: Mohd Hammad

Indexer: Rekha Nair

Production Designer: Jyoti Kadam

DevRel Marketing Coordinator: Marylou De Mello

First published: December 2024

Production reference: 1281124

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB, UK

ISBN 978-1-83588-082-1

www.packtpub.com

To my wife, Kim Campbell, with love and gratitude for your unwavering love, support, strength, and encouragement. To my mother, Sandra Crawford, with a deep appreciation for your love and understanding. To my project team at Packt, with gratitude for your wisdom and guidance.

– Steve Campbell

Foreword

Throughout my years as a red teamer and penetration tester, one of the most powerful tools in my arsenal has been a deep understanding of Linux—particularly Bash shell scripting. Whether I’m combing through vast amounts of data or developing a custom exploit, Bash’s versatility is unmatched. It provides the flexibility to automate processes, manipulate system functionality, and streamline tasks that would otherwise take significant time. In offensive security, having a solid grasp of Bash is not just helpful—it’s essential. This book captures the essence of why Bash scripting is such a critical skill for penetration testers, showing you how to unlock its full potential.

This guide walks you through every phase of a penetration test, demonstrating how to apply Bash scripting to enhance your effectiveness. From reconnaissance, where gathering information is key, to exploitation and post-exploitation, where precision matters most, this book provides practical examples of how Bash can be utilized in offensive security scenarios. It’s not just about following scripts—it’s about gaining the understanding needed to write your own custom scripts tailored to the specific challenges you face in the field. The hands-on approach offered here ensures that you build both confidence and competence in using Bash for every aspect of an attack.

Beyond the technical skills, this book gives you a framework for thinking like a penetration tester—how to assess situations, adapt to new challenges, and create solutions on the fly. Steve not only covers specific techniques but also teaches you how to approach problem-solving, providing a foundation that empowers you to develop your own tools as needed. For anyone serious about learning Bash scripting for offensive operations, this book is an invaluable resource. It equips you with the skills and mindset necessary to be adaptable, innovative, and, ultimately, successful in your security career.

David Kennedy

Founder of TrustedSec and Binary Defense

Contributors

About the author

Steve Campbell is a technical lead on the CDW Offensive Security team. He is a retired Navy veteran who previously worked with aviation electrical and electronics systems before transitioning to information technology (IT). He possesses over 19 years of combined experience in IT and penetration testing. He has planned, scoped, led, and performed penetration testing engagements on various major enterprises, such as Fortune 500, government institutions, banking, finance, healthcare and insurance, e-commerce, legal, and energy sector clients. His achievements include the identification of seven vulnerabilities published as CVE, along with contributions to open source tools such as the Metasploit Framework.

About the reviewers

Jayaraman Manimaran is a seasoned security tester with over 9 years of expertise in DevSecOps, penetration testing, red teaming, and purple teaming. Having navigated the complexities of testing a service for diverse sectors, including banking, finance, and telecommunications, he brings a wealth of practical knowledge to the evaluation process. His commitment to knowledge dissemination is evident through his tech blogs, security research, and the publication of scripts aimed at simplifying the challenges faced by penetration testers. He holds certifications including CHMRTS, MCRTA, CARTP, CRTP, CRTA, eCPPT, CRT-ID, eWPT, CRT-COL, eJPT, PTF, and C|EH.

I extend my heartfelt gratitude to my family, particularly my supportive wife, for standing by me and understanding my demanding schedule. Special thanks to the author and Packt for the invaluable opportunity to contribute to this publication. Your support and understanding have made my role as a technical reviewer possible.

Andrew Aurand is a current adjunct instructor at Wilmington University. He has worked in the IT field for 14 years. He has taught introductory Python, introductory Linux, advanced Linux topics, and ethical hacking to undergraduate students. He is also the co-founder of a solutions-oriented cybersecurity company called Cipherlock Solutions. He has a master’s degree in cybersecurity from Wilmington University.

Anthony “RedHatAugust” Radzykewycz is a seasoned cybersecurity professional with over 10 years of experience in penetration testing, threat analysis, and vulnerability assessment. An OSCP-certified expert, Anthony has taught Linux and cybersecurity as an adjunct professor and has developed secure Linux distributions for high-stakes environments. His career includes serving as a penetration test lead for a Fortune 100 company and as a content developer at OffSec, where he authored comprehensive educational materials. As a dedicated reviewer and industry expert, Anthony provides insightful, accessible commentary that bridges technical depth with an engaging approach for all readers.

Table of Contents

Preface

Part 1: Getting Started with Bash Shell Scripting

1

Bash Command-Line and Its Hacking Environment

Technical requirements

Introduction to Bash

Lab setup

Virtual machines

Docker containers

Live USB

Cloud-based systems

Vulnerable lab targets

Configuring your hacker shell

Customizing the Bash prompt

Setting up essential pentesting tools

Update the package manager

Install ProjectDiscovery tools

Install NetExec

Summary

2

File and Directory Management

Technical requirements

Working with files and directories

Directory navigation and manipulation

Filesystem design and hierarchy

Filesystem navigation commands

File permissions and ownership

Ownership and groups

Special permissions – SUID and SGID

Linking files – hard links and symlinks

Summary

3

Variables, Conditionals, Loops, and Arrays

Technical requirements

Introducing variables

Declaring variables

Accessing variables

Environment variables

A review of variables

Branching with conditional statements

The if statement

Adding else

The power of elif

Beyond simple comparisons

Combining conditions

Case statements

Repeating with loops

The for loop

The while loop

The until loop

Select – interactive menus made easy

Advanced usage – nested loops

Using break and continue

Using arrays for data containers

Looping through arrays

Summary

4

Regular Expressions

Technical requirements

The basics of regex

Using character classes

Flags – modifying your search

Applying basic regex examples

Advanced regex patterns and techniques

Practical example – extracting data using regex

Utilizing alternations

Demonstrating practical applications

Matching IP addresses with grep

Using handy grep flags

Redacting IP addresses

Regex tips and best practices

Summary

5

Functions and Script Organization

Introduction to Bash functions

Code reuse

Modularity

Encapsulation

Testability

Performance

Defining and calling a function

Passing arguments to functions

Handling a variable number of arguments

Default values for arguments

The scope and lifetime of variables in functions

Global variables

Local variables

Variable lifetime

Modifying global variables inside functions

Advanced function techniques

Function return values

Recursive functions

Importing functions

Functions versus aliases

Summary

6

Bash Networking

Technical requirements

Networking basics with Bash

Understanding IP addresses and subnets (IPv4)

Understanding IP addresses and subnets (IPv6)

Configuring network interfaces using Bash commands

Troubleshooting network connectivity with Bash tools

Scripting network enumeration

Network exploitation

Network service exploitation

Network traffic analysis

Capturing and analyzing network traffic

Interpreting packet captures

Summary

7

Parallel Processing

Understanding parallel processing in Bash

Implementing basic parallel execution

Advanced parallel processing with xargs and GNU parallel

Introducing xargs for robust parallel processing

Using GNU parallel for enhanced control

Comparing xargs and parallel

Achieving parallelism using screen

Practical applications and best practices

Practical applications of Bash parallel processing

Best practices for parallel execution in Bash

Summary

Part 2: Bash Scripting for Pentesting

8

Reconnaissance and Information Gathering

Technical requirements

Introducing reconnaissance with Bash

Formatting usernames and email addresses

Using Bash for DNS enumeration

Expanding the scope using Bash

Automating subdomain enumeration with Bash

Using Bash to identify web applications

Using Bash for certificate enumeration

Using Bash to format vulnerability scan targets

Summary

9

Web Application Pentesting with Bash

Technical requirements

Automating HTTP requests in Bash

Analyzing web application security with Bash

ProjectDiscovery

Running command-line scans with ZAP

Learning advanced data manipulation techniques

Summary

10

Network and Infrastructure Pentesting with Bash

Technical requirements

Fundamentals of network pentesting with Bash

Core methodologies in network pentesting

Setting up the pentest environment

Using tmux for persistent sessions

Basic network scanning with Nmap

Fast network scanning with Masscan

Processing scan results with Bash

Conclusion

Advanced network scanning techniques in Bash

Enumerating network services and protocols using Bash

Infrastructure vulnerability assessment with Bash

Enumerating network hosts with NetExec

Automating vulnerability scanning with Greenbone

Summary

11

Privilege Escalation in the Bash Shell

Technical requirements

Understanding privilege escalation in Unix/Linux systems

Enumeration techniques for privilege escalation

Initial access

System information gathering

Exploiting SUID and SGID binaries with Bash

Leveraging misconfigured services and scheduled tasks

Summary

12

Persistence and Pivoting

Technical requirements

The fundamentals of persistence with Bash

Creating a new user in Bash

Backdooring the Bash shell

Creating backdoor cron jobs

Backdooring system files for persistence

Backdooring with SSH authorized keys

Learning advanced persistence techniques

The basics of network pivoting with Bash

Mastering advanced pivoting and lateral movement

Dynamic chain pivoting

DNS tunneling

Cleanup and covering tracks

Summary

13

Pentest Reporting with Bash

Technical requirements

Automating data collection for reporting with Bash

Identifying key data points

Parsing and cleaning raw data using Bash

Storing and managing pentest data with SQLite

Integrating Bash with reporting tools

Summary

Part 3: Advanced Applications of Bash Scripting for Pentesting

14

Evasion and Obfuscation

Technical requirements

Enumerating the environment for AV and EDR

Basic obfuscation techniques in Bash

Advanced evasion tactics using Bash

Automating evasion script generation in Bash

Summary

15

Interfacing with Artificial Intelligence

Technical requirements

Ethical and practical considerations of AI in pentesting

The basics of AI in pentesting

Basic terminology and definitions of ML and AI

Creating a foundation for successful AI use in pentesting

Redefining the system prompt

Enhancing vulnerability identification with AI

AI-assisted decision-making in pentesting

Testing the Pentest Hero AI agent

Summary

16

DevSecOps for Pentesters

Technical requirements

Introduction to DevSecOps for pentesters

Understanding the intersection of DevOps and security

Common use cases for Bash in security automation

Configuring the CI/CD pipeline with Bash

Initial setup and error handling

Logging functions

Error handler and initialization

System checks

Development tools installation

Security tools installation

GitLab CI/CD setup

Workspace creation

Crafting security-focused Bash scripts for DevSecOps

Creating the scan script

Creating vulnerable artifacts

Integrating real-time security monitoring with Bash

Automating custom Kali Linux builds for pentesting

Summary

Index

Other Books You May Enjoy

Part 1: Getting Started with Bash Shell Scripting

In this part, you will establish a solid foundation in Bash scripting specifically tailored for pentesting. Beginning with setting up a proper hacking environment and configuring the Bash shell, this section progresses through essential file and directory management techniques needed for security assessments. You will master core programming concepts, including variables, conditionals, loops, and arrays, before diving into pattern matching with regular expressions – a crucial skill for parsing security tool outputs. The section then advances to function creation and script organization, ensuring you can build maintainable, professional-grade security tools. Moving into networking fundamentals, you will learn how Bash interacts with network services and protocols. The section concludes with parallel processing techniques, enabling you to develop efficient scripts that can handle multiple tasks simultaneously – an essential capability for large-scale security assessments. By the end of Part 1, you will have all the fundamental skills needed to begin writing sophisticated security-focused Bash scripts.

This part has the following chapters:

Chapter 1, Bash Command-Line and Its Hacking EnvironmentChapter 2, File and Directory ManagementChapter 3, Variables, Conditionals, Loops, and ArraysChapter 4, Regular ExpressionsChapter 5, Functions and Script OrganizationChapter 6, Bash NetworkingChapter 7, Parallel Processing

1

Bash Command-Line and Its Hacking Environment

In this foundational chapter, you will embark on your journey into the world of Bash shell scripting for penetration testers (pentesters). You will gain a clear understanding of what Bash is, why it is essential for penetration testing (pentesting), and how to set up your scripting environment. Through hands-on examples and explanations, you will lay the groundwork for becoming a proficient Bash scripter in the context of cybersecurity.

Bash is more than just a command interpreter – it’s a tool for automating the complex and tedious tasks that we encounter daily in cybersecurity. In the hands of the untrained, Bash is a club. It seems heavy, overly complex, and uncomfortable. In the hands of those able to see the benefits and invest time in learning its intricacies, it’s a scalpel that you can use to slice through data with the skill of a surgeon and automate pentesting methodology like a robotics engineer.

In this chapter, we’re going to cover the following main topics:

Introduction to BashLab setupConfiguring your hacker shellSetting up essential pentesting tools

Technical requirements

To follow along with the exercises in this chapter, you’ll need a Linux environment. This book assumes you have enough skill to install an operating system and are familiar with installing and configuring virtual machine environments. If you need help setting up your lab environment, the VirtualBox online manual (Oracle VM VirtualBox User Manual, (https://download.virtualbox.org/virtualbox/UserManual.pdf) and several YouTube videos (VirtualBox – YouTube https://www.youtube.com/results?search_query=virtualbox) will be helpful.

Fortunately, there are many ways to configure a Bash learning environment for free. All examples will be shown using Kali Linux. However, any Linux or macOS environment will work.

“Kali Linux is an open source, Debian-based Linux distribution geared toward various information security tasks, such as penetration testing, security research, computer forensics, and reverse engineering.” (Kali Linux, https://www.kali.org/)

I strongly suggest that you use a fresh Kali Linux virtual machine to follow along with the exercises or when performing pentests. Throughout this book and your pentests, you will be installing a lot of tools and their dependencies. It’s common for tool dependencies to clash and create what is known as dependency hell. This could result in damage to your main system if you haven’t properly isolated the tools during installation. You also don’t want to risk infecting your main system with malware.

Kali offers a wide variety of solutions. They provide installer images, virtual machines, cloud images, and Windows Subsystem for Linux (WSL) packages.

You can download Kali from https://www.kali.org/get-kali/#kali-platforms.

All the commands that will be used in this chapter can be found in this book’s GitHub repository at https://github.com/PacktPublishing/Bash-Shell-Scripting-for-Pentesters/tree/main/Chapter01.

Introduction to Bash

Bash, also known as the Bourne Again Shell, is a command-line shell interpreter and scripting language. Bash was created by Brian Fox in 1989 as a free software replacement for the Bourne shell, which was proprietary software. (Bash – GNU Project – Free Software Foundation, https://www.gnu.org/software/bash/). It’s the most common Linux shell. Bash also introduced the ability to combine multiple commands into shell scripts that could be run by entering one command.

When you open a Terminal on a Linux system and enter commands, your Bash shell manages interactions with the operating system and running executables and scripts. Bash and Linux executables form a symbiotic relationship, each enhancing the functionality and efficiency of the other. Bash serves as the gateway for users and scripts to interact with the Linux kernel, the core of the operating system. It interprets user commands, whether entered directly into the Terminal or scripted in files, and initiates actions within the system. Linux executables, on the other hand, are the workhorses that carry out these actions. They are binary files, often written in programming languages such as C or C++, compiled to run efficiently on Linux systems. When a user issues a command in Bash, it often involves invoking one or more of these executables to perform a task.

The following are some of Bash’s features:

Command execution with arguments: Commands can be binaries, built-in shell commands, and scripts.Command completion: A feature that helps the user by automatically completing partially typed commands or filenames upon the Tab key being pressed.Command history: Command history allows you to quickly reuse commands previously entered in the shell.Job control: Sending commands to the background and bringing them to the foreground.Shell functions and alias: A function groups related code under a name that can be called when needed. An alias allows the user to shorten complex commands to a single name.Arrays: Arrays store elements in a list that we can later retrieve and process.Command and brace expansion: Command expansion uses the result of a command as the input for another command. Brace expansion allows strings to be generated.Pipelines and redirection: The output of one command is used as input for another command.Environment variables: Dynamic values are assigned to name tags, which are frequently used to represent system configuration or store information about the environment.Filesystem navigation: Bash provides commands to change directories, print the current directory, and find files and directories.Help: The man command, short for manual, provides the user with information and examples of how to execute commands.

Bash scripting is one of the most important skills that I’ve learned in my pentesting career and that I use daily. When you’re developing applications, at some point, you’ll find the need to use another scripting language, such as Python. In most cases, anything you could want to do at the Terminal can be done in Bash, with Bash orchestrating the input and output and parsing the data from multiple tools. Bash is so deeply integrated with the Linux operating system that it makes sense to learn it before branching out into scripting languages such as Python or Ruby. Despite knowing multiple scripting and programming languages, Bash is the one that I use most often due to its tight integration with the shell and how easy it is to quickly get results with a one-line or even one-word command.

On any given day in my work as a pentester, I use Bash to parse data or automate chaining together multiple tools. When a customer gives me scoping data, I must frequently copy a list of scoped IP addresses or hostnames from a Rules of Engagement document, email, or Excel spreadsheet and paste it into a text file. Inevitably, there are stray characters in the data, or the data isn’t formatted cleanly for use as a list of scan targets. I can use Bash to clean up the file data and format it as I need for testing purposes with one simple line of code entered in my Terminal.

Pentesting tools accept data in various formats, and output scan results or data in common formats such as XML, JSON, or plain text. The plain text output may be formatted with multiple spaces, tabs, or a combination. I pipe the contents of a source file and pass it through the Bash pipeline to parse, clean, reformat, and sort the data. I may use a combination of Bash commands to perform these actions in between the output of one command and the input of another in an automation pipeline. Bash truly is an indispensable tool in a pentester’s toolbox.

The following are some common uses for Bash scripting in my pentesting workflow:

Automated network scanning: I frequently process the output of Masscan, a fast TCP scanner, and feed it to Nmap for in-depth service detection and script scanning.Password cracking: I use a Bash script for a complex series of password cracking functions related to cracking Microsoft LM and NTLM hashes and formatting the output of Hashcat for input into a reporting tool.Searching text: Searching for IP addresses or other details in text.Scoping automation: I use subdomain enumeration tools with a Bash script to ensure the discovered subdomains are in the scope of the pentest rules of engagement.Formatting data: I use Bash to parse and reformat the output of Nuclei scans to enumerate subdomains and web applications from TLS certificates and reformat the data for use in bypassing content delivery networks (CDNs) to bypass a web application firewall (WAF) and scan the target directly.Searching and sorting Nmap reports: After scanning hundreds or even thousands of IP addresses, I use Bash to parse the gnmap files to create text files containing targets organized by TCP or UDP ports for use in more targeted scans. For example, all SMB servers or HTTP servers’ IP addresses are carved out and placed into files named smb.txtand http.txt.Sorting data and deduplication: Sort the unique IP addresses into a file for deduplication.Data conversion: Convert first and last names into various formats for password spraying. If I can get a list of employee names through Open Source Intelligence (OSINT), I’ll look at anything that may tip me off to how their Active Directory names are formatted, such as f.last or first.last, and use Bash to format the names appropriately.Data filtering: Occasionally, I have to remove Terminal color codes from tool output log files for use in reporting because I forgot to include a command-line flag for no color, or the tool may not have this option. I don’t want to screenshot data for my customer’s report with it containing color codes that make the data confusing to read.Iterating over data: I use Bash for and while loops to loop through a file and run a command on each line. A good example of this is when you need to use a tool that scans one host at a time with no option to process multiple targets.

I’m confident that learning Bash scripting will make you more efficient with your time and more effective in your job. When you can automate time-intensive, boring tasks using Bash, it frees up your time to take on more important things. Wouldn’t it be great to have more time for learning or research instead of wasting it on manual tasks that can be automated with little effort?

Now that we have a basic understanding of Bash and why it’s useful in our pentesting endeavors, let’s explore how to set up a lab environment where you can safely learn and follow along with the exercises. In the next section, we’ll explore setting up your lab environment so that you can follow along with me.

Lab setup

Bash isn’t the only shell interpreter for Linux and Unix systems, but it is the most common. Other shells were influenced by Bash. You may also encounter Zsh on macOS and Kali Linux.

You might be wondering why this book has chosen to focus on Bash, despite some operating systems switching to Zsh. While macOS and Kali have switched to Zsh for new user accounts, they still have Bash installed. Most code written for Bash will also work on Zsh with a few minor changes. You can include a shebang line in your shell scripts to ensure that the Bash interpreter runs your script on systems where multiple shells are installed. While performing security assessments, you’re very likely to encounter Linux servers where Bash is the default shell. It will be essential for a pentester to understand how to interact with Bash to exploit applications, escalate privileges, and move laterally.

Fortunately, there are many ways you can access a Bash shell for free. This section will explore a variety of ways you can access a Bash shell in an ideal setting so that you can follow along and learn how to use Bash for pentesting. We will also explore vulnerable lab environments where you can safely practice using Bash and pentesting tools.

Virtual machines are the preferred way to follow along with this book’s activities, as well as when performing pentesting. You may be tempted to install your pentesting tools and exploit code on the same system you use for business or personal activities. It’s easy to damage your system by installing software prerequisites for various tools. There’s always a risk of hacking tools containing malware and infecting the same system that you use from day to day to send emails or access the web. A virtual machine provides a convenient sandbox environment with everything you need to quickly refresh or replace a testing environment. I have chosen to use Kali Linux in all demonstrations. We want to avoid installing pentesting tools and exploit code in the same system we use daily for business or personal use. It’s best to use a clean testing environment to avoid creating software dependency issues for us. Kali makes it easy to install the needed software packages related to pentesting.

Virtual machines

Using a virtual machine is the preferred method. During a pentest, you’ll likely install a multitude of tools and exploit proof of concept code. At some point, you’ll also save sensitive data about your customer or target. A virtual machine provides a convenient container that you can snapshot and restore, or delete and replace easily after an assessment.

There are numerous free and paid virtualization solutions to fit any need:

Oracle VirtualBox is a free x86 virtualization hypervisor. It’s available for Windows, macOS (Intel chipset), and Linux. VirtualBox is user-friendly, making it a popular choice for beginners and professionals alike. It supports a wide range of guest operating systems and offers features such as snapshots, seamless mode, and shared folders.VMware offers a free version of their virtualization software called VMware Workstation Player for non-commercial use. It’s compatible with Windows and Linux hosts. Workstation Player is easy to use and supports VMware’s VMDK virtual disk format, and it’s also compatible with virtual machines created by other VMware products.Microsoft Hyper-V is free and available on Windows 10 Pro, Enterprise, and Education editions. While it’s more commonly used in server environments, Hyper-V can also be a good option for desktop virtualization on Microsoft Windows hosts.

Tip

For those on macOS with the Apple CPU, your virtualization options are UTM, Parallels, and VMWare Fusion. UTM is the only free option.

Docker containers

Docker containers offer a lightweight option over virtual machines. Docker offers a runtime for Windows, Linux, and macOS. Containers are more lightweight and efficient on lower-end hardware than virtual machines because they use the host’s kernel, so they don’t have to virtualize hardware as traditional hypervisors do.

Because Docker uses the host’s kernel, you’re limited to running containers while utilizing the same operating system as the host. Docker Desktop is an alternative that uses a virtual machine to run containers with a different operating system from the host.

Based on my experience, there are some positive and negative points to consider about using Docker.

Docker is more lightweight and is a good alternative to traditional hypervisors when your hardware is less robust. The minimum hardware resources I would assign to a virtual machine running Kali Linux is 4 GB of RAM and 40 GB of disk space. You’re not always going to be using that 4 GB/40 GB. At the same time, you’re limited to those values unless you shut down the virtual machine, adjust the RAM, and extend the disk. A Docker container runs in a native process (excluding Docker Desktop), so it uses only as much memory and disk space as needed to run the container.

On a Linux host, you can attach a container directly to the host network and open and close ports as needed, provided you include specific command-line arguments. This allows you to dynamically open listening server ports on the host’s network adapter without stopping and starting the container. You can also attach a container to a USB or serial port to interface with hardware devices. I sometimes use this option when I need to run an old Python2 pentesting application that interfaces with a USB or serial device for radio frequency and hardware hacking.

When using Docker Desktop, NAT is used to connect container network ports to the host’s network, so the container must be stopped and restarted if you need to close or open additional ports. With Docker Desktop, it isn’t possible to attach a container to hardware devices. This can be aggravating when you’ve configured an application and its dependencies on a container and then lose your work and have to start over when you destroy the container and start a new instance, just to open another TCP port for a reverse listener or server application.

In summary, my preference is to use Docker only on a Linux host, and I use it for three specific pentesting use cases:

It provides an easy way to isolate old Python 2 applications and avoid dependency hell. There are official Docker containers for all Python 2 and 3 versions.I use it to create and run applications that aren’t available through my package manager, and I want to avoid wasting time solving dependency issues. For example, a particular hacking tool is available through the Kali software repository, but not in Ubuntu. I can create a thin Kali container that uses only enough resources to run the contained application and use an alias in my ~/.bashrc file to reduce a long docker run command to a single word I can enter in my Terminal. This is a much faster and more lightweight option than a heavy virtual machine when I just want to run a single application that can’t be run or would be difficult to run on my host system.When I want to practice exploiting or creating an exploit tool for a recently announced vulnerable web application, I can frequently find a Docker container that allows me to immediately start the vulnerable application without spending precious time installing and configuring it.

Docker containers are perfect for specific use cases. However, they’re less preferred than virtual machines. Next, we’ll explore using live USB systems as an alternative to virtual machines and containers.

Live USB

A live USB is an operating system image written to a USB disk in a way that makes it bootable. Live USB is a good choice to use when your computer doesn’t have the hardware resources to run a virtual machine. You can use imaging software to burn Linux ISO disks to USB and boot a Linux operating system. After you finish your work on Linux, you simply restart the computer and remove the USB drive to revert to the installed operating system. Some Linux distributions enable you to create persistent storage on the USB drive so that you don’t lose your changes when you reboot.

The following are some general steps for running a Linux distribution from live USB:

Download the ISO image. Some popular Linux distributions for pentesters include Kali, Parrot Security OS, and BlackArch.Create a live USB drive. Common tools for this purpose include Rufus, balenaEtcher, and the Linux dd command.Configure persistence (optional). This usually involves creating a separate partition on the USB drive and configuring the bootloader to recognize and use this partition. You can find the documented steps required to create a live USB Kali system at https://www.kali.org/docs/usb/usb-persistence/.

There are some considerations and drawbacks to using live USB:

USB storage is usually much slower than running directly from an SSD. If you use live USB, be sure to use the USB 3.0 or 3.1 standard for best performance.Always download the ISO image from official sources and verify the checksum before trusting it.If you’re planning to use it for production use, be sure to use encrypted persistence due to the risk of exposing the sensitive data on the drive to someone not authorized to have it.

Now, let’s move on and discuss cloud-based systems.

Cloud-based systems

Many cloud platforms create free tiers for access to Linux systems with enough resources for modest workloads. Cloud providers with free tiers include Google Cloud Platform (GCP), Microsoft Azure, and Amazon EC2. Be aware that the free tier may not provide enough RAM for production use and will not be suitable for running Kali Linux images.

Kali Linux provides documentation and marketplace images for running on AWS, Digital Ocean, Linode, and Azure (https://www.kali.org/docs/cloud/). I have experience with customers who have configured Kali in the cloud for cloud security assessments, or connected via VPN to their internal network infrastructure to facilitate internal network pentests. If the customer’s internal network is already connected to a cloud provider over VPN, it’s relatively easy for them to spin up a Kali image and create a firewall rule to allow SSH access from my IP address. Now that we’ve explored options for running a pentesting system with Bash, let’s discover a few vulnerable systems we can use to practice within our lab.

Vulnerable lab targets

While following along with some of the later chapters related to pentesting methodology, it will be beneficial for you to have access to vulnerable targets when running the commands and developing your Bash scripts. There are several great sources of vulnerable targets you can use for practice in your lab.

Metasploitable 2 is a vulnerable virtual machine provided by Rapid7. It was designed to showcase the capabilities of the Metasploit Framework. Metasploitable 2 is also a good beginner-level challenge for developing your hacker methodology and learning Bash for pentesting. The project requires modest resources to run the virtual machine and includes documentation on the machine’s vulnerabilities (Metasploitable 2 | Metasploit Documentation, https://docs.rapid7.com/metasploit/metasploitable-2/).

Game of Active Directory (GOAD) is also an option.

“GOAD is a pentesting Active Directory LAB project. The purpose of this lab is to give pentesters a vulnerable Active Directory environment ready to use for them to practice usual attack techniques.” (Game of Active Directory – Orange-CyberDefense, https://github.com/Orange-Cyberdefense/GOAD)

Note that GOAD is free to use and uses free Microsoft Windows licenses that are activated for 180 days. GOAD is the best resource I’ve found for practicing hacking on internal Active Directory network environments.

MayFly is the creator of GOAD. Their website contains plenty of articles on how to set GOAD up on different virtual machine hypervisors, as well as lab guides for using common pentesting tools to hack Active Directory.

Tip

MayFly also published a comprehensive mind map for pentesting Active Directory. Despite having years of experience in hacking Active Directory, I still find times that I’m running out of things to test and will refer to this mind map when I get stuck or want to ensure that I’ve left no stone unturned. This mind map is also the number one resource I recommend to junior pentesters who are learning Active Directory hacking techniques and tools (you can find more details at https://orange-cyberdefense.github.io/ocd-mindmaps/img/pentest_ad_dark_2022_11.svg).

If you wish to practice your Bash scripts, tools, and methodology on web applications, OWASP Juice Shop is a great resource.

“OWASP Juice Shop is probably the most modern and sophisticated insecure web application! It can be used in security training, awareness demos, CTFs, and as a guinea pig for security tools! Juice Shop encompasses vulnerabilities from the entire OWASP Top Ten (https://owasp.org/www-project-top-ten/), along with many other security flaws found in real-world applications!” (OWASP Juice Shop – OWASP Foundation, https://owasp.org/www-project-juice-shop/)

An older yet still very relevant vulnerable web application is Mutillidae II.

“OWASP Mutillidae II is a free, open source, deliberately vulnerable web application that provides a target for web-security training. With dozens of vulnerabilities and hints to help the user, this is an easy-to-use web hacking environment designed for labs, security enthusiasts, classrooms, CTF, and vulnerability assessment tool targets.” (OWASP Mutillidae II – OWASP Foundationhttps://owasp.org/www-project-mutillidae-ii/)

One of the things I love about Mutillidae is that it embeds hints, tutorials, and video tutorials in the content. Mutillidae is a resource I used many years ago to learn web app testing when I was a junior pentester. The difference between Juice Shop and Mutillidae is that Juice Shop is a modern web application that uses JavaScript frameworks, whereas Mutillidae is a more traditional web application. While Juice Shop has a scoreboard and you can find third-party walkthroughs online, Mutillidae has a large amount of training text and video embedded in the application.

The cybersecurity landscape is always changing, and new vulnerabilities are discovered regularly. A lab setup is an ideal place for research and development, allowing you to experiment with these vulnerabilities safely. It’s where you can contribute to the cybersecurity community by discovering new vulnerabilities or enhancing existing pentesting methodologies.

Now that we’ve explored vulnerable targets for your pentesting lab, next up, we’ll talk about customizing your Bash shell so that it suits your needs and personal style.

Configuring your hacker shell

If you’re following along using Kali Linux or macOS, note that your Terminal shell uses Zsh by default instead of Bash. Zsh has more features (such as better tab completion and theme support) but Bash is more widespread and standard. Bash has been around since the late 80s, making it a veteran in the shell world. It’s the default on most Linux distributions and macOS (up until Catalina, where Zsh took over). Bash’s longevity means it’s extremely stable and well-supported.

Zsh, on the other hand, came a bit later. It’s known for its improvements over Bash, including better interactive use and more powerful scripting capabilities.

You can determine which shell is configured by entering the echo $SHELL command in your terminal. Almost all code shown in this book will work in both Bash and Zsh, except where noted. In my day-to-day pentesting activities, I rarely notice any difference. However, if you want to change your shell from Zsh to Bash, execute the chsh -s /bin/bash command in your Terminal, then log out and log in to see the change take place.

Bash configuration files can be found in the user home directory, /home/username. Because the filenames begin with a period character, they are commonly referred to as dotfiles. The following configuration files are used to configure the Bash shell:

~/.bash_profile: This file is executed at the start of an interactive login and is used to initialize the user environment. Think of an interactive login as logging in via the command line via a text-based Terminal such as an SSH session.~/.bashrc: This file is used to configure the Terminal when you’ve logged in through the graphical user interface (GUI). This file contains settings including aliases, functions, prompt customizations, and environment variables.~/.bash_logout: This file is executed when your session ends. It’s used to perform tasks related to cleaning up the environment when you log out.

Tip

If you don’t understand the purpose of the tilde (~) character and forward slash preceding the dotfile name, the tilde character represents the user’s home directory. The ~/.bashrc path is equivalent to /home/username/.bashrc. This concept will be covered in Chapter 2.

The most common edits you’ll want to make will include adding aliases and functions and customizing your command prompt in your ~/.bashrc file. Aliases are a great way to shorten long or complex commands to a single word. Functions are more complex. Think of functions as a short script that you can include in your shell configuration and call by name in your Terminal. Functions will be discussed later in Chapter 5.

Here’s an example alias from my ~/.bashrc file that I use to search for IP addresses in text:

alias grepip="grep -Eo '([0-9]{1,3}\.){3}[0-9]{1,3}'"

You can see how this command would be difficult to remember, so it helps to create an alias for any complex command you may need to use repeatedly.

Important note

When you make edits to your Bash configuration files, you must either log out and log in or source the file to enact the change.

Enter the following command to source a file and enact changes immediately:

$ source ~/.bashrc

Now that you understand the purpose of Bash’s dotfiles, let’s move on and take a look at how we can edit them to personalize our environment.

Customizing the Bash prompt

The prompt is where you enter commands in the Bash Terminal. Your prompt can be as simple or complex as you desire to meet your tastes and reflect your personality. Think of your prompt design choices like how a painter chooses their palate.

You can find your currently configured prompt by looking in your ~/.bashrc file for a line that begins with PS1. A common Bash prompt would use a PS1 value such as export PS1="\u@\h \w\$ ", and it would look like username@hostname ~$ at the prompt. Let’s break this down. Here’s what each part does:

\u will be replaced by the current username.@ is a literal character and will appear after the username.\h will be replaced by the hostname up to the first period.\w will be replaced by the current working directory, with $HOME abbreviated to a tilde character.\$ displays a $ character for a regular user, or # for the root user.

Once you edit your PS1 prompt, be sure to source the file to see the changes take effect.

You can also get really fancy with your prompt. I’ve been known to insert the $(ip a show eth0 | grep -m 1 inet | tr -s ' ' | cut -d ' ' -f 3) string in the middle of my PS1 prompt to show my IP address for capturing in my logs or report screenshots for the customer to correlate my activity with their Security Information and Event Management (SIEM) alerts. See https://bash-prompt-generator.org/ for a graphical Bash prompt generator, or the official Bash manual for all options.

Customizing your Bash environment is about making your Terminal work for you. It’s a process of trial and improvement, finding what makes you more productive, and what brings a bit of joy into your command-line sessions. Start small, experiment, and see how a few changes can make a big difference in your daily tasks.

Setting up essential pentesting tools

In this section, we’ll go over setting up our pentesting environment by updating system software packages and installing the tools required to follow along. Most of the tools needed will already be installed in Kali, so we’ll only need to install a few more software packages.

Update the package manager

Your first step when using a new Linux installation should be updating packages. As stated earlier, I’ll be using Kali Linux in all demonstrations. Kali is based on the Debian Linux distribution, which uses the Advanced Package Tool (APT) package manager. At its core, apt streamlines software management. It automates the process of retrieving, configuring, and installing software packages from predefined repositories. This automation not only saves time but also ensures that software dependencies are resolved without manual intervention.

Running sudo apt update refreshes the local database of available packages and their versions, ensuring you have the latest information from the repositories. This step is crucial before installing new software or updating existing packages to ensure you’re getting the latest versions. If you’re using Kali, Ubuntu, or Debian Linux, the following commands to update and upgrade will work as expected because they all use the aptpackage manager:

$ sudo apt update && sudo apt upgrade -y && reboot

In the preceding command, we use sudo to elevate privileges and apt to update the list of available packages. The double ampersand symbols (&&) operate like a logical AND operator; the second command to upgrade packages without prompting (-y) is only run if the first command results in success. Finally, we reboot to ensure that all services and kernel updates take effect.

Install ProjectDiscovery tools

ProjectDiscovery offers some great tools I recommend for pentesting (PDTM – ProjectDiscovery, https://github.com/projectdiscovery/pdtm). Before we can install them, we must install the Go programming language runtime and libraries. Follow these steps to do so:

In your web browser, navigate to https://go.dev/dl/.Download the correct package for your Linux distribution. Be sure to look closely at the processor architecture. Typically, this would be a Kind value of archive, an OS value of Linux, and an Arch value of either x86-64or ARM64.Extract the downloaded archive. Be sure to change the package version so that it matches what you downloaded:$ sudo tar -C /usr/local -xzf go1.22.0.linux-amd64.tar.gzAdd /usr/local/go/bin to the PATH environment variable in your ~/.bashrc file. The PATH environment variable tells your Bash shell where to find the full path to an executable program when you don’t include the path before your command. The echo command prints the text inside quotes to the Terminal and the greater-than symbol (>) redirects the output to a file. Notice that we use two greater-than symbols here to redirect the output. If we were to use only one, it would overwrite the file contents. We want to append to the file by using two:$ echo "export PATH=$PATH:/usr/local/go/bin" >> ~/.bashrcSource the file to enact your changes:$ source ~/.bashrcCheck to ensure that /usr/local/go/bin has been appended to your PATH (look after the last colon character):$ echo $PATH/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games:/usr/local/go/binVerify that Go has been installed properly and can be found in your PATH. Your version and architecture may vary:$ go versiongo version go1.22.0 linux/arm64Install pdtm from ProjectDiscovery. This is the tool that installs and manages updates for all of ProjectDiscovery’s tools:$ go install -v github.com/projectdiscovery/pdtm/cmd/pdtm@latestAdd pdtm to your path:$ echo "export PATH=$PATH:$HOME/.pdtm/go/bin" >> ~/.bashrcRun the following pdtm command to install all tools:$ pdtm -install-allInstall libpcapfor naabu:$ sudo apt install -y libpcap-dev

That wraps up installing all of the needed ProjectDiscovery tools.

Install NetExec

NetExec is a network service exploitation tool that helps automate assessing the security of large networks (NetExec wiki, https://www.netexec.wiki/).

In my opinion, NetExec is one of the most useful tools for internal network pentesting. It supports most of the network protocols needed during internal network pentesting, plus Microsoft Active Directory testing.

There are far too many features to list here. Some of the things I use NetExec for include the following:

Scanning for vulnerabilities; NetExec includes some useful modules to test for common vulnerabilitiesBrute-force attacks on authentication to test for weak passwordsSpraying a password or password hash against servers to find where the supplied credentials have local administrator accessCommand executionGathering credentialsEnumerating SMB shares for read/write access

Enter the following command to install NetExec:

$ sudo apt install -y pipx git && pipx ensurepath && pipx install git+https://github.com/Pennyw0rth/NetExec

That wraps up the process of installing the most common pentesting tools that are not installed by default.

Summary

In this chapter, you were introduced to the indispensable world of Bash shell scripting, a cornerstone skill for anyone aspiring to excel in pentesting. This chapter began by demystifying what Bash is and underscoring its significance in cybersecurity tasks. It wasn’t just about memorizing commands; it was about leveraging Bash to automate repetitive tasks, manipulate data, and conduct security assessments with efficiency. The journey continued with guidance on selecting the appropriate operating system that supports Bash, setting the stage for successful scripting endeavors. Then, we rolled up our sleeves to configure our hacker shell, customizing its appearance and behavior to reflect personal tastes and preferences. This customization wasn’t just for aesthetics; it was about creating a functional and efficient working environment. Finally, this chapter introduced essential pentesting tools, walking you through their installation and basic usage. At this point, you're equipped with a well-prepared environment and a foundational understanding of how Bash scripting can significantly enhance your pentesting capabilities.

The next chapter will cover techniques for working with files and directories.

2

File and Directory Management

Mastering Bash file and directory management equips you with the skills to navigate the filesystem efficiently, manipulate files and directories, control access through permissions, and automate routine tasks. These abilities are essential for anyone looking to harness the full power of their Linux or Unix system. With practice, patience, and a bit of creativity, you can turn the complexity of the filesystem into a well-organized collection of files and directories at your command.

By the end of this chapter, you will become skilled at creating, deleting, copying, and moving files. You will understand the significance of absolute and relative paths. This will also include an introduction to directory structures and how to efficiently navigate the filesystem in a Bash environment. You’ll grasp the concept of user and group permissions in a Linux environment. You’ll learn the difference between hard links and symbolic links (symlinks or soft links), how to create them, and scenarios where each type of link is useful.

In this chapter, we’re going to cover the following main topics:

Working with files and directoriesDirectory navigation and manipulationFile permissions and ownershipLinking files—hard links and symlinks

Technical requirements

Access to a Linux system with a Bash shell is required to follow along. All commands used in this chapter can be found in the GitHub code repository located at https://github.com/PacktPublishing/Bash-Shell-Scripting-for-Pentesters/tree/main/Chapter02.

Working with files and directories