Linux for System Administrators E-Book

Viorel Rudareanu

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 399

Veröffentlichungsjahr: 2023

Beschreibung

Linux system administration is an essential aspect of maintaining and managing Linux servers within an organization. The role of a Linux system administrator is pivotal in ensuring the smooth functioning and security of these servers, making it a critical job function for any company that relies on Linux infrastructure.
This book is a comprehensive guide designed to help you build a solid foundation in Linux system administration. It takes you from the fundamentals of Linux to more advanced topics, encompassing key areas such as Linux system installation, managing user accounts and filesystems, networking fundamentals, and Linux security techniques. Additionally, the book delves into the automation of applications and infrastructure using Chef, enabling you to streamline and optimize your operations.
For both newcomers getting started with Linux and professionals looking to enhance their skills, this book is an invaluable hands-on guide with a structured approach and concise explanations that make it an effective resource for quickly acquiring and reinforcing Linux system administration skills. With the help of this Linux book, you’ll be able to navigate the world of Linux administration confidently to meet the demands of your role.

Leseprobe

Linux for System Administrators

Navigate the complex landscape of the Linux OS and command line for effective administration

Viorel Rudareanu

Daniil Baturin

BIRMINGHAM—MUMBAI

Linux for System Administrators

Copyright © 2023 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Pavan Ramchandani

Publishing Product Manager: Neha Sharma

Senior Editor: Arun Nadar

Technical Editor: Yash Bhanushali

Copy Editor: Safis Editing

Project Coordinator: Ashwin Kharwa

Proofreader: Safis Editing

Indexer: Pratik Shirodkar

Production Designer: Vijay Kamble

Marketing Coordinator: Marylou De Mello

First published: 1250823

Production reference: 1280823

Published by Packt Publishing Ltd.

Grosvenor House

11 St Paul’s Square

Birmingham

B3 1RB

ISBN: 978-1-80324-794-6

www.packtpub.com

Contributors

About the authors

Viorel Rudareanu has over 15 years of experience working with Linux. He possesses a comprehensive understanding of operating systems and their intricacies. He has successfully implemented and managed Linux-based infrastructures, leveraging the platform’s robustness and flexibility to optimize system performances.

I want to thank the people who have been close to me and supported me, especially my wife, Janina, and my two daughters, who sometimes missed me while writing the book.

Daniil Baturin was first introduced to Linux by older friends when he was a high school student in 2002, and he has worked with Linux systems professionally since 2007 – as a systems administrator, software engineer, and contributor to multiple open source projects. He’s currently a maintainer of VyOS – a Linux-based network operating system focused on enterprise and service provider routers.

I would like to say thanks to all people who work on open source software.

About the reviewers

Himanshu Sharma has nearly 18 years of experience in designing, architecting, and developing cloud and network software. He has worked for some of the biggest companies, such as Brocade and Juniper, and start-ups, such as Netskope. Currently, he works with Netskope as a principal engineer, responsible for Netskope’s security service offering. He designed, architected, and developed Netskope’s advanced threat protection services from the ground up. He has a keen interest and experience in developing scalable cloud services with cutting-edge technologies. His favorite hobbies are skiing and playing video games.

I would like to thank my wife, Puja, who gave me all the support I needed, and my two loving and beautiful daughters, Navya and Jaanvi.

Also, I want to thank my brother Sudhanshu for always having my back and my parents for all their sacrifices to get me where I am today.

Lekshmi Narayan Kolappan is an exceptional trailblazer in the realm of SRE/DevOps engineering. With an illustrious decade-long career in the software industry, his expertise shines through in the areas of AWS DevOps, SRE, Kubernetes, Docker, and Linux.

He is currently a site reliability engineer at Extreme Reach Limited, London, with a focus on implementing cutting-edge observability tools and performance monitoring.

Beyond technical brilliance, his dedication to refining the software development life cycle (SDLC) is unparalleled. He streamlines processes, crafts infrastructure audits, and automates testing frameworks.

Beyond the corporate world, in his spare time, he delights in exploring photography, literature, and tech blogs.

I wish to extend my heartfelt gratitude to my wife and my little son, who wholeheartedly comprehend the time and dedication involved in my pursuit of growth and development in the open source world. Their unwavering support is the cornerstone of my journey, and I truly cherish their understanding and encouragement.

Table of Contents

Preface

Part 1: Linux Basics

1

Getting to Know Linux

The structure of a Linux system

The Linux kernel and Linux-based operating systems

Kernel versus user space

The Linux kernel and the GNU project

Open source software licenses

Permissive and copyleft licenses

Patent grant, tivoization, and SaaS concerns

Linux distributions

Package managers and package repositories

Differences between distributions

Linux usage in the modern world

Summary

2

The Shell and Its Commands

What is a shell?

Basic shell commands

Intermediate shell commands

Summary

3

The Linux Filesystem

What is a filesystem?

High scalability

High performance

What filesystem does my system use?

FUSE filesystem

The directory tree and standard directories

Links (hard and symbolic)

What is an inode?

What is a hard link?

What are symbolic links?

Mounting and unmounting filesystems

How to unmount the filesystem

Pseudo-filesystems

Processes

Kernel and system information

CPU usage

Summary

4

Processes and Process Control

Executables versus processes

Process termination and exit codes

Exit codes

Signals

The kill command

The process tree

Process search and monitoring

The ps command

Process monitoring tools

The /proc filesystem

Summary

5

Hardware Discovery

Discovering CPU model and features

Feature discovery on different platforms

The /proc/cpuinfo file

Multi-processor systems

High-level CPU discovery utilities

Memory discovery

Discovering PCI devices

Discovering USB devices

Discovering storage devices

High-level discovery tools

dmidecode

lshw

Summary

Part 2: Configuring and Modifying Linux Systems

6

Basic System Settings

Overview of basic settings

The hosts configuration file

The resolv configuration file

The network-scripts configuration file

The dhclient configuration file

The sysctl configuration file

Summary

7

User and Group Management

Overview of managing accounts/groups

How to add a new account

Using useradd

Using adduser

How to delete an account

Understanding the/etc/sudoers file

Switching users

Managing account passwords

Locking/unlocking user accounts

Setting password expiration

Group management

Permissions

Changing groups

Summary

8

Software Installation and Package Repositories

Software installation, packages, and dependencies

Package managers

Package files

Inspecting package files

Inspecting installed packages

Installing and removing package files

Package repositories and high-level package managers

Package repositories

High-level package managers

Searching for packages

System upgrade

Upgrading a system with YUM or DNF

Upgrading a system with APT

Summary

Further reading

9

Network Configuration and Debugging

Linux network stack

Network interfaces and addresses in Linux

Discovering physical network devices

Network interface names

Using the ip command

Discovering and inspecting logical links

Viewing and changing Ethernet link MAC addresses

Viewing and changing IP addresses

Routes and neighbor tables

ARP and NDP neighbor tables

Routes and route tables

NetworkManager

Distribution-specific configuration methods

Debian

Old Red Hat-based distributions

Network troubleshooting

Using ping

Using traceroute

Summary

10

Storage Management

Adding additional storage volumes

Formatting and partitioning storage devices

Formatting a newly created partition

Mounting and unmounting volumes

Updating the /etc/fstab file

Editing /etc/fstab file

Utilizing LVM

Getting started with LVM

Creating a format for logical disks

Deleting volumes with LVM

Summary

Part 3: Linux as a Part of a Larger System

11

Logging Configuration and Remote Logging

Logging configuration

How does log data get sent and gathered?

Checking rsyslog service on all servers

Configuring rsyslog for centralized logging

Sending logs to a centralized rsyslog server

Log rotation

Journald

DMESG

Understanding the ring buffer in Linux

Summary

12

Centralized Authentication

The AAA framework

Authentication mechanisms in Linux

Information lookup

Name Service Switch

Pluggable Authentication Modules

PAM configuration

Limitations of PAM

System Security Services Daemon

Active Directory authentication with Samba 4

Setting up the domain controller

Setting up the client

Summary

Further reading

13

High Availability

Types of redundancy and load balancing

Link layer redundancy

Network layer redundancy and load balancing

Transport layer load balancing with LVS

LVS load-balancing methods

Saving and restoring LVS configurations

Additional LVS options

Active/backup configurations and load balancing with Keepalived

Installing Keepalived

Basics of the VRRP protocol operation

Configuring VRRP

Configuring virtual servers

Application layer load balancing

Web application load balancing with HAProxy

Summary

Further reading

14

Automation with Chef

Overview of infrastructure automation

Benefits of automation in Linux

Introduction to Chef

What is Chef?

Key features of Chef

Overview of Chef’s architecture

Chef server

Chef server components

Cookbooks and recipes

Chef workstation

Development workflow

Managing environments and roles

Chef nodes

Node registration

Communication with the Chef server

Cookbook development

Cookbook structure and components

Writing recipes and resources

Managing infrastructure with Chef

Configuration management

Chef client-server interaction

Reporting and monitoring

Data synchronization

Benefits of automating infrastructure with Chef in Linux

Consistency and scalability

Reduced human error

Enhanced security

Challenges and best practices

Best practices for effective Chef implementation

Summary

15

Security Guidelines and Best Practices

Common threats and attack types

The motivation of attackers and the possible consequences

Information security properties and attacks on them

Keeping your system secure

Reducing the attack surface

Compartmentalization and privilege separation

Preventing credential theft and brute-force attacks

Reducing the risk of software vulnerabilities

Summary

Further reading

Index

Other Books You May Enjoy

Preface

Linux has become one of the most widely used operating systems in the world, powering everything from servers to smartphones. With its robust security, stability, and flexibility, Linux has become the go-to choice for many system administrators who seek a reliable and efficient platform to manage their organization’s IT infrastructure.

As a system administrator, you are responsible for managing the day-to-day operations of your organization’s IT infrastructure. This includes everything from setting up and configuring servers and maintaining network connectivity to troubleshooting issues when they arise. To do all of this effectively, you need a solid understanding of Linux and its various tools and utilities.

Throughout the book, you will find real-world examples and hands-on exercises that will help you build practical skills and gain confidence in your ability to manage Linux systems. You will also learn about the latest tools and techniques for managing large-scale Linux environments.

We hope that this book will serve as a valuable resource for you as you navigate the world of Linux system administration. Whether you are just starting out or looking to deepen your knowledge and skills, Linux for System Administrators will provide you with the knowledge and tools you need to succeed.

Who this book is for

Whether you are new to Linux or have been using it for years, this book provides a comprehensive overview of the operating system, its tools, and its best practices. The book covers everything from basic Linux concepts to more advanced topics, such as server virtualization, network configuration, and system security.

What this book covers

Chapter 1, Getting to Know Linux, provides an overview of the Linux operating system. It covers the basics of Linux’s history, its features, and how it differs from other operating systems such as Windows and macOS. The goal of this chapter is to provide you with a foundational understanding of Linux and its key concepts so that you can use the operating system effectively.

Chapter 2, The Shell and its Commands, provides examples of how to use specific commands and options, and it also covers more advanced topics such as shell scripting, piping and redirection, and using regular expressions. The goal of this chapter is to provide you with a solid understanding of the shell and its basic commands, enabling you to efficiently navigate and manage your Linux or Unix-like systems from the command line.

Chapter 3, The Linux Filesystem, covers the structure and organization of the filesystem used by the system. The chapter begins with an overview of the filesystem hierarchy, including the root directory and its subdirectories, such as /bin, /etc, /home, /usr, and /var. It also covers the different types of files and directories found in the filesystem, including regular files, directories, symbolic links, and special files such as device files. It also discusses file permissions and ownership, and how to use commands such as chmod and chown to modify them.

Chapter 4, Processes and Process Control, begins with an overview of processes and their properties, including process IDs, parent process IDs, and process statuses. It then covers how to view and manage running processes, using tools such as ps, top, and kill. Understanding these concepts can help system administrators optimize system performance and troubleshoot issues related to process management.

Chapter 5, Hardware Discovery, gives an overview of the different types of hardware components found in a typical computer system, including processors, memory, storage devices, and input/output devices. It also covers how the operating system detects and identifies these components, using tools such as dmesg, lspci, and lsusb.

Chapter 6, Basic System Settings, covers the configuration of basic system settings that affect a system’s behavior and performance. This chapter is essential for system administrators and users who need to customize the system to meet their specific requirements.

Chapter 7, User and Group Management, begins with an overview of user accounts and groups and their properties, including user IDs, group IDs, home directories, and shell settings. It then covers how to create and manage user accounts and groups, using tools such as useradd, usermod, groupadd, and groupmod.

Chapter 8, Software Installation and Package Repositories, starts with the installation and management of software packages on the system. This chapter is essential for system administrators and users who need to install, upgrade, and manage software packages to meet their requirements.

Chapter 9, Network Configuration and Troubleshooting, begins with an overview of network configuration and networking tools available on the system, such as ifconfig, ip, and netstat. It then covers how to configure network interfaces, assign IP addresses and netmasks, and configure network routing. This chapter is essential for system administrators and users who need to set up and maintain network connectivity and services.

Chapter 10, Storage Management, gives an overview of storage devices and filesystems and their properties, such as device names, device IDs, and mount points. It then covers how to create, manage, and mount filesystems, using tools such as fdisk, mkfs, and mount. Other topics covered include managing storage devices, such as partitioning and formatting disks, and managing Logical Volume Manager (LVM) volumes. Understanding these concepts and how to manage them is essential to ensure reliable and secure storage infrastructure. This chapter is essential for system administrators and users who need to manage storage resources, such as hard drives, solid-state drives, and network-attached storage.

Chapter 11, Logging Configuration and Remote Logging, includes configuring log forwarding and aggregation, setting up centralized logging servers, and analyzing system logs. Understanding these concepts and how to manage them is essential to ensure reliable and secure system logging infrastructure.

Chapter 12, Centralized Authentication, begins with an overview of authentication and authorization mechanisms available on the system, such as local password files, Lightweight Directory Access Protocol (LDAP), and Active Directory. It then covers how to configure and manage user authentication using tools such as Pluggable Authentication Module (PAM) and Name Service Switch (NSS). It also covers the configuration and management of user authentication and authorization on the system. This chapter is essential for system administrators who need to manage user access and privileges across multiple systems.

Chapter 13, High Availability, includes configuring and managing cluster resources, such as IP addresses, network interfaces, and shared storage devices; configuring and managing cluster services, such as web servers, databases, and email servers; and monitoring and troubleshooting cluster operations. Understanding these concepts and how to manage them is essential to ensure the high availability and reliability of critical applications and services.

Chapter 14, Automation with Chef, gives an overview of infrastructure automation and configuration management concepts, such as idempotence, the declarative approach, and the Infrastructure as Code (IaC) paradigm. It then covers how to use Chef to automate the configuration and management of systems, including nodes, cookbooks, recipes, and resources.

Chapter 15, Security Guidelines and Best Practices, covers how to implement security measures and best practices.

To get the most out of this book

To follow along the topics covered in this book, you just need a Linux VM or a Linux machine.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “For example, the configuration file directory for the Apache HTTP server is /etc/httpd on Red Hat Linux derivatives, but /etc/apache2 on Debian derivatives.”

A block of code is set as follows:

global_defs {     notification_email {         [email protected]         [email protected]     }     notification_email_from [email protected]     smtp_server 203.0.113.100     smtp_connect_timeout 30 }

Any command-line input or output is written as follows:

$ echo '#!/bin/bash' >> hello.sh $ echo 'echo "hello world"' >> hello.sh $ chmod +x ./hello.sh $ ./hello.sh hello world

Bold: Indicates a new term, an important word, or words that you see on screen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “The other is Red Hat Package Manager (RPM), which is used with the rpm utility and is developed by Red Hat.”

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Linux for System Administrators, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content..

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781803247946

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

Part 1: Linux Basics

The highest priority for a beginner systems administrator is to learn what the operating system consists of and how to interact with it. In the first part of this book, you will learn a brief history of Linux, how Linux-based systems are used in the real world, and what the relationship between the Linux kernel and the wider open source software ecosystem is. By the end of this part, you will be able to use basic commands to navigate the system and manage files, processes, and hardware devices.

This part has the following chapters:

Chapter 1, Getting to Know LinuxChapter 2, The Shell and its CommandsChapter 3, The Linux FilesystemChapter 4, Processes and Process ControlChapter 5, Hardware Discovery

1

Getting to Know Linux

Linux is a family of operating systems based on the same kernel. Since it’s a family of independently developed systems that have different design principles, goals, and implementation details, it’s important to understand what makes that very situation possible and how those systems are structured. In this chapter, we will discuss the concept of Linux distributions and open source software licensing and see how Linux-based systems are used. We will cover the following topics:

The structure of a Linux systemOpen source software licensesLinux usage in the modern world

The structure of a Linux system

Linux and its multiple distributions often seem complicated for beginners. To clarify this, let’s examine the structure and evolution of operating systems in general.

The Linux kernel and Linux-based operating systems

When people say Linux, they may mean different things. In the narrow sense, Linux is an operating system kernel that was created in the early 90s by Linus Torvalds and is now developed and maintained by a large international community. However, when people say they are using Linux, they usually mean a family of operating systems that use that kernel and usually (but not always) a set of system libraries and utilities created by the GNU project, which is why some insist that such systems should be referred to as GNU/Linux instead.

Note

The GNU project is a free software project that was launched in 1983 by Richard Stallman. His goal was to create a complete Unix-like operating system composed entirely of free software. GNU stands for GNU’s Not Unix, which reflects the project’s goal of creating a free software alternative to the proprietary Unix operating system.

To fully understand how that unusual situation became possible, let’s briefly discuss the history of operating systems.

Kernel versus user space

The earliest computers had very low computational power, so they would only have one program in their memory at a time, and that program had complete control over the hardware. As computing power increased, it became feasible to have multiple users use the same computer at the same time and run multiple programs – an idea known as time-sharing or multitasking. Shared computers would run a program known as a supervisor that would allocate resources to end user programs. A set of supervisor programs and system utilities became known as an operating system. The earliest time-sharing systems used cooperative multitasking, where programs were expected to transfer control back to the supervisor on their own. However, if a programming mistake made a program run into an endless loop or write data to a wrong memory address, such a program could cause the entire computer to hang or corrupt the memory of another program, including the supervisor.

To make multitasking more reliable, newer generations of hardware introduced protection mechanisms that allowed a supervisor program to take control of the CPU back from end user programs and forcibly terminate programs that tried to write something to memory that belonged to other programs or the supervisor itself.

That brought a separation between the operating system kernel and user space programs. End user programs physically couldn’t control the hardware directly anymore, and neither could they access memory that wasn’t explicitly allocated to them. Those privileges were reserved for the kernel – the code that includes a process scheduler (serving the same purpose as old supervisor programs) and device drivers.

Inside a single program, programmers are free to organize their code as they see fit. However, when multiple independently developed components need to work together, there needs to be a well-defined interface between them. Since no one writes directly in machine code anymore, for modern systems, this means two interfaces: the Application Programming Interface (API) and the Application Binary Interface (ABI). The API is for programmers who write source code and define function names they can call and parameter lists for those functions. After compilation, such function calls are translated into executable code that loads parameters into the correct places in memory and transfers control to the code to be called – where to load those parameters and how to transfer control is defined by the ABI.

Interfaces between user space programs and libraries are heavily influenced by the programming language they are written in.

On the contrary, interfaces between kernels and user space programs look more similar to hardware interfaces. They are completely independent of the programming language and use software interrupts or dedicated system call CPU instructions rather than the function calls familiar to application programmers.

Note

A system call in Linux is a mechanism that allows user-level processes to request services from the kernel, which is the core of the operating system. These services can access hardware devices, manage processes and threads, allocate memory, and perform other low-level tasks that require privileged access.

Those interfaces are also very low-level: for example, if you want to use the write() system call to print a string to standard output, you must always specify how many bytes to write – it has no concept of a string variable or a convention for determining its length.

For this reason, operating systems include standard libraries for one or more programming languages, which provide an abstraction layer and a stable API for end user programs.

Most operating systems have the kernel, the standard libraries for programming languages, and often the basic system utilities developed by a single group of people in close collaboration, and all those components are versioned and distributed together. In that case, the kernel interface is usually treated as purely internal and isn’t guaranteed to remain stable.

The Linux kernel and the GNU project

Linux is unique in that it was developed to provide a replacement kernel for an existing user space part of an operating system. Linus Torvalds, the founder of the project, originally developed it to improve the functionality of MINIX – an intentionally simplified Unix-like operating system meant for instruction rather than production use. He’s since been using the GNU C compiler and user space programs from the GNU project – the project that Richard Stallman started with the goal to create a complete Unix-like operating system that would be free (as in freedom) and open source, and thus available for everyone to use, improve, and redistribute.

At the time, the GNU project had all the user space parts of an operating system, but not a usable kernel. There were other open source Unix projects, but they were derived from the BSD Unix code base, and in the early 90s, they were targets of lawsuits for alleged copyright infringement. The Linux kernel came at a perfect time since Linus Torvalds and various contributors developed it completely independently and published it under the same license as the GNU project software – the GNU General Public License (GPL). Due to this, a set of GNU software packages, plus the Linux kernel, became a possible basis for a completely open source operating system.

However, Linus Torvalds wasn’t a GNU project member, and the Linux kernel remained independent from the Free Software Foundation (FSF) – it just used a license that the FSF developed for the GNU project, but that any other person could also use, and many did.

Thus, to keep new Linux kernel versions useful together with the GNU C library and software that relied on that library, developers had to keep the kernel interface stable.

The GNU C library wasn’t developed to work with a specific kernel either – when that project started, there wasn’t a working GNU kernel, and GNU software was usually run on other Unix-like operating systems.

As a result, both Linux and the GNU software can be and still are used together and in different combinations. The GNU user space software set can also be used with the still-experimental GNU hard kernel, and other operating systems use it as system or add-on software. For example, Apple macOS used GNU Bash as its system shell for a long time, until it was replaced by zsh.

The stability guarantees of the Linux kernel interface make it attractive to use as a basis for custom operating systems that may be nothing like Unix – some of them just have a single program run on top of the kernel. People have also created alternative standard libraries for different programming languages to use with Linux, such as Musl and Bionic for the C programming language, which use more permissive licenses and facilitate static linking. But to understand those licensing differences, we need to discuss the concept of software licenses.

Open source software licenses

A software license is an agreement between a copyright holder and a recipient of the software. Modern copyright laws are designed to give authors complete control over the use and distribution of their work – copyright automatically exists from the moment a piece of work is fixed on any medium and no one can use or copy that work without explicit permission from the author. Thus, a license agreement is required to grant a user some of the permissions that are reserved for the author by default. Authors are free to specify any conditions, and many individuals and companies use that to restrict what users can do – for example, only permit non-commercial use. A license agreement is also normally made between an author or a copyright holder and a specific person.

However, in the late 1980s, programmers and lawyers came up with the idea to use authors’ unlimited control over their works to ensure that anyone can use, distribute, and modify them rather than prevent that. They introduced public licenses, which grant permissions to everyone rather than just people who signed or otherwise accepted the agreement, and wrote several reusable license agreements that anyone could apply to their software. That concept became known as copyleft – a reversal of copyright. Those licenses became known as open source licenses because they explicitly permit the distribution and modification of software source code.

All the classic licenses were born in that period: the MIT license, the BSD license, the GNU GPL, and the GNU Lesser/Library General Public License (LGPL). None of those licenses limit users’ rights to use software distributed under them. Conditions, if any, apply only to the distribution and modification of executables and source code.

Permissive and copyleft licenses

When it comes to distribution, two schools of thought differ in their approach to distribution conditions.

Proponents of permissive licenses believe that recipients of software must have absolute freedom to do anything with it, even to incorporate it into other software that isn’t open source or to create closed source derivatives. The MIT and BSD licenses are typical examples of permissive open source licenses.

Proponents of copyleft believe that it’s important to protect open source software from attempts to appropriate the work of its authors and create a closed source derivative. The GNU GPL is the purest example of this – if anyone distributes executables of programs under the GPL or programs that link with libraries under the GPL, they must also distribute the source code of that program under that license. This is the most radical approach and is known as strong copyleft.

Licenses that allow you to link libraries to programs under any other license but require library code modifications to be under the same license are known as weak copyleft licenses. The most widely used example is the GNU LGPL.

Patent grant, tivoization, and SaaS concerns

The GNU GPL was created as a response to the rise of proprietary software distributed without source code, which prevented end users from improving it and sharing their improvements. However, the software industry is evolving, and new trends are appearing that some see as threats to the existence or financial sustainability of open source software.

One such threat is patent trolling – the use of software patents (in jurisdictions where they exist) in bad faith. As a response to it, some newer licenses and new versions of old licenses, such as the Apache license and the GNU GPLv3, introduced a patent grant clause. Such a clause prevents contributors to the source code of software from making patent claims against its users. If they make such legal threats, their licenses are revoked.

A more controversial point of the GPLv3 is its attempts to protect users’ rights to run modified versions on their hardware. The practice of preventing hardware from running custom software through digital signatures and similar mechanisms is sometimes called tivoization, after a Linux-based digital video recorder named TiVo that was an early example of such lock-in. While some projects supported the idea to prevent it, for others, the GPLv3 clause was a reason not to switch from GPLv2 – the Linux kernel is among those projects that stayed at the old GPL version.

Finally, all classic licenses were written in a time when all software was deployed on-premises, while in the modern world, a lot of software is delivered over a network and its executables aren’t accessible to end users – an approach known as Software-as-a-Service (SaaS). Since the GPL says that every recipient of a binary executable is entitled to receive its source code, it does not apply to SaaS since the user never receives any executables. This allows vendors to create modified versions of the software under the GPL without sharing their improvements with the community. Several licenses were developed in response to that trend, such as the Affero GPL.

In the last few years, big technology companies that provide hosted versions of open source software started to be seen as undermining project maintainers’ ability to earn money from services since it’s very difficult to compete on price with effective monopolies. In response, some projects started switching to licenses that have restrictions on usage, which many argue are no longer open source licenses, even though the source code is still available. The future of such licenses is an open question.

Linux distributions

The fact that software under open source licenses is free to modify and distribute made it possible to assemble complete operating systems with kernels, system libraries, and utilities, as well as a selection of application software. Since open source licenses have no restrictions on usage, there is no need to make the user accept a license agreement for each component.

In the early days of Linux, setting up a usable Linux environment was a complicated and tedious endeavor. To make that process simpler, Linux enthusiasts started preparing the first distributions – sets of packages and scripts to automate their installation. Many of those early distributions, such as Softlanding Linux System and Yggdrasil, are now defunct, but some are still maintained – Slackware Linux is a prominent example.

Package managers and package repositories

Early distributions had a relatively humble goal, which was to provide users with a working barebones system that they could then install their application software on. However, later distributions set out to rethink the process of software installations. The number of open source software projects was growing, and CD drives and internet connections were also becoming more affordable, so it was feasible to include much more software in a distribution than ever before.

However, many applications depend on shared libraries or other applications. Traditionally, installation packages would either include all dependencies or leave that dependency management to the user. Since distribution is managed by a single group of maintainers, developers came up with the idea of sharing dependencies between all packages that need them and automatically installing all dependencies when a user requested the installation of a package. That gave rise to package managers and package repositories – collections of files in a special format, including compiled binaries and metadata such as the package version and its dependencies.

The two most popular package formats and package managers that work with them were developed in the mid-90s and are still in use today. One is the DEB format, which is used with the dpkg utility, developed by Debian. The other is Red Hat Package Manager (RPM), which is used with the rpm utility and is developed by Red Hat.

The dpkg and rpm tools are responsible for installing package files on local machines. To install a package, the user needs to retrieve the package itself and all packages it depends on. To automate that process, distributions developed high-level package managers that can automatically download packages from online repositories, check for updates, search metadata, and more. Those high-level package managers usually rely on low-level ones to manage the installation. Debian’s Advanced Packaging Tool (APT) usually works with DEB packages, although it’s technically possible to use it with RPM. High-level package managers that primarily use the RPM format are more numerous: YUM and DNF, which are maintained by Red Hat, zypper from openSUSE, and urpmi, which is developed for the now-defunct Mandrake Linux and still used by its forks.

Many of the currently existing distributions have either been actively maintained since the 90s or are forks that split off at different points in time. For example, Ubuntu Linux was forked from Debian GNU/Linux in the early 2000s, while Rocky Linux is a Red Hat Enterprise Linux derivative that started in 2021.

However, completely independent distributions also appear once in a while. Some of them are special-purpose systems that have requirements that classic general-purpose distributions cannot fulfill. For example, OpenWrt is a Linux-based system for consumer routers, originally developed for the Linksys WRT-54G device, hence the name. Such devices often have just a few megabytes of flash drive space, so operating systems for them have to be very compact, and they also have to use special filesystems such as JFFS that are designed for NAND flash drives.

Other independent distributions experiment with different package management and installation principles. For example, NixOS and GNU Guix use an approach that allows the user to revert system updates if anything goes wrong with new package versions.

In this book, we will focus on Debian/Ubuntu and Red Hat-based systems because they have been the most popular distributions for a long time and remain popular.

Differences between distributions

The differences between distributions do not stop at package managers. Configuration file locations may differ, and default configurations for the same packages may also differ dramatically. For example, the configuration file directory for the Apache HTTP server is /etc/httpd on Red Hat Linux derivatives, but /etc/apache2 on Debian derivatives.

Some distributions also use high-level configuration tools and you may take them into account.

The choice of software and its ease of installation may also differ. Debian, Fedora, and many other distributions leave the choice of a desktop environment to the user and make it easy to install multiple different desktop environments on the same system so that you can switch between GNOME3, KDE, MATE, or anything else for different login sessions. In contrast, the Ubuntu family of distributions includes multiple flavors for different desktop environments and expects that if you don’t like its default choice (the Unity desktop environment), you should use Kubuntu for KDE, for example, rather than the default Ubuntu. Finally, some distributions come with a custom desktop environment and don’t support anything else, such as elementary OS.

However, experienced Linux users can usually find their way around any distribution.

Linux usage in the modern world

The open source nature of the Linux kernel and its support for multiple hardware architectures made it a very popular choice for custom operating systems, while general-purpose Linux distributions also found wide use in every niche where proprietary Unix systems were used before.

The most popular Linux-based operating system in the world is Android. While most Android applications are written for a custom runtime and never use any functionality of the Linux kernel directly, it’s still a Linux distribution.

Network devices are usually managed through a web GUI or a custom command-line interface, but they still often have Linux under the hood. This applies to consumer-grade Wi-Fi routers, as well as high-performance enterprise and data center routers and switches alike.

General-purpose Linux distributions are also everywhere. Thanks to built-in support for running virtual machines (through Xen or KVM hypervisors), Linux powers the largest cloud computing platforms, including Amazon EC2, Google Cloud Platform, and DigitalOcean. A lot of guest systems are also Linux machines running web servers, database systems, and many other applications.

Linux is also widely used in high-performance computing: all of the most powerful supercomputers in the world are now running Linux on their control and I/O nodes.

Last but not least, the author of this chapter typed these words on a Linux desktop.

Summary

A Linux distribution is a complete operating system that includes the Linux kernel and a set of libraries and programs developed by various people and companies. The Linux kernel and core system libraries are not developed by the same group. Instead, the Linux kernel provides a stable ABI that allows anyone to develop a standard library for a programming language to run on top of it.

Open source licenses come with different conditions, but they all allow anyone to use the software for any purpose and distribute its copies, and that’s what makes the existence of Linux distributions possible.

Different distributions use different approaches to package management and configuration, but experienced users can learn how to use a new distribution fairly quickly if they know the fundamentals. These days, Linux can be found everywhere, from mobile phones to the most powerful supercomputers.

In the next chapter, we will learn about the various shells available on Linux systems, as well as basic commands.

2

The Shell and Its Commands

We’ll be doing a lot of things in the shell, such as installing packages, making new users, creating directories, modifying permissions of files, and so on. These will be the basics but will be your first interaction with the shell to understand what is happening behind it and to get more confident. In order to improve our effectiveness with the shell, we’ll be devoting an entire chapter to it this time around.

In this chapter, we will cover the following topics:

A basic definition of a shell in order to understand how it works, including an overview of its features and a description of the most common shellsHow to use basic commands in order to get familiarized with Linux (in this chapter, CentOS version 8 will be used)Basic notions about how to use commands to change the ownership of files and directories

What is a shell?

Computer software known as a shell makes an operating system’s services accessible to users or other programs.

A shell is a program that receives commands and sends them to the operating system for processing, to put it simply. In an interactive session, the user has the option of typing commands from the keyboard, or they can be written in a shell script that can be reused. On a Unix-type system such as Linux in the past, it was the sole user interface (UI) accessible. Today, in addition to command-line interfaces (CLIs) such as shells, we also have graphical UIs (GUIs).

The fundamental capability of shells is the ability to launch command-line programs that are already installed on the system. They also offer built-ins and scripting control structures such as conditionals and loops. Each shell has its own way of doing that. Some shells still support the Bourne shell, one of the original shells that was created for an early Unix system by a programmer named Steve Bourne and later standardized in the Portable Operating System Interface (POSIX) standard. Other projects, such as csh/tcsh, zsh, and fish, purposefully utilize a different syntax.

In order to use command-line shells, a user must be knowledgeable about commands, their calling syntax, and the fundamentals of the shell’s specific scripting language.

A Linux user can utilize a variety of shells, including the following:

sh: A POSIX-compatible Bourne shell. In modern distros, it’s usually just Bourne again shell (Bash) running in compatibility mode.csh/tcsh: These come from the Berkeley Software Distribution (BSD) Unix system family but are also available on Linux; their scripting syntax is similar to that of C, and they are incompatible with the Bourne shell.ksh: A Bourne shell derivative that was once very popular.bash: Bash is the most common Linux shell and was created for the GNU project.zsh and fish: These are highly customizable and feature-rich shells that are intentionally different from sh derivatives and require learning, but have large communities of enthusiasts.

They all share similar properties, but each has its own unique attributes.

In this book, we will assume you are using Bash since it’s the default in most Linux distributions.

The Unix shell and Bash command language were both developed by Brian Fox for the GNU project. These were intended to serve as free software replacements for the Bourne shell. Since its introduction in 1989, it has remained the default login shell for the vast majority of Linux distributions. Linus Torvalds ported Bash and the GNU Compiler Collection (GCC) to Linux as one of the initial applications.

Bash has the following features:

The shell will check to see whether a command is built in before searching through a list of directories to locate the program if not. This set is known as the search path. By running the echo $PATH command in Bash, you can view it. The home directory and its subdirectory are included in the search path in addition to the current directory. You are able to create your own programs and call them up just by inputting their names. No matter which directory you are now in, a program such as this will be found and launched if it is stored in the bin directory. We will find out more about the Linux directory structure in Chapter 3, The Linux Filesystem.As with other Linux programs, the shell has a current directory linked to it. When looking for files, Linux-based programs begin in the current directory. To move the current directory to another location in the Linux filesystem, use the cd shell command. The current working directory is typically visible in the command prompt of modern shells. To check the version of your shell, run the echo $SHELL command. You will get an output such as /bin/bash.A command is executed by designating it. The majority of Linux commands are just programs that the shell runs. For instance, the following ls command scans the current directory and lists the names of its files: ls -la.Commands frequently have argument strings that could, for example, be filenames. For instance, the following command switches to the tmp directory in your home directory. The shell interprets the tilde character as your home directory:cd ~/tmpMultiple arguments are required for some commands. The copy command, for instance, requires two arguments: the file to copy and its destination. This is demonstrated as follows by copying file1 to a new file, file2:cp file1 file2The flag or option argument strings for some commands typically start with -. The flags change how the invoked application behaves. When the following command is used, ls outputs a lengthy listing of files arranged by creation time:ls -ltWildcards will be expanded by the shell to match filenames in the current directory. For example, to display a directory listing of files named anything.sh, type the following:ls -l *.shStandard input (stdin) and standard output (stdout) are concepts that are followed by the