Demystifying Cryptography with OpenSSL 3.0 E-Book

Alexei Khlebnikov

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 464

Veröffentlichungsjahr: 2022

Beschreibung

Security and networking are essential features of software today. The modern internet is full of worms, Trojan horses, men-in-the-middle, and other threats. This is why maintaining security is more important than ever.
OpenSSL is one of the most widely used and essential open source projects on the internet for this purpose. If you are a software developer, system administrator, network security engineer, or DevOps specialist, you’ve probably stumbled upon this toolset in the past – but how do you make the most out of it? With the help of this book, you will learn the most important features of OpenSSL, and gain insight into its full potential.
This book contains step-by-step explanations of essential cryptography and network security concepts, as well as practical examples illustrating the usage of those concepts. You’ll start by learning the basics, such as how to perform symmetric encryption and calculate message digests. Next, you will discover more about cryptography: MAC and HMAC, public and private keys, and digital signatures. As you progress, you will explore best practices for using X.509 certificates, public key infrastructure, and TLS connections.
By the end of this book, you’ll be able to use the most popular features of OpenSSL, allowing you to implement cryptography and TLS in your applications and network infrastructure.

Leseprobe

Demystifying Cryptography with OpenSSL 3.0

Discover the best techniques to enhance your network security with OpenSSL 3.0

Alexei Khlebnikov

BIRMINGHAM—MUMBAI

Demystifying Cryptography with OpenSSL 3.0

Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author nor Packt Publishing or its dealers and distributors will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Associate Group Product Manager: Mohd Riyan Khan

Publishing Product Manager: Shrilekha Malpani

Senior Content Development Editor: Adrija Mitra

Technical Editor: Nithik Cheruvakodan

Copy Editor: Safis Editing

Book Project Manager: Kirti Pisat

Proofreader: Safis Editing

Indexer: Sejal Dsilva

Production Designer: Alishon Mendonca

Marketing Coordinator: Ankita Bhonsle

First published: October 2022

Production reference: 1071022

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

978-1-80056-034-5

www.packt.com

To my beloved mother, Tatyana Khlebnikova, who, through a lot of effort, care, love, and support, brought me up to the point where I could continue my own development further.

– Alexei Khlebnikov

Foreword

Having been a coder for more than three decades, I’ve come across fellow techies who could write a book, or even 10, about common but intricate technologies many times. However, very rarely have I come across someone who can explore topics as deeply as Alexei Khlebnikov.

I’ve had the pleasure of working with him and being his friend for several years now and I am looking forward to really diving into this book. I would certainly say it’s a subject that deserves our greatest attention.

In this book, we will be taken on a journey through the basics of OpenSSL, general cryptography, cryptography modes, the “joys” of certificates, and the making of TLS connections. All in great detail if I understand Alexei correctly, which I am very certain I do.

This is an important book, looking closely at technologies we take for granted and that are used basically everywhere to secure our online presence.

There are practical examples and step-by-step explanations of essential concepts to help you along. By the end of the book, you’ll be able to use the most popular features of OpenSSL in your products, whether web or desktop.

The book is certainly interesting for the doers, but also for managers and others who think security is important but lack knowledge about it. Don't worry – an in-depth understanding of mathematics is not needed to read this book and learn from it.

In my view as a lifelong techie, learning new things or maybe diving deeper into topics you have some starting knowledge on is rewarding and helpful and it keeps us all on our toes!

Take it away, Alexei!

– Jarle Adolfsen

Serial entrepreneur, CTO at bspoke, former CTO at Link Mobility, and a pioneer in computer graphics in the late 1980s and early 1990s

Contributors

About the author

Alexei Khlebnikov has more than 20 years of professional experience in IT where he has worked in a host of different roles – software developer, system administrator, DevOps engineer, technical leader, architect, and project manager. During these years, Alexei has worked with many technologies – security, artificial intelligence, web development, embedded, mobile, and robotics. Among other companies, Alexei worked for Opera Software on the famous Opera internet browser. Alexei has always been interested in security. He was one of the maintainers of the security-related Opera browser modules, responsible for cryptography, SSL/TLS, and integration with OpenSSL. He was also a member of the security architect group, responsible for the security of the Opera browser. Now, Alexei lives in Oslo, Norway, and works as a senior consultant for bspoke AS. He is also the leader of the architect group at his current employer.

First and foremost, I want to thank my beloved wife, Larisa, and son, Dmitry, for their continued love and support, and for supporting me while I was writing this book and spending less time with them. I also want to thank all the editors, managers, and other people at Packt Publishing who worked on this book, as well as the technical reviewer, Kris. Their help and valuable advice helped me improve the book to the benefit of its readers.

About the reviewer

Krzysztof Kwiatkowski is a cryptography engineer who focuses on problems at the intersection of cryptographic research and implementation. He holds an MSc degree in mathematics with a specialization in computational methods. With a career spanning over 15 years, Kris has worked on a variety of topics related to cryptography, communication, and software security from compact embedded to large distributed systems. Currently, he is concentrating on the implementation of modern, quantum-safe cryptographic schemes, and helping organizations to migrate towards them.

I’d like to thank my wonderful and loving family, who understand my busy schedule and always stand by my side.

Table of Contents

Prefacexv

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Share Your Thoughts

Part 1: Introduction

1

OpenSSL and Other SSL/TLS Libraries3

What is OpenSSL?

The history of OpenSSL

What’s new in OpenSSL 3.0?

Comparing OpenSSL with GnuTLS

Comparing OpenSSL with NSS

Comparing OpenSSL with Botan

Comparing OpenSSL with lightweight TLS libraries

Comparing OpenSSL with LibreSSL

Comparing OpenSSL with BoringSSL

Summary

Part 2: Symmetric Cryptography

2

Symmetric Encryption and Decryption15

Technical requirements

Understanding symmetric encryption

An overview of the symmetric ciphers supported by OpenSSL

Comparing block ciphers and stream ciphers

Understanding symmetric cipher security

How many bits of security is enough?

Reviewing the AES cipher

Reviewing DES and 3DES ciphers

Reviewing the RC4 cipher

Reviewing the ChaCha20 cipher

Reviewing other symmetric ciphers supported by OpenSSL

Block cipher modes of operation

Reviewing the Electronic Code Book mode

Reviewing CBC mode

Reviewing CTR mode

Reviewing GCM

Reviewing AES-GCM-SIV

Other block cipher operation modes

Choosing the block cipher operation mode

Padding for block ciphers

How to generate a symmetric encryption key

Downloading and installing OpenSSL

How to encrypt and decrypt with AES on the command line

Initializing and uninitializing OpenSSL library

How to compile and link with OpenSSL

How to encrypt with AES programmatically

Implementing the encryption program

Running the encrypt program

How to decrypt with AES programmatically

Implementing the decrypt program

Running the decrypt program

Summary

3

Message Digests55

Technical requirements

What are message digests and cryptographic hash functions?

Why are message digests needed?

Data integrity verification

Basis for HMAC

Digital signatures

Network protocols

Password verification

Content identifier

Blockchain and cryptocurrencies

Proof-of-work

Assessing the security of cryptographic hash functions

Overview of the cryptographic hash functions supported by OpenSSL

Reviewing the SHA-2 family of hash functions

Reviewing the SHA-3 family of hash functions

Reviewing the SHA-1 and SHA-0 hash functions

Reviewing the MD family of hash functions

Reviewing the BLAKE2 family of hash functions

Reviewing less popular hash functions supported by OpenSSL

Which cryptographic hash function should you choose?

How to calculate a message digest on the command line

How to calculate the message digest programmatically

Implementing the digest program

Running the digest program

Summary

4

MAC and HMAC71

Technical requirements

What is a MAC?

Understanding MAC function security

HMAC – a hash-based MAC

MAC, encryption, and the Cryptographic Doom Principle

How to calculate HMAC on the command line

How to calculate HMAC programmatically

Implementing the hmac program

Running the hmac program

Summary

5

Derivation of an Encryption Key from a Password83

Technical requirements

Understanding the differences between a password and an encryption key

What is a key derivation function?

Overview of key derivation functions supported by OpenSSL

Deriving a key from a password on the command line

Deriving a key from a password programmatically

Implementing the kdf program

Running the kdf program

Summary

Part 3: Asymmetric Cryptography and Certificates

6

Asymmetric Encryption and Decryption95

Technical requirements

Understanding asymmetric encryption

Understanding a Man in the Middle attack

Meeting in person

Verifying a key fingerprint over the phone

Key splitting

Signing the key by a trusted third party

What kind of asymmetric encryption is available in OpenSSL?

Understanding a session key

Understanding RSA security

How to generate an RSA keypair

How to encrypt and decrypt with RSA on the command line

How to encrypt with RSA programmatically

Implementing the rsa-encrypt program

Running the rsa-encrypt program

Understanding the OpenSSL error queue

How to decrypt with RSA programmatically

Implementing the rsa-decrypt program

Running the rsa-decrypt program

Summary

7

Digital Signatures and Their Verification119

Technical requirements

Understanding digital signatures

Difference between digital signatures and MACs

Overview of digital signature algorithms supported by OpenSSL

Reviewing RSA

Reviewing DSA

Reviewing ECDSA

Reviewing EdDSA

Reviewing SM2

Which digital signature algorithm should you choose?

How to generate an elliptic curve keypair

How to sign and verify a signature on the command line

How to sign programmatically

Implementing the ec-sign program

Running the ec-sign program

How to verify a signature programmatically

Implementing the ec-verify program

Running the ec-verify program

Summary

8

X.509 Certificates and PKI137

Technical requirements

What is an X.509 certificate?

Understanding certificate signing chains

How are X.509 certificates issued?

What are X509v3 extensions?

Understanding X.509 Public Key Infrastructure

How to generate a self-signed certificate

How to generate a non-self-signed certificate

How to verify a certificate on the command line

How to verify a certificate programmatically

Implementing the x509-verify program

Running the x509-verify program

Summary

Part 4: TLS Connections and Secure Communication

9

Establishing TLS Connections and Sending Data over Them163

Technical requirements

Understanding the TLS protocol

Understanding a TLS handshake

What happens after the TLS handshake?

The history of the TLS protocol

Establishing a TLS client connection on the command line

Preparing certificates for a TLS server connection

Accepting a TLS server connection on the command line

Understanding OpenSSL BIOs

Establishing a TLS client connection programmatically

Implementing the tls-client program

Running the tls-client program

Accepting a TLS server connection programmatically

Implementing the tls-server program

Running the tls-server program

Summary

10

Using X.509 Certificates in TLS193

Technical requirements

Custom verification of peer certificates in C programs

Registering the verification callback

Implementing the verification callback

Running the program

Using Certificate Revocation Lists in C programs

Registering the CRL lookup callback

Implementing the CRL lookup callback

Implementing the function for downloading a CRL from a distribution point

Implementing the function for downloading a CRL from an HTTP URL

Running the program

Using the Online Certificate Status Protocol

Understanding the Online Certificate Status Protocol

Using OCSP on the command line

Using OCSP in C programs

Using TLS client certificates

Generating TLS client certificates

Packaging client certificates into PKCS #12 container files

Requesting and verifying a TLS client certificate on the server side programmatically

Establishing a TLS client connection with a client certificate programmatically

Summary

11

Special Usages of TLS243

Technical requirements

Understanding TLS certificate pinning

Using TLS certificate pinning

Changing the run_tls_client() function

Implementing the cert_verify_callback() function

Running the tls-cert-pinning program

Understanding blocking and non-blocking sockets

Using TLS on non-blocking sockets

Changing the run_tls_client() function

Running the tls-client-non-blocking program

Understanding TLS on non-standard sockets

Using TLS on non-standard sockets

Implementing the service_bios() function

Reimplementing the run_tls_client() function

Running the tls-client-memory-bio program

Summary

Part 5: Running a Mini-CA

12

Running a Mini-CA279

Technical requirements

Understanding the openssl ca subcommand

Generating a root CA certificate

Generating an intermediate CA certificate

Generating a certificate for a web server

Generating a certificate for a web and email client

Revoking certificates and generating CRLs

Providing certificate revocation status via OCSP

Summary

Index305

Other Books You May Enjoy318

Preface

Security and networking are essential features of software today. The modern internet is full of worms, Trojan horses, men-in-the-middle, and other threats. This is why maintaining security is more important than ever.

OpenSSL is one of the most widely used and essential open-source projects on the internet for this purpose. If you are a software developer, system administrator, network security engineer, or DevOps specialist, you’ve probably stumbled upon this toolset in the past – but how do you make the most out of it? With the help of this book, you will learn the most important features of OpenSSL, and gain insight into its full potential.

This book contains step-by-step explanations of essential cryptography and network security concepts, as well as practical examples illustrating usage of those concepts. You’ll start by learning the basics such as how to perform symmetric encryption and calculate message digests. Next, you will discover more about cryptography: MAC and HMAC, public and private keys, and digital signatures. As you progress, you will explore best practices for using X.509 certificates, public key infrastructure, and TLS connections.

By the end of this book, you’ll be able to use the most popular features of OpenSSL, allowing you to implement cryptography and TLS in your applications and network infrastructure.

Who this book is for

This book is for software developers, system administrators, DevOps specialists, network security engineers, and analysts, or anyone who wants to keep their applications and infrastructure secure. Software developers will learn how to use the OpenSSL library to empower their software with cryptography and TLS. DevOps professionals and sysadmins will learn how to work with cryptographic keys and certificates on the command line, and how to set up a mini-CA for their organization. A basic understanding of security and networking is required.

What this book covers

Chapter 1, OpenSSL and Other SSL/TLS Libraries, will outline what OpenSSL is and what its strengths are and take a look into OpenSSL’s history and at what’s new in OpenSSL 3.0. We will also compare OpenSSL to other SSL/TLS libraries.

Chapter 2, Symmetric Encryption and Decryption, will cover the important concepts in symmetric encryption – ciphers, encryption modes, and padding. We will overview modern ciphers, encryption modes, and padding types and recommend which technology to use in which situation. Usage of these technologies will be illustrated by command-line and C code examples.

Chapter 3, Message Digests, will explore why message digests, also known as cryptographic hashes, are needed and where they are used. We will get an overview of modern cryptographic hash functions that calculate message digests and recommend which hash function to use in which situation. The calculation of message digests will be illustrated by command-line and C code examples.

Chapter 4, MAC and HMAC, will explain why Message Authentication Codes (MACs) are needed and where they are used. Since it’s a popular MAC type, Hash-based MAC (HMAC) will be discussed. We will also learn about how to combine HMAC with encryption and about the Cryptographic Doom Principle. The calculation of HMAC will be illustrated by a code example.

Chapter 5, Derivation of an Encryption Key from a Password, will show why a password itself cannot be used for encryption and why key derivation is needed. We will overview modern key derivation functions and recommend which one to use when. Then, encryption key derivation will be illustrated by command-line and C code examples.

Chapter 6, Asymmetric Encryption and Decryption, will unpack why asymmetric encryption is needed, how it works, and how private and public keys are used to achieve encryption and decryption. Encryption and decryption using RSA will be illustrated by command-line and C code examples.

Chapter 7, Digital Signatures and Their Verification, will clarify why digital signatures are needed and where they are used. We will overview modern digital signature algorithms, such as RSA, ECDSA, and EdDSA, and recommend which digital signature scheme to use in which situation. Digital signing and signature verification will be illustrated by command-line and C code examples.

Chapter 8, X.509 Certificates and PKI, will detail what X.509 certificates are, why they are needed, and where they are used. We will also explain how certificates sign other certificates and how certificate signing chains are formed, as well as what Public Key Infrastructure (PKI) is and how certificate verification is used to verify identities – for example, the identities of websites. The usage of the techniques mentioned will be illustrated by command-line and C code examples.

Chapter 9, Establishing TLS Connections and Sending Data over Them, will break down what the TLS protocol is, why it is needed, and why it is used so widely. We will also learn what the difference between SSL and TLS is. Then, we will learn how to establish and shut down a TLS connection, as well as how to send and receive data over TLS. Working with TLS will be illustrated by command-line and C code examples.

Chapter 10, Using X.509 Certificates in TLS, will elaborate on how to work with X.509 certificates in TLS and why certificates are important for TLS. We will also learn how to verify a remote certificate. Then, we will learn how to further check the certificate validity using a CRL and OCSP. Finally, we will learn how to use a client certificate. Working with certificates will be illustrated by command-line and C code examples.

Chapter 11, Special Usages of TLS, will look into special usages of TLS: TLS pinning, using non-blocking networking mode, and TLS connections over non-standard sockets or special networking layers using OpenSSL Basic Input-Output Objects (BIOs). The usage of the techniques mentioned will be illustrated by C code examples.

Chapter 12, Running a Mini-CA, will instruct you on how to run your own mini-CA in order to control certificates and build PKI into an organization. Running a mini-CA will be illustrated by example configuration files and commands.

To get the most out of this book

You will have to install OpenSSL on your computer in order to run the command-line and C code examples. If you haven’t installed it yet, Chapter 2, Symmetric Encryption and Decryption, will help you to do so. To build the C code examples, you will need a C11-compatible C compiler and a linker. You will have to install these development tools following their respective documentation. All the examples have been tested on Kubuntu Linux 22.04 using GNU C Compiler, GNU Linker (LD), and GNU Make from the Linux distribution mentioned. Other development tools, such as LLVM Clang or Microsoft Visual C++, should also be compatible with the code examples in this book.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to copying and pasting code.

While explanations of OpenSSL features and code examples are sometimes very detailed, the book is meant to provide guidance, not to replace the OpenSSL documentation. If you are wondering about details of OpenSSL functionality that are not covered by the book, feel free to consult the OpenSSL documentation, the OpenSSL source code, or just experiment with your own code using OpenSSL!

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Demystifying-Cryptography-with-OpenSSL-3. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots and diagrams used in this book. You can download it here: https://packt.link/c0WEO.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “SSH user public keys are pinned on the server in the authorized_keys file.”

A block of code is set as follows:

if (pinned_server_cert)     X509_free(pinned_server_cert); if (pinned_server_cert_file)     fclose(pinned_server_cert_file);

Any command-line input or output is written as follows:

$ ./tls-server 4433 server_keypair.pem server_cert.pem *** Listening on port 4433

Bold: Indicates a new term, an important word, or words that you see onscreen. For instance, words in menus or dialog boxes appear in bold. Here is an example: “Reduced maintenance because you don’t need to make a Certificate Signing Request (CSR) and communicate with a CA. You can even use a self-signed certificate.”

Tips or Important Notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [email protected] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you’ve read Demystifying Cryptography with OpenSSL 3, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Part 1: Introduction

Each journey starts with a first step. Our first step will be to learn about what OpenSSL is, take a look at its history, and learn about what’s new in OpenSSL 3. We will also review other SSL/TLS libraries that are competitors of OpenSSL.

This part contains the following chapter:

Chapter 1, OpenSSL and Other SSL/TLS Libraries

1

OpenSSL and Other SSL/TLS Libraries

Currently, there are several libraries available to developers who want to add support for cryptography or SSL/TLS features to their applications. This chapter will help you compare different libraries and learn about the strengths of OpenSSL. You will also learn a bit about OpenSSL’s history and what’s new in OpenSSL 3.0.

In this chapter, we are going to cover the following topics:

What is OpenSSL?The history of OpenSSLWhat’s new in OpenSSL 3.0?Comparing OpenSSL with GnuTLSComparing OpenSSL with NSSComparing OpenSSL with BotanComparing OpenSSL with lightweight TLS librariesComparing OpenSSL with LibreSSLComparing OpenSSL with BoringSSL

What is OpenSSL?

OpenSSL is an open source software toolkit that includes a cryptography and SSL/TLS library, as well as command-line utilities that use the library to provide some useful functionality on the command line, such as generating encryption keys and X.509 certificates. The main part of OpenSSL is its library, which means that OpenSSL is mainly useful for software developers. However, system administrators and DevOps specialists will also find OpenSSL’s command-line utilities very useful.

SSL stands for Secure Sockets Layer. It is a protocol designed to provide secure communications over insecure computer networks. An insecure computer network means a network where the transmitted data can be read or even changed by a malicious intermediate network node. An example of such an insecure network is the internet. Secure communication is where transmitted data cannot be read or changed. SSL achieves communication security by using symmetric and asymmetric cryptography. The SSL protocol was invented in 1995 by the Netscape Communications Corporation and was deprecated in 2015 in favor of its successor, the TLS protocol. TLS stands for Transport Layer Security.

The OpenSSL toolkit was created before the SSL protocol became deprecated, so it contains “SSL” instead of “TLS” in its name.

OpenSSL was historically licensed under the BSD-style license, but since version 3.0, it is licensed under Apache 2.0 license, which is also BSD-style. This license allows OpenSSL to be used in both open source and closed source applications.

OpenSSL supports a lot of cryptographic algorithms, among which are algorithms for symmetric and asymmetric encryption, digital signatures, message digests, and key exchange. OpenSSL supports X.509 certificates, SSL, TLS, and DTLS protocols, as well as other cryptography-related technologies that are less popular.

OpenSSL has been around for a while and during its development, it gained support for a lot of operating systems. OpenSSL was originally developed for Unix-like operating systems. Up until now, OpenSSL supports different variations of Unix, including GNU/Linux, BSDs, and old and new commercial Unixes, such as IBM AIX and macOS. OpenSSL also supports popular non-Unix operating systems such as Microsoft Windows, mobile OSes such as Android and iOS, and even old and exotic OSes such as MS-DOS and VMS.

Through many years of OpenSSL development, the library has received numerous optimizations, including assembly optimizations for the most popular CPU architectures, such as x86, x86_64, and ARM. OpenSSL is currently one of the fastest existing crypto and TLS libraries.

Because of its universality, support for a lot of algorithms and operating systems, and because of its speed, OpenSSL has become the de facto industry standard. OpenSSL is so popular that other TLS libraries make so-called OpenSSL compatibility layers so that those libraries can be used via OpenSSL application programming interfaces (APIs).

OpenSSL is quite a popular library but what did its path to the widespread adoption look like? Let’s find out by walking through the OpenSSL history.

The history of OpenSSL

The OpenSSL library is based on the SSLeay library by Eric Andrew Young. The eay in SSLeay stands for Eric Andrew Young. SSLeay’s development started in 1995 as an open source implementation of the SSL library. Back then, the NSS library was not available. Later, Tim Hudson joined the development team. But in 1998, both Eric and Tim were hired by the RSA Corporation and did not have time to develop SSLeay further.

SSLeay was forked as OpenSSL in 1998, which means that OpenSSL has become SSLeay’s successor. The initial founding members were Mark Cox, Ralf Engelschall, Stephen Henson, Ben Laurie, and Paul Sutton. The very first version of OpenSSL, numbered 0.9.1, was released on December 23, 1998, merely a week after Eric and Tim joined the RSA Corporation and effectively stopped working on SSLeay.

Over many years, a lot of people and companies contributed a lot of code and other work to OpenSSL. The list of contributing companies is impressive: Oracle, Siemens, Akamai, Red Hat, IBM, VMware, Intel, and Arm, among others.

Currently, OpenSSL development is managed by the OpenSSL Management Committee, which consists of seven members. The core development team, which has commit rights, consists of approximately 20 people. Only two people work full-time on OpenSSL. The other contributors either do so in their spare time or as a part of their work in the contributing companies.

The history of OpenSSL is interesting, but what does the future hold for it? Let’s find out about the latest changes in the toolkit.

What’s new in OpenSSL 3.0?

One major change in OpenSSL 3.0 is its license. A software project does not change its license very often during its lifetime. The OpenSSL project used its BSD-style open source license until version 3.0. Since version 3.0, it uses Apache License 2.0.

OpenSSL 3.0 is a release with big changes in the internal architecture of the library. The architectural changes are not finished and will be continued in OpenSSL 4.0. The concept of OpenSSL operation implementation providers was introduced. A provider is a unit of code that provides the implementation of cryptographic algorithms. The existing OpenSSL cryptography code will mostly be available via Default and Legacy providers. Engines are still supported in OpenSSL 3.0 but have been deprecated in favor of providers. Support for ENGINE API may be removed in OpenSSL 4.0. There is also support for third-party providers that allow independent developers to plug their cryptographic algorithms into OpenSSL.

Another interesting feature of OpenSSL 3.0 is Kernel TLS (KTLS). When using KTLS, an application can create a special TLS socket, similar to a TCP socket. OpenSSL then performs a TLS handshake and hands the negotiated encryption key and other data to the operating system kernel in the form of TLS socket options. Then, the actual data transmission in the TLS protocol is handled by the KTLS code. Such TLS offloading to the kernel can speed up data transmission on high-load systems where performance is important, especially when the kernel can use hardware acceleration for Advanced Encryption Standard (AES) and thus offload the main CPU. Of course, KTLS support is needed both in the TLS library and in the operating system kernel. At the time of writing, only the Linux and FreeBSD kernels support KTLS.

Some other notable changes in OpenSSL 3.0 include the following:

Support for the Certificate Management Protocol.A simple HTTP/HTTPS client.Advanced Encryption Standard Galois/Counter Mode with Synthetic Initialization Vector (AES-GCM-SIV) encryption.New Message Authentication Code (MAC) algorithms, such as GMAC and KMAC.New Key Derivation Function (KDF) algorithms, such as SSKDF and SSHKDF.New high-level APIs, such as EVP_MAC, EVP_KDF, and EVP_RAND.Low-level APIs deprecated in favor of newer higher-level APIs.Code cleanup.Error handling reworked.Old insecure algorithms are no longer available at the default security level.Interactive mode has been removed from the openssl command-line tool.

OpenSSL is a solid mature software toolkit, so the most important features are already implemented in it. As a result, the latest changes don’t contain that much new functionality for a lot of users. The latest release focuses on architectural improvements to the library.

While OpenSSL is the most popular crypto/TLS library, it’s not the only one. We’ll compare OpenSSL to its competitors in the following sections.

Comparing OpenSSL with GnuTLS

GnuTLS is a free software TLS library that was created for the needs of the GNU Project. When GnuTLS was created, most applications of the GNU Project were distributed under the GPL 2.0 license, which is incompatible with the old OpenSSL license. The authors of the GPL 2.0 licensed software had to include a licensing exception if they wished to link with OpenSSL. GnuTLS was originally licensed under LGPL 2.0, so it did not require such licensing exceptions.

Currently, GnuTLS is licensed under LGPL 2.1. This license allows users to use the library in free and open source software (FOSS) projects. It is also allowed to use the library in closed source projects, but with certain conditions, such as only dynamic linking.

GnuTLS does not include cryptography, big-number arithmetic functionality, and some other functionality that OpenSSL includes. Instead, GnuTLS uses other libraries from the GNU ecosystem that provide the needed functionality: Nettle for cryptography, GMP for big-number arithmetic, Libtasn1 for ASN.1 (short for Abstract Syntax Notation One), and so on.

An interesting feature of GnuTLS is that it supports not only X.509 certificates but also OpenPGP certificates. Unlike an X.509 certificate, which is signed by its issuer at the time of being issued, an OpenPGP certificate supports the so-called web of trust, can have multiple signatures, and signatures can be added once the certificate has been issued. Unfortunately, OpenPGP certificates have not gained popularity for usage in TLS connections.

Apart from OpenPGP certificate support, GnuTLS and its crypto library, Nettle, support fewer crypto algorithms than OpenSSL, and performance-wise, they are a bit slower than OpenSSL. GnuTLS and Nettle, however, support all popular algorithms.

Should you choose OpenSSL or GnuTLS? I recommend that you choose GnuTLS if you are developing GPL-licensed software; otherwise, choose OpenSSL.

Another competitor to OpenSSL is the NSS library, which we will discover in the next section.

Comparing OpenSSL with NSS

Network Security Services (NSS) is the first SSL/TLS library. It originated as security code inside the Netscape Navigator 1.0 browser, released in 1994 by Netscape Communications Corporation. Netscape also invented the SSL protocol, the predecessor of the TLS protocol. Netscape Navigator was succeeded by Netscape Communicator, which was succeeded by the Mozilla browser, which was finally succeeded by Firefox. Soon after the release of Netscape Communicator 4.0 in 1997, the Netscape security code was released as a separate library, named Hard Core Library (HCL). HCL was later renamed NSS.

The NSS library is licensed under Mozilla Public License 2.0, which allows application developers to use the library in both open source and closed source applications.

Notable applications that use the NSS library are applications of Mozilla Foundation, such as Firefox and Thunderbird, some applications from Oracle and Red Hat, and some open source office applications such as LibreOffice and Evolution. While this is a list of solid and respected companies and applications, despite NSS having a long history, it did not become as popular as OpenSSL. I am unsure why. Some people say that OpenSSL is easier to use and has better documentation.

Should you choose NSS or OpenSSL? I recommend OpenSSL because it has better documentation and a larger community of developers and users.

Both NSS and GnuTLS provide their main API in C. However, the Botan library is different, as we’ll find out in the next section.

Comparing OpenSSL with Botan

Most TLS libraries are written in C and provide their main API in C. Botan is a TLS library written in C++11 that adopted C++17 in version 3.0. Botan provides its main API in C++ but also provides API bindings for C and Python. Third-party projects provide API bindings for Ruby, Rust, and Haskell. There are also experimental API bindings for Java and Ocaml.

It’s also worth mentioning that the Botan library has good documentation. Botan is distributed under a simple two-clause BSD license, which allows users to use the library in both open source and closed source applications.

I recommend Botan for developers who want to use the C++ API and are willing to accept a less popular library with a smaller developer community. If you want to use the C API or want a more performant library with a larger developer community, then stick with OpenSSL.

The Botan library has a larger footprint on your storage than OpenSSL, unlike lightweight TLS libraries, which we will discuss next.

Comparing OpenSSL with lightweight TLS libraries

Some lightweight TLS libraries are available. They are targeted at embedded markets such as Internet of Things (IoT) devices or other small devices that many people use but they aren’t considered computers. This includes payment terminals at your local grocery store, smartwatches, smart light bulbs, and industrial sensors of different kinds. Such devices usually have weak CPUs, low memory, and small storage space.

Lightweight TLS libraries usually become lightweight through modularity, the possibility to compile only the needed modules, support fewer cryptographic algorithms and protocols, and have less advanced APIs; that is, they expose fewer features and settings to application developers using the library.

The most well-known lightweight TLS libraries are wolfSSL (formerly yaSSL, also known as Yet Another SSL), Mbed TLS (formerly PolarSSL), and MatrixSSL.

The most advanced of the three libraries seems to be wolfSSL. It supports a lot of cryptographic algorithms, including new algorithms such as ChaCha20 and Poly1305.

The latest versions of wolfSSL have assembly optimizations and are very performant. According to the benchmarks on the wolfSSL website, woflSSL’s raw encryption performance is often on par and will in some cases even noticeably faster than OpenSSL – about 50% faster. Other information on the internet suggests that older versions of wolfSSL are noticeably slower than OpenSSL. If you are interested in performance, then I suggest that you run benchmarking on the target hardware and with the newest versions of the libraries. This is because both libraries are in constant development and newer versions may contain more optimizations.

Mbed TLS and MatrixSSL support noticeably fewer cryptographic algorithms.

WolfSSL and MatrixSSL are dual-licensed under GPL 2.0 and a commercial license. The GPL 2.0 license only allows users to use the library in GPL-compatible FOSS applications. The commercial license allows users to use the library in closed source applications.

Mbed TLS is licensed under Apache License 2.0, which allows users to use the library both in open source and closed source applications.

While the lightweight TLS libraries claim to be significantly smaller, approximately 20 times smaller than OpenSSL, they are not so small in their default configurations. Here are the sizes of the libraries, compiled for an Ubuntu 22.04 x86_64 machine:

wolfSSL (version 5.2.0, libwolfssl.so): 1,768 KiBMbed TLS (version 2.28.0, libmbedtls.so + libmbedcrypto.so): 664 KiBMatrixSSL (version 4.5.1, libssl_s.a + libcrypt_s.a + libcore_s.a): 1,772 KiBOpenSSL (version 3.0.2, libssl.so + libcrypto.so): 5,000 KiB

To cut down on the library’s size, the user of the library has to compile it themselves and disable all the modules that they do not need.

OpenSSL also has a modular design and allows you to exclude unneeded modules from the compilation. However, this is more difficult than with lightweight libraries. Some OpenSSL modules are hard to exclude because other modules have dependencies on them, even if they shouldn’t. And some OpenSSL modules are just very big, particularly the x509 module, since it contains the code for working with X.509 certificates. Thus, while it is possible to cut down the size of compiled OpenSSL, it is not possible to cut down as much as with the lightweight TLS libraries.

Use a lightweight TLS library if OpenSSL does not fit into your device. Otherwise, use a full-size TLS library, such as OpenSSL.

The last two TLS libraries that we will review originated from OpenSSL itself. Let’s find out why the pristine OpenSSL was not good enough for some.

Comparing OpenSSL with LibreSSL

LibreSSL is a fork (derived code) of OpenSSL that was created in 2014 by the OpenBSD Project as a response to the infamous Heartbleed vulnerability that was found in OpenSSL. LibreSSL was founded to increase the security and maintainability of the library by removing old, unpopular, and no longer secure cryptographic algorithms and other features.

OpenBSD is a Unix operating system, one of the BSD systems family aimed at security. OpenBSD developers do not only develop the operating system kernel and utilities. Another famous software project is OpenSSH. Other software projects started as applications for OpenBSD, such as OpenNTPD, OpenSMTPD, and others. This now includes LibreSSL.

After forking, the LibreSSL developers removed a lot of the original OpenSSL code that they considered old or insecure. They claimed that they removed approximately half of the OpenSSL code within the first week. They also added a few new cryptographic algorithms such as Advanced Encryption Standard in Galois/Counter Mode (AES-GCM) and ChaCha-Poly1305. In addition to adding and removing code, some existing code was reworked and hardened in some places.

Despite LibreSSL’s aim to increase security, there were some vulnerabilities in LibreSSL that did not affect OpenSSL, such as CVE-2017-8301.

In the last few years, OpenSSL also did some work on improving the security and maintainability of the library and deprecated/removed some code – though not as much code was removed as in LibreSSL. However, more code and new features were added.

OpenSSL has much more development resources than LibreSSL, which means it advances much faster. For example, from the end of 2018 until the beginning of 2021, LibreSSL has merged approximately 1,500 patches from 36 developers. During the same time, OpenSSL has merged more than 5,000 patches from 276 developers. Apart from the core OpenSSL development team, OpenSSL receives code contributions from big companies such as Oracle, Red Hat, IBM, and others.

While the focus of LibreSSL is providing a TLS library for the OpenBSD project, it also aims to support other platforms and remain API-compatible with OpenSSL. At the time of writing, the LibreSSL API is compatible with OpenSSL 1.0.1, but does not include all the newest APIs from OpenSSL 1.0.2 and later.

Since its fork from OpenSSL, LibreSSL became the default TLS library on OpenBSD, but on most other operating systems, the adoption has been low. Most Linux distributions continued using OpenSSL, and some decided to try LibreSSL as a system-wide option but decided to drop such support later. Most commercial application vendors that used OpenSSL also decided to stick with it.

In the competition between OpenSSL and LibreSSL, OpenSSL is winning. I recommend that you choose OpenSSL unless you are developing software specifically for the OpenBSD community, in which case you should consider LibreSSL.

Another fork of OpenSSL has been created by the mighty Google corporation, as we’ll explore in the next section.

Comparing OpenSSL with BoringSSL

BoringSSL is another fork of OpenSSL that was made public in 2014. BoringSSL was made for the needs of the Google corporation. For years, Google maintained its own patches for OpenSSL for use in various Google products, such as Chrome, Android, and the server’s infrastructure. Finally, they decided to fork OpenSSL and maintain their fork as a separate library.

Like LibreSSL, in BoringSSL, Google removed a lot of the original OpenSSL code, which was responsible for supporting old and unpopular algorithms and features. Google also added some functionality that does not exist in OpenSSL. For example, the CRYPTO_BUFFER functionality allows you to deduplicate X.509 certificates in memory, thus reducing memory usage. It also allows you to remove OpenSSL’s X.509 and ASN.1 code from the application if OpenSSL is linked statically to the application. The X.509 code is a sizeable part of OpenSSL.

Unlike LibreSSL, BoringSSL does not aim for API compatibility with OpenSSL, or even with former versions of BoringSSL. Google wants to change the library API at will. It makes sense because Google controls both BoringSSL and the major software projects that use the library, which makes it possible to synchronize the API changes in BoringSSL and those projects. If the API is not kept stable, it is possible to free up development resources that would otherwise be spent on maintaining the old APIs.

But this also means that if someone outside Google wants to use BoringSSL, they should be ready for breaking changes in the library API, at the least suitable times. This is very inconvenient for developers who use the library. Google understands this and states that although BoringSSL is an open source project, it is not for general use.

My opinion is that BoringSSL was made open source mostly for third-party contributors to Google projects, such as Chrome and Android.

I do not recommend using BoringSSL in your applications due to its API instability. Furthermore, OpenSSL has more features, better documentation, and a much larger community.

With that, we have reviewed several competitors of OpenSSL. You should now understand the main differences between the popular TLS libraries and which library should be used in which case.

Let’s proceed to the summary.

Summary

In this chapter, we have learned what OpenSSL is and why it’s needed. We also walked through the history of OpenSSL and had a glimpse into its future. We reviewed other TLS libraries and competitors to OpenSSL and highlighted their strengths and weaknesses.

You should now have a better understanding of which TLS library you should choose in any given situation. And if the situation is not special, I recommend that you choose OpenSSL since many other people have chosen the same, which implies that OpenSSL is the most popular TLS library and the de facto industry standard.

Now that you know why you should choose OpenSSL, you need to learn how to use it. The next few chapters will be practical ones in that we will learn how to use OpenSSL’s powerful features – from symmetric encryption to creating TLS connections with X.509 certificates.

Part 2: Symmetric Cryptography

In this part, we will learn about symmetric cryptography and its accompanying technologies, such as cryptographically strong message digests, message authentication codes, and generating symmetric cryptographic keys from passwords. This section contains a review of the symmetric ciphers, such as AES and ChaCha, and message digest algorithms, such as SHA-256, supported by OpenSSL. The usage of all the technologies mentioned will be illustrated using command-line and C code examples. We will learn how to initialize and un-initialize OpenSSL and will write our first programs that use the OpenSSL library.

This part contains the following chapters:

Chapter 2, Symmetric Encryption and DecryptionChapter 3, Message DigestsChapter 4, MAC and HMACChapter 5, Derivation of an Encryption Key from a Password

2

Symmetric Encryption and Decryption

In this chapter, we will learn about the important concepts of symmetric encryption – cipher, encryption mode, and padding. There will be an overview of modern ciphers, encryption modes, and padding types and recommendations on which technology to use in which situation. The usage of those technologies will be illustrated by code examples. This is the first chapter that will contain code examples; thus, we will also need to learn how to initialize older versions of the OpenSSL library if we ever want to run our code with an older version of OpenSSL.

We are going to cover the following topics in this chapter:

Understanding symmetric encryptionAn overview of the symmetric ciphers supported by OpenSSLBlock cipher modes of operationPadding for block ciphersHow to generate symmetric encryption keysDownloading and installing OpenSSLHow to encrypt and decrypt with AES on the command lineInitializing and uninitializing OpenSSL libraryHow to compile and link with OpenSSLHow to encrypt with AES programmaticallyHow to decrypt with AES programmatically

Technical requirements

This chapter will contain commands that you can run on a command line and C source code that you can build and run. For the command-line commands, you will need the openssl command-line tool with OpenSSL dynamic libraries. For building the C code, you will need OpenSSL dynamic or static libraries, library headers, a C compiler and, a linker. If you don’t have OpenSSL components installed, this chapter will teach you how to install them.

We will implement some example programs in this chapter in order to practice what we are learning. The full source code of those programs can be found here: https://github.com/PacktPublishing/Demystifying-Cryptography-with-OpenSSL-3/tree/main/Chapter02.

Understanding symmetric encryption

Symmetric encryption is used for encrypting data with an encryption key. Note that an encryption key is not the same as a password, but an encryption key can be derived from a password. How to do so will be explained in a Chapter 5, Derivation of an Encryption Key from a Password.

Symmetric encryption is called symmetric because the same key is used both for encryption and decryption operations. There is also asymmetric encryption, where different keys (public and private) are used for encryption and decryption. Asymmetric encryption will be covered in Chapter 6, Asymmetric Encryption and Decryption.

In order to symmetrically encrypt something, you have to use a symmetric encryption algorithm. This kind of algorithm is also called a cipher. It is worth noting the term cipher has a broad meaning and can refer to any encryption or encryption-related algorithm, including algorithms for symmetric and asymmetric encryption, message digests, and even a set of algorithms used during a TLS connection – for example, ECDHE-ECDSA-AES256-GCM-SHA384. In the course of this book, we will use the term cipher to mean different things depending on the current context. For example, right now, the context is symmetric encryption – thus, we are using cipher to denote a symmetric encryption algorithm.

Useful data that is being encrypted is called plaintext, even if it is not text. Images, music, videos, archives, or databases, anything you want to encrypt is plaintext in encryption terms.

The result of encryption process, encrypted data, is called ciphertext. Ciphertext almost never looks like text, but this terminology is still used in the industry.

Using the given terminology, we can say that encryption transforms plaintext into ciphertext and decryption transforms ciphertext back into plaintext.

In order to encrypt some plaintext, you have to do the following:

Choose an encryption algorithm (cipher).Generate an encryption key.Generate an initialization vector (IV).Choose the cipher operation mode.Choose the padding type.Encrypt the plaintext with the cipher, key, and padding using the chosen cipher operation mode.

Depending on the cipher and its operation mode, some of these steps may not be required. For instance, stream ciphers and some block cipher operation modes do not require padding.

Someone with little experience in cryptography may ask, “why do I have to choose so many parameters? When I used encrypted archives, my archiver only asked me for a password, not for a cipher, a key, an IV, a mode, or a padding type!” The answer is that an archiver or another user-friendly tool handles all those things behind the scenes, makes most of these choices on behalf of the user, and generates an encryption key from the user-supplied password.

Nevertheless, we are going to learn a bit about the basic encryption concepts mentioned in order to make conscious choices and understand how those concepts work together. We will start by learning about encryption algorithms (or ciphers).

An overview of the symmetric ciphers supported by OpenSSL

In this section, we will review the symmetric encryption algorithms supported by OpenSSL, but, first, we need to introduce some concepts that will help us to understand the differences between ciphers, their properties, and their advantages and disadvantages. Symmetric ciphers are divided into two categories: block ciphers and stream ciphers.

Comparing block ciphers and stream ciphers

Block ciphers operate on blocks of data. For example, a popular Advanced Encryption Standard (AES) cipher has a block size of 128 bits, meaning that the cipher encrypts or decrypts data in 128-bit blocks. If the amount of data is larger than the block size, the data is split into blocks of the needed size needed for processing. If the plaintext length is not multiple of the block size, the last block is usually padded up to the block size according to the chosen padding type. Thus, in most block cipher operation modes, the ciphertext length is always a multiple of the block size.

What happens during encryption by the block cipher, explained simply? Input to the encryption is a block of plaintext (for example, 16 bytes), the encryption key, and possibly some cipher operation mode-specific data (for example the previous ciphertext block or the IV). Individual bytes of the plaintext are put into a list or a matrix. Then, different logical and arithmetical operations are performed on the plaintext bytes, the encryption key bytes, and the additional operation mode-specific data bytes: bit rotations, eXclusive OR (XOR), addition or subtraction, or the exchange of full or partial bytes in the list or matrix. The output of those operations is a ciphertext block of the same size as the input plaintext block. For decryption, the input is a ciphertext block, the encryption key, and the operation mode-specific data. The operations from the encryption phase are performed in the opposite order and, as a result, the ciphertext block is transformed back into the plaintext block:

Figure 2.1 – How block ciphers work

Stream ciphers do not operate on blocks but operate on single bytes or even bits of data. This means that stream ciphers do not need padding. They also do not have operating modes, although some researchers are trying to introduce operation modes for stream ciphers too. These characteristics of stream ciphers make them easier to use, especially for streamed data, easier to implement, and faster to perform than block ciphers. However, cryptographers believe that the existing stream ciphers are generally less secure than block ciphers and it is easier to further weaken the security of a stream cipher through an implementation mistake than it is with a block cipher.

How do stream ciphers work, explained simply? During both encryption and decryption, a stream cipher generates a so-called pseudorandom cipher digit stream or keystream, based on the encryption key and the IV or nonce (number used once). The cipher takes bytes of the encryption key and the IV and performs logical and mathematical operations on them in order to generate an infinite stream of seemingly random bytes. This is the keystream we refer to. We can say that a stream cipher, when generating the keystream, works like a pseudorandom number generator (PRNG), which is seeded by the encryption key and the IV. On encryption, the ciphertext stream is produced by combining the keystream with the plaintext stream. Usually, the keystream bytes and the plaintext bytes are just XOR-ed. On decryption, the same keystream is generated and combined with the ciphertext stream. If the ciphertext stream was produced by XOR-ing the keystream with the plaintext stream, then the plaintext stream is recovered by using the same XOR operation, but this time, on the keystream and the ciphertext stream. In order to produce the same keystream for decryption, the same encryption key and IV must be used:

Figure 2.2 – How stream ciphers work

Understanding symmetric cipher security

When cryptographers are talking about security of a cipher, they usually measure cipher security or cipher strength in security bits. For example, if an attacker needs to perform 2256 computational operations to break a cipher, then the cipher security is 256 bits.

The maximum amount of security bits a cipher can have is the length of the used encryption key. For example, if a cipher uses a 256-bit encryption key, its strength is no more than 256 bits. It’s rather easy to prove. Let’s imagine an attacker has some ciphertext, does not have the encryption key and the corresponding plaintext, but can distinguish the valid plaintext from random data. The attacker’s goal is to recover the encryption key or the plaintext. If the key length is 256 bits, then there are 2256 possible encryption keys that can be used with the current cipher. The attacker can use brute force, also known as the exhaustive search approach – that is, trying every possible key for decrypting the ciphertext. Then, they must try no more than 2256 times – that is, perform 2256 computational operations at most.

Here, a computational operation does not mean a very basic operation, such as a Central Processing Unit (CPU) instruction, but a complete attempt at decrypting a small amount of ciphertext, such as one ciphertext block. The important thing here is that the amount of the computational operations depends on the key length. Decryption of one ciphertext block is very fast and cheap on modern CPUs, but doing such an operation 2256 times would take billions of years. Encryption security is based on the attacker’s necessity to perform an infeasible number of operations to decrypt the ciphertext or recover the key.