KVM Virtualization Cookbook - Konstantin Ivanov - E-Book

KVM Virtualization Cookbook E-Book

Константин Иванов

0,0
41,99 €

-100%
Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.
Mehr erfahren.
Beschreibung

Deploy, manage, and scale virtual instances using Kernel-based Virtual Machines

About This Book

  • Build, manage and scale virtual machines with practical step-by-step examples
  • Leverage the libvirt user-space tools and libraries to manage the life-cycle of KVM instances
  • Deploy and scale applications inside KVM virtual machines with OpenStack

Who This Book Is For

If you are a system administrator working KVM virtualization, this book will help you grow on your expertise of working with the infrastructure to manage things in a better way. You should have a knowledge of working with Linux based systems.

What You Will Learn

  • Deploy different workloads in isolation with KVM virtualization and better utilize the available compute resources
  • Explore the benefits of running applications with KVM and learn to prevent the “bad-neighbor” effect
  • Leveraging various networking technologies in the context of virtualization with Open vSwitch and the Linux bridge.
  • Create KVM instances using Python and inspect running KVM instances
  • Understand Kernel Tuning for enhanced KVM performance and better memory utilization

In Detail

Virtualization technologies such as KVM allow for better control over the available server resources, by deploying multiple virtual instances on the same physical host, or clusters of compute resources. With KVM it is possible to run various workloads in isolation with the hypervisor layer providing better tenant isolation and higher degree of security.

This book will provide a deep dive into deploying KVM virtual machines using qemu and libvirt and will demonstrate practical examples on how to run, scale, monitor, migrate and backup such instances. You will also discover real production ready recipes on deploying KVM instances with OpenStack and how to programatically manage the life cycle of KVM virtual machines using Python. You will learn numerous tips and techniques which will help you deploy & plan the KVM infrastructure. Next, you will be introduced to the working of libvirt libraries and the iPython development environment.

Finally, you will be able to tune your Linux kernel for high throughput and better performance. By the end of this book, you will gain all the knowledge needed to be an expert in working with the KVM virtualization infrastructure.

Style and approach

This book takes a complete practical approach with many step-by-step example recipes on how to use KVM in production. The book assumes certain level of expertise with Linux systems and virtualization in general. Some knowledge of Python programming is encouraged, to fully take advantage of the code recipes.

Sie lesen das E-Book in den Legimi-Apps auf:

Android
iOS
von Legimi
zertifizierten E-Readern

Seitenzahl: 309

Bewertungen
0,0
0
0
0
0
0
Mehr Informationen
Mehr Informationen
Legimi prüft nicht, ob Rezensionen von Nutzern stammen, die den betreffenden Titel tatsächlich gekauft oder gelesen/gehört haben. Wir entfernen aber gefälschte Rezensionen.



KVM Virtualization Cookbook

 

 

 

 

 

 

 

 

 

 

Learn how to effectively use KVM in production

 

 

 

 

 

 

 

 

 

 

Konstantin Ivanov

 

BIRMINGHAM - MUMBAI

< html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">

KVM Virtualization Cookbook

 

 

Copyright © 2017 Packt Publishing

 

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

 

First published: June 2017

Production reference: 1140617

 

 

Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.

 

ISBN 978-1-78829-467-6

 

www.packtpub.com

Credits

 

Author

Konstantin Ivanov

Copy Editor

Dipti Mankame

Reviewer

Jay Payne

Project Coordinator

Judie Jose

Acquisition Editor

Rahul Nair

 

Proofreader

Safis Editing

Content Development Editor

Devika Battike

 

Indexer

Rekha Nair

Technical Editor

Prachi Sawant

 

Graphics

Kirk D'Penha

Production Coordinator

Aparna Bhagat

 

 

 

About the Author

Konstantin Ivanov is a Linux systems engineer, an open source developer, and a technology blogger who has been designing, configuring, deploying, and administering large-scale, highly available Linux environments for more than 15 years.

His interests include large distributed systems and task automation, along with solving technical challenges involving multiple technology stacks.

Konstantin received two master of science in computer science degrees from universities in Bulgaria and the United States, specializing in system and network security and software engineering.

In his spare time, he loves writing technology blogs and spending time with his two boys.

 

To my amazing wife, Deepa, my trusty companion through the many long days of writing.

About the Reviewer

Jay Payne has been a database administrator 5 at Rackspace for over 10 years, working on the design, development, implementation, and operation of storage systems.

Previously, Jay worked on billing and support systems for hosting companies. For the last 20 years, he has primarily focused on the data life cycle, from database architecture, administration, operations, and reporting to disaster recovery and compliance. He has domain experience in hosting, finance, billing, and customer-support industries.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

 

https://www.packtpub.com/mapt

 

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/178829467X.

If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

Table of Contents

Preface

What this book covers

What you need for this book

Who this book is for

Sections

Getting ready

How to do it…

How it works…

There's more…

See also

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color images of this book

Errata

Piracy

Questions

Getting Started with QEMU and KVM

Introduction

Installing and configuring QEMU

Getting ready

How to do it...

How it works...

Managing disk images with qemu-img

Getting ready

How to do it...

How it works...

There's more...

Preparing images for OS installation with qemu-nbd

Getting ready

How to do it...

How it works...

Installing a custom OS on the image with debootstrap

Getting ready

How to do it...

How it works...

Resizing an image

Getting ready

How to do it...

How it works...

Using pre-existing images

Getting ready

How to do it...

How it works...

There's more...

See also

Running virtual machines with qemu-system-*

Getting ready

How to do it...

How it works...

Starting the QEMU VM with KVM support

Getting ready

How to do it...

How it works...

There's more...

Connecting to a running instance with VNC

Getting ready

How to do it...

How it works...

Using libvirt to Manage KVM

Introduction

Installing and configuring libvirt

Getting ready

How to do it...

How it works...

Defining KVM instances

Getting ready

How to do it...

How it works...

There's more...

Starting, stopping, and removing KVM instances

Getting ready

How to do it...

How it works...

Inspecting and editing KVM configs

Getting ready

How to do it...

How it works...

Building new KVM instances with virt-install and using the console

Getting ready

How to do it...

How it works...

Managing CPU and memory resources in KVM

Getting ready

How to do it...

How it works...

There's more...

Attaching block devices to virtual machines

Getting ready

How to do it...

How it works...

Sharing directories between a running VM and the host OS

Getting ready

How to do it...

How it works...

There's more...

Autostarting KVM instances

Getting ready

How to do it...

How it works...

Working with storage pools

Getting ready

How to do it...

How it works...

There's more...

Managing volumes

Getting ready

How to do it...

How it works...

Managing secrets

Getting ready

How to do it...

How it works...

KVM Networking with libvirt

Introduction

The Linux bridge

Getting ready

How to do it...

How it works...

There's more...

The Open vSwitch

Getting ready

How to do it...

How it works...

There's more...

Configuring NAT forwarding network

Getting ready

How to do it...

How it works...

Configuring bridged network

Getting ready

How to do it...

How it works...

Configuring PCI passthrough network

Getting ready

How to do it...

How it works...

Manipulating network interfaces

Getting ready

How to do it...

How it works...

Migrating KVM Instances

Introduction

Manual offline migration using an iSCSI storage pool

Getting ready

How to do it...

How it works...

There's more...

Manual offline migration using GlusterFS shared volumes

Getting ready

How to do it...

How it works...

Online migration using the virsh command with shared storage

Getting ready

How to do it...

How it works...

There's more...

Offline migration using the virsh command and local image

Getting ready

How to do it...

How it works...

Online migration using the virsh command and local image

Getting ready

How to do it...

How it works...

Monitoring and Backup of KVM Virtual Machines

Introduction

Resource usage collection with libvirt

Getting ready

How to do it...

How it works...

There's more...

Monitoring KVM instances with Sensu

Getting ready

How to do it...

How it works...

There's more...

Simple KVM backups with tar and rsync

Getting ready

How to do it...

How it works...

Creating snapshots

Getting ready

How to do it...

How it works...

Listing snapshots

Getting ready

How to do it...

How it works...

Inspecting snapshots

Getting ready

How to do it...

How it works...

Editing snapshots

Getting ready

How to do it...

How it works...

Reverting snapshots

Getting ready

How to do it...

How it works...

Deleting snapshots

Getting ready

How to do it...

How it works...

Deploying KVM Instances with OpenStack

Introduction

Preparing the host for the OpenStack deployment

Getting ready

How to do it...

How it works...

Installing and configuring the OpenStack Keystone identity service

Getting ready

How to do it...

How it works...

Installing and configuring the OpenStack Glance image service

Getting ready

How to do it...

How it works...

Installing and configuring the OpenStack Nova compute service

Getting ready

How to do it...

How it works...

Installing and configuring the OpenStack Neutron networking service

Getting ready

How to do it...

How it works...

Building and inspecting KVM instances with OpenStack

Getting ready

How to do it...

How it works...

Stopping KVM instances with OpenStack

Getting ready

How to do it...

How it works...

Terminating KVM instances with OpenStack

Getting ready

How to do it...

How it works...

Using Python to Build and Manage KVM Instances

Introduction

Installing and using the Python libvirt library

Getting ready

How to do it...

How it works...

Defining KVM instances with Python

Getting ready

How to do it...

How it works...

There's more...

Starting, stopping, and deleting KVM instances with Python

Getting ready

How to do it...

How it works...

There's more...

Inspecting KVM instances with Python

Getting ready

How to do it...

How it works...

There's more...

Building a simple REST API server with libvirt and bottle

Getting ready

How to do it...

How it works...

There's more...

Kernel Tuning for KVM Performance

Introduction

Tuning the kernel for low I/O latency

Getting ready

How to do it...

How it works...

Memory tuning for KVM guests

Getting ready

How to do it...

How it works...

CPU performance options

Getting ready

How to do it...

How it works...

NUMA tuning with libvirt

Getting ready

How to do it...

How it works...

There is more...

Tuning the kernel for network performance

Getting ready

How to do it...

How it works...

Preface

The foundation of most modern cloud deployments is some sort of virtualization technology, such as Kernel-based Virtual Machine (KVM). KVM has been part of the mainstream Linux kernel since version 2.6.20, released in February 2007, and since then has enjoyed wide system adoption.

Virtualization in general not only provides a way to fully utilize server resources, but also allows greater multitenancy, along with running various workloads on the same system.

The OpenStack cloud operating system uses KVM as its default compute driver, providing a centralized way of managing the lifecycle of virtual machines: from building, resizing, and migrating to pausing and terminating.

This book is about KVM and how to build and manage virtual machines in the most efficient way. Unlike containerization solutions such as Docker, KVM is designed to run an entire operating system rather than a single process. Although containerization has its advantages, full virtualization provides extra security by having the hypervisor layer between the guest OS and the host and added stability by running different guest kernels (kernel panic of the guest instance will not bring the entire host down) or entirely different operating systems.

This book takes a rather direct and pragmatic step-by-step approach--you will learn how to create custom guest images, install and configure QEMU and libvirt, resize and migrate instances, deploy monitoring, and provision guests using OpenStack and Python.

What this book covers

Chapter 1, Getting Started with QEMU and KVM, provides recipes for installing and configuring QEMU, creating and managing disk images, and running virtual machines with the qemu-system utility.

Chapter 2, Using libvirt to Manage KVM, covers everything that is needed to install, configure, and run KVM instances using libvirt. You will learn what packages and tools are required, along with different ways of configuring virtual machines using XML definition files. By the end of this chapter, you will have a Linux system with running KVM instances.

Chapter 3, KVM Networking with libvirt, will present recipes for working with the Linux Bridge and Open vSwitch and will demonstrate how to connect KVM instances using NAT, bridged, and PCI pass-through networking.

Chapter 4, Migrating KVM Instances, will show examples on how to perform offline and online migration of running KVM virtual machines.

Chapter 5, Monitoring and Backup of KVM Virtual Machines, will present examples on how to deploy complete monitoring systems with Sensu and Uchiwa and demonstrate how to create snapshots to use as backups.

Chapter 6, Deploying KVM Instances with OpenStack, demonstrates how to provision KVM instances with OpenStack. It begins by introducing the various components that make OpenStack and how to use the LXC Nova driver to automatically provision virtual machines.

Chapter 7, Using Python to Build and Manage KVM Instances, will present recipes for building, starting, and managing the lifecycle of KVM instances using the Python libvirt library. We will also see examples on how to build a simple RESTful API to work with KVM.

Chapter 8, Kernel Tuning for KVM Performance, shows recipes for tuning the host OS for better I/O, CPU, memory, and network utilization. The presented examples can also be used inside the KVM instances, depending on their workload.

 

What you need for this book

A beginner-level knowledge of Linux and the command line is required to follow along and run the recipes. Some Python experience is required to fully understand and be able to run the examples in Chapter 7, Using Python to Build and Manage KVM Instances.

Most recipes in this book have been tested on bare metal servers with processors supporting virtualization and the latest version of Ubuntu Linux.

Who this book is for

This book is for anyone who is curious about virtualization with KVM--from Linux administrators who are looking for in-depth understanding of how KVM can be deployed and managed in large scale production environments to software developers that need a quick and easy way to prototype code in isolated guests. A DevOps engineer is most likely the best job title for those who want to read the book from cover to cover and try all examples.

Sections

In this book, you will find several headings that appear frequently (Getting ready, How to do it, How it works, There's more, and See also).

To give clear instructions on how to complete a recipe, we use these sections as follows:

Getting ready

This section tells you what to expect in the recipe, and describes how to set up any software or any preliminary settings required for the recipe.

How to do it…

This section contains the steps required to follow the recipe.

How it works…

This section usually consists of a detailed explanation of what happened in the previous section.

There's more…

This section consists of additional information about the recipe in order to make the reader more knowledgeable about the recipe.

See also

This section provides helpful links to other useful information for the recipe.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.

To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

You can download the code files by following these steps:

Log in or register to our website using your e-mail address and password.

Hover the mouse pointer on the

SUPPORT

tab at the top.

Click on

Code Downloads & Errata

.

Enter the name of the book in the

Search

box.

Select the book for which you're looking to download the code files.

Choose from the drop-down menu where you purchased this book from.

Click on

Code Download

.

You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/KVM-Virtualization-Cookbook. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Downloading the color images of this book

We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/KVMVirtualizationCookbook_ColorImages.pdf.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.

To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at [email protected] with a link to the suspected pirated material.

We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.

 

Getting Started with QEMU and KVM

In this chapter, we will cover the following topics:

Installing and configuring QEMU

Managing disk images with qemu-img

Preparing images for OS installation with

qemu-nbd

Installing a custom OS on the image with debootstrap

Resizing an image

Using pre-existing images

Running virtual machines with qemu-system-*

Starting the QEMU VM with KVM support

Connecting to a running instance with VNC

Introduction

Quick Emulator (QEMU) is the main component of the QEMU/KVM virtualization technology suit. It provides hardware virtualization and processor emulation. QEMU runs in userspace and, without the need for kernel, drivers can still provide fast system emulation. QEMU supports two operating modes:

Full system emulation, where QEMU emulates an entire computer system, including the CPU type and peripherals

User mode emulation, where QEMU can run a process that has been compiled on a different CPU architecture natively

In this book, we are going to focus on full system emulation with the hardware acceleration support provided by the Kernel-based Virtual Machine (KVM) hypervisor.

In this chapter, we will start by installing QEMU on Linux, then explore various examples of building, managing, and using disk images for the virtual instances. We will then have an in-depth look at running QEMU in full system emulation mode, using the provided binaries. We will see examples of using the KVM kernel module to accelerate the QEMU processes. Finally, we are going to end the chapter with details on how to connect to the virtual machines we started earlier, using VNC clients.

Installing and configuring QEMU

In this recipe, we will look at installing QEMU on a single server with the provided distribution packages. For production environments, we recommend using precompiled, packaged versions of QEMU for easier and more consistent deployments. However, we are going to see an example of how to compile QEMU from source, in case you need a certain version that you might want to package later.

Getting ready

Depending on your Linux distribution, the package name and installation commands will differ. You can use your system's package manager, such as apt, dnf, or yum to search for any packages containing the QEMU string and get familiar with what is available for your particular Linux variant. The source code can be downloaded from the official QEMU project website at http://www.qemu-project.org/download/#source.

How to do it...

Perform the following steps to install QEMU from packages on Ubuntu/Debian and RHEL/CentOS distributions:

On Ubuntu/Debian distributions, update your packages index:

root@kvm:~# apt-get update

Install the package:

root@kvm:~# apt-get install -y qemu

On CentOS/RHEL distributions execute:

root@kvm:~# yum install qemu-kvm

To install from source, execute the following:

Download the archive first:

root@kvm:~#cd /usr/src && wget http://download.qemu-project.org/qemu-2.8.0.tar.xz

Extract the files from the archive:

root@kvm:/usr/src# tar xvJf qemu-2.8.0.tar.xz && cd qemu- 2.8.0

Configure and compile the source code:

root@kvm:/usr/src/qemu-2.8.0# ./configure root@kvm:/usr/src/qemu-2.8.0# make && make install

How it works...

Installing QEMU is quite trivial, as we just saw. Let's have a look at what the QEMU metapackage installed on Ubuntu looks like:

root@kvm:~# dpkg --list | grep qemuii ipxe-qemu 1.0.0+git-20150424.a25a16d-1ubuntu1 all PXE boot firmware - ROM images for qemuii qemu 1:2.5+dfsg-5ubuntu10.8 amd64 fast processor emulatorii qemu-block-extra:amd64 1:2.5+dfsg-5ubuntu10.8 amd64 extra block backend modules for qemu-system and qemu-utilsii qemu-slof 20151103+dfsg-1ubuntu1 all Slimline Open Firmware -- QEMU PowerPC versionii qemu-system 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binariesii qemu-system-arm 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (arm)ii qemu-system-common 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (common files)ii qemu-system-mips 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (mips)ii qemu-system-misc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (miscelaneous)ii qemu-system-ppc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (ppc)ii qemu-system-sparc 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (sparc)ii qemu-system-x86 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU full system emulation binaries (x86)ii qemu-user 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU user mode emulation binariesii qemu-user-binfmt 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU user mode binfmt registration for qemu-userii qemu-utils 1:2.5+dfsg-5ubuntu10.8 amd64 QEMU utilitiesroot@kvm:~#

From the preceding output, we can see that there are few packages involved. If you are interested, you can read the individual description to get more familiar with what each package provides.

It's worth mentioning that all binaries provided from the earlier-mentioned packages start with the prefix QEMU. You can use tab completion to see the list of available executables:

root@kvm:~# qemu-qemu-aarch64 qemu-io qemu-mips64el qemu-ppc64 qemu-sparc32plus qemu-system-lm32 qemu-system-mipsel qemu-system-sh4 qemu-system-xtensaqemu-alpha qemu-m68k qemu-mipsel qemu-ppc64abi32 qemu-sparc64 qemu-system-m68k qemu-system-moxie qemu-system-sh4eb qemu-system-xtensaebqemu-arm qemu-make-debian-root qemu-mipsn32 qemu-ppc64le qemu-system-aarch64 qemu-system-microblaze qemu-system-or32 qemu-system-sparc qemu-tilegxqemu-armeb qemu-microblaze qemu-mipsn32el qemu-s390x qemu-system-alpha qemu-system-microblazeel qemu-system-ppc qemu-system-sparc64 qemu-unicore32qemu-cris qemu-microblazeel qemu-nbd qemu-sh4 qemu-system-arm qemu-system-mips qemu-system-ppc64 qemu-system-tricore qemu-x86_64qemu-i386 qemu-mips qemu-or32 qemu-sh4eb qemu-system-cris qemu-system-mips64 qemu-system-ppc64le qemu-system-unicore32qemu-img qemu-mips64 qemu-ppc qemu-sparc qemu-system-i386 qemu-system-mips64el qemu-system-ppcemb qemu-system-x86_64root@kvm:~#

We can see that there's a single executable for each CPU architecture type that can be emulated.

Managing disk images with qemu-img

To run virtual machines, QEMU needs images to store the filesystem of the guest OS. The image itself is a type of file, and it represents the guest filesystem residing on a virtual disk. QEMU supports various images and provides tools to create and manage them. In this recipe, we are going to build a blank disk image with the qemu-img utility.

Getting ready

To use this recipe, we need to have the qemu-img utility installed. If you followed the steps in the first recipe, you should have that covered. To check what image types are supported on your Linux distribution, run the following command:

root@kvm:~# qemu-img -h | grep SupportedSupported formats: bochs vvfat rbd vpc parallels tftp ftp ftps raw https qcow dmg http qcow2 quorum null-aio cloop vdi iscsi null-co vhdx blkverify file vmdk host_cdrom blkdebug host_device sheepdog qed nbdroot@kvm:~#

From the preceding output, we can see that there are many supported images on the test system that we are using. Make sure that your QEMU version supports the raw image type, as it's the default and that is what we are going to use in this recipe. One of the most commonly used image type is qcow2, which supports copy on write, compression, encryption, and snapshotting. We are going to leverage that in later recipes.

Please note that even though QEMU supports multiple formats, that does not necessarily mean that you can run virtual machines on them. However, qemu-img can be used to convert different images to raw and qcow2 formats. For best performance, use raw or qcow2 image formats.

How to do it...

Perform the following steps to create a blank raw image of a specified size and to verify that the file was created on the host:

Create a raw image named

debian.img

with size of 10 GB:

root@kvm:~# qemu-img create -f raw debian.img 10G Formatting 'debian.img', fmt=raw size=10737418240 root@kvm:~#

Check that the file was created:

root@kvm:~# ls -lah debian.img -rw-r--r-- 1 root root 10G Feb 10 16:58 debian.img root@kvm:~#

 

 

Examine the file type:

root@kvm:~# file -s debian.img debian.img: data root@kvm:~#

Obtain more information about the image:

root@kvm:~# qemu-img info debian.img

image: debian.img

file format: raw

virtual size: 10G (10737418240 bytes)

disk size: 0

root@kvm:~#

How it works...

The qemu-img utility allows us to create, convert, and modify guest images.

In step 1, we used the -f flag specifying the image format; in this case, raw, the name of the image to be created and the size in gigabytes.

In step 4, we used the info subcommand to gather additional information about the existing image. Note how the disk size is showing as currently being zero. This is due to the fact that this is a blank image, not containing a filesystem. We are going to create one in the next recipe.

There's more...

In this recipe, we listed the supported disk image formats by QEMU. The following is a brief description of the most common types that you might encounter:

raw

: Raw disk image format. This is the default format and can be one of the fastest file-based formats. If you format this image with a filesystem that supports holes, for example, EXT3, then only sectors that have data will use space. The main drawback of the raw images is the lack of features, making them ideal for testing and quick prototyping.

qcow2

: As we mentioned in the previous section, this is one of the most feature-rich formats. It supports VM snapshots, compression, and encryption for the price of slightly reduced performance.

qcow

: This is an older QEMU image format that

supports backing files, compact image files, encryption, and compression.

dmg

: This is the Mac disk image format. The Mac disk image provides secure password protection and compression, and it is most commonly used to distribute software, rather than running virtual machines.

nbd

: The network block device, typically used for accessing remote storage devices.

vdi

: This disk format is used by the Oracle VirtualBox software and can be used to run virtual machines on various CPU platforms.

vmdk

: This is the VMware disk image type, where a single virtual hard disk can span multiple files.

vhdx

: Microsoft Hyper-V uses this image format. It provides large storage capacity, data corruption protection during power failures and read/write optimization for larger disk images.

In this book, we are going to use the raw and qcow2 disk formats, as they provide the best performance and toolset for running and manipulating them.

Preparing images for OS installation with qemu-nbd

In the previous recipe, we created a blank raw image. In this recipe, we are going to make a partition and a filesystem on it, getting the image ready for full guest OS installation. When creating the partition and file system, you should consider the type of load that the virtual instance will create. If your applications running inside VM are IO bound, you might consider XFS for the image filesystem. For this recipe, we are going to use EXT4, as most Linux distributions support it out of the box.

Getting ready

For this recipe, we are going to use the following tools:

qemu-nbd

sfdisk

The

nbd

kernel module

mkfs

Most Linux distributions should already have the tools installed. If that's not the case, consult your distribution's documentation on how to install them.

How it works...

We take advantage of the functionality that the nbd kernel module provides by allowing us to associate a raw image file to a block device using the qemu-nbd utility. To get more information about the kernel module run the following code:

root@kvm:~# modinfo nbdfilename: /lib/modules/4.4.0-62-generic/kernel/drivers/block/nbd.kolicense: GPLdescription: Network Block Devicesrcversion: C67096AF2AE3C738DBE0B7Edepends:intree: Yvermagic: 4.4.0-62-generic SMP mod_unload modversionsparm: nbds_max:number of network block devices to initialize (default: 16) (int)parm: max_part:number of partitions per device (default: 0) (int)root@kvm:~#

We can examine the block device metadata created in step 2 by running the following command:

root@kvm:~# file -s /dev/nbd0/dev/nbd0: x86 boot sectorroot@kvm:~#

After creating the two new partitions in step 3, the type of the image file has changed. Let's examine it again:

root@kvm:~# file -s debian.imgdebian.img: x86 boot sectorroot@kvm:~#

We chose to use the sfdisk utility to create the partitions, but you can use the fdisk utility interactively instead if you prefer. The end result will be the same.

Now that we have an image file that contains two partitions and a filesystem, we can proceed with installing the guest OS in the next recipe.

Installing a custom OS on the image with debootstrap

In this recipe, we are going to use the debootstrap utility to install a Debian distribution on the raw image we prepared in the previous two recipes. The debootstrap command is used to bootstrap a basic Debian system using a specific public mirror. By the end of this recipe, we should have an image containing an entire Linux distribution, ready for QEMU execution.

Getting ready

We are going to need the following in order to complete this recipe:

The block devices created in the previous recipe

The

debootstrap

utility

The

chroot

utility

To ensure that the swap and root block devices are still present on the system, run the following:

root@kvm:~# ls -la /dev/nbd0*

brw-rw---- 1 root disk 43, 0 Feb 10 18:24 /dev/nbd0brw-rw---- 1 root disk 43, 1 Feb 10 18:24 /dev/nbd0p1brw-rw---- 1 root disk 43, 2 Feb 10 18:24 /dev/nbd0p2root@kvm:~#

If that's not the case, please refer to the Preparing images for OS installation with qemu-nbd recipe on how to associate the raw image with the /deb/nbd0 block device.