31,19 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
Proxmox is an open source server virtualization solution that has enterprise-class features for managing virtual machines, for storage, and to virtualize both Linux and Windows application workloads.
You'll begin with a refresher on the advanced installation features and the Proxmox GUI to familiarize yourself with the Proxmox VE hypervisor. Then, you'll move on to explore Proxmox under the hood, focusing on storage systems, such as Ceph, used with Proxmox. Moving on, you'll learn to manage KVM virtual machines, deploy Linux containers fast, and see how networking is handled in Proxmox. You'll also learn how to protect a cluster or a VM with a firewall and explore the new high availability features introduced in Proxmox VE 5.0. Next, you'll dive deeper into the backup/restore strategy and see how to properly update and upgrade a Proxmox node. Later, you'll learn how to monitor a Proxmox cluster and all of its components using Zabbix. Finally, you'll discover how to recover Promox from disaster strikes through some real-world examples.
By the end of the book, you'll be an expert at making Proxmox work in production environments with minimal downtime.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 493
Veröffentlichungsjahr: 2017
Ähnliche
Third Edition
BIRMINGHAM - MUMBAI
Mastering Proxmox
Third Edition
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: July 2014
Second edition: May 2016
Third edition: November 2017
Production reference: 1141117
ISBN 978-1-78839-760-5
www.packtpub.com
Credits
Author
Wasim Ahmed
Copy Editors
Safis Editing
Madhusudan Uchil
Reviewers
Nicolas Ledez
Jorge Moratilla Porras
Project Coordinator
Virginia Dias
Commissioning Editor
Vijin Boricha
Proofreader
Safis Editing
Acquisition Editor
Rahul Nair
Indexer
Francy Puthiry
Content Development Editor
Sharon Raj
Graphics
Kirk D'Penha
Technical Editors
Vishal Kamal Mewada
Khushbu Sutar
Production Coordinator
Nilesh Mohite
About the Author
Wasim Ahmed, born in Bangladesh and now a citizen of Canada, is a veteran of the IT world. He first came into close contact with computers in 1992 and never looked back. Wasim has a deep understanding of networks, virtualization, big data storage, and network security.
By profession, Wasim is the CEO of a global IT support and cloud service provider based in Calgary, Alberta. He serves many companies and organizations through his company on a daily basis. Wasim's strength comes from his experience, which comes from learning and serving continually. Wasim strives to find the most effective solution at the most competitive price. He has built over 20 enterprise production virtual infrastructures using Proxmox and the Ceph storage system.
Wasim and his team are notorious for not simply accepting a technology based on its description alone, but putting it through rigorous testing to check its validity. Any new technology that his company provides goes through months of continuous testing before it is accepted. Proxmox made the cut superbly.
About the Reviewers
Nicolas Ledez has been working as a system administrator since 2000. He has been in big businesses such as Orange (a French telecom company) and in small organizations too. His skills are in DevOps, Linux, Ruby, Python, Ansible, Chef, Saltstack, and others. Currently, he is a DevOps architect at Cozy Cloud. You can find him on the internet with the pseudonym nledez.
Jorge Moratilla Porras has a bachelor's degree in computer science and has been working for internet companies since 1998. He has been working as a contractor for companies such as Sun Microsystems and Oracle. His passions are teaching and improving workloads using automation techniques. He has been working as a Sun Microsystems certified instructor and field engineer for several years. He has a large background working with products such as Sun Solaris, Linux, LDAP services, and CheckPoint. Recently, he has been working with configuration management products such as Puppet and Chef on his assignments and has been taking part in Madrid DevOps (a group of technicians devoted to continuous deployment and DevOps culture) as coordinator. He promotes the adoption of a culture of continuous improvement in enterprise and startups as the baseline to do great things. You can meet him at talks and hangouts that he organizes in the community.
He has collaborated as a reviewer on other Packt titles as well:
Configuration Management with Chef-Solo
by
Naveed ur Rahman
Proxmox Cookbook
by
Wasim Ahmed
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1788397606.
If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the color images of this book
Errata
Piracy
Questions
Understanding Proxmox VE and Advanced Installation
Understanding Proxmox features
It is free!
Built-in firewall
Open vSwitch
The graphical user interface
KVM virtual machines
Linux containers, or LXC
Storage plugins
Vibrant culture
The basic installation of Proxmox
The advanced installation option
Debugging the Proxmox installation
Proxmox subscription and repositories
Proxmox VE Enterprise repository
Type
Subscription key
Status
Server ID
Sockets
Last checked
Next due date
Proxmox VE No-Subscription repository
Proxmox VE Test repository
Summary
Creating a Cluster and Exploring the Proxmox GUI
Creating a Proxmox cluster
Exploring the Proxmox GUI
The GUI menu system
Cluster tree view
Server View
Folder View
Storage View
Pool View
The Datacenter menu
Datacenter | Search
Datacenter | Summary
Datacenter | Options
Datacenter | Storage
Datacenter | Backup
Datacenter | Permissions
Datacenter | Permissions | Users
Datacenter | Permissions | Groups
Datacenter | Permissions | Pools
Datacenter | Permissions | Roles
Datacenter | Permissions | Authentication
Datacenter | HA
Datacenter | Firewall
Datacenter | Support
Node-specific menus
Node | Search
Node | Summary
Node | Shell
Node | System
Node | Network
Node | DNS
Node | Time
Node | Syslog
Node | Updates
Node | Firewall
Node | Disks
Node | Ceph
Node | Task History
Node | Subscription
KVM menu
KVM VM | Summary
KVM | Console
KVM | Hardware
KVM | Options
KVM VM | Task History
KVM | Monitor
KVM | Backup
KVM VM | Snapshot
KVM | Firewall
KVM | Permissions
LXC container menu
LXC container | Summary
LXC container | Resources
LXC container | Network
LXC container | DNS
LXC container | Options
LXC container | Task History
LXC container | Backup
LXC container | Snapshots
LXC container | Firewall
LXC container | Permissions
Pool menu
Pool | Summary
Pool | Members
Pool | Permissions
Summary
Proxmox under the Hood
The Proxmox cluster file system
Proxmox directory structure
Dissecting the configuration files
The cluster configuration file
logging { }
nodelist { }
quorum { }
totem { }
interface { }
Storage configuration file
User configuration files
The password configuration file
KVM virtual machine configuration file
Arguments in the KVM configuration file
LXC container configuration file
Version configuration file
Member nodes
Virtual machine list file
The cluster log file
Ceph configuration files
Firewall configuration file
Summary
Storage Systems
Local storage versus shared storage
Live migration of a virtual machine
Seamless expansion of multinode storage space
Centralized backup
Multilevel data tiering
Central storage management
Local and shared storage comparison
A virtual disk image
Supported image formats
The .qcow2 images
The .raw image type
The .vmdk image type
Virtual device types
Managing disk images
Resizing a virtual disk image
Moving a virtual disk image
Throttling a virtual disk image
Caching a virtual disk image
VirtIO bus type for Windows VMs
Installing VirtIO drivers during Windows installation
Installing VirtIO drivers after Windows installation
Storage types in Proxmox
Directory
iSCSI
Logical Volume Management
NFS
ZFS
Ceph RBD
GlusterFS
Noncommercial/commercial storage options
Summary
Installing and Configuring Ceph
Ceph components
A physical node as cluster member
Maps
A cluster map
A CRUSH map
Monitor
OSD
OSD journal
Metadata server
PG
Pools
Ceph components summary
Virtual Ceph for training
Installing a Ceph cluster
Installing Ceph on Proxmox
Preparing a Proxmox node for Ceph
Installing Ceph
Creating mons from the Proxmox GUI
Creating OSDs from Proxmox GUI
Managing a Ceph pool using Proxmox GUI
Creating a Ceph pool using Proxmox GUI
Connecting Ceph to Proxmox
Ceph command list
Summary
KVM Virtual Machines
Exploring KVM
Creating a KVM
Creating a KVM using an ISO image
General tab
Node
VM ID
Name
Resource Pool
Help
The OS tab
The CD/DVD tab
The Hard Disk tab
Bus/Device
Storage
Disk size (GB)
Format
Cache
No backup
Discard
IO thread
The CPU tab
Sockets
Cores
Enabling NUMA
Type
The Memory tab
The Network tab
Bridged mode
Firewall
NAT mode
No network device
Model
MAC address
Rate limit (MB/s)
Multiqueues
Disconnect
Creating VM by cloning
Creating VMs from a template
Target node
Mode
Advanced configuration options for VMs
Configuring a sound device
Configuring PCI passthrough
Configuring GPU passthrough
Preparing for hotplug
Configuring VMs with hotplug
Hotplugging vCPUs
Hotplugging memory
Hotplugging disks/vNICs
Migrating KVM virtual machines
Summary
LXC Virtual Machines
Exploring LXC virtual machines
Understanding container templates
Creating an LXC container
General tab
Node
CT ID
Hostname
Unprivileged container
Resource Pool
The Template tab
The Root Disk tab
Storage
ACLs
Enable quota
The CPU tab
Cores
The Memory tab
The Network tab
Name
MAC address
Bridge
The VLAN Tag
Rate limit
Firewall
IPv4/IPv6
The DNS tab
The Confirm tab
Managing an LXC container
Adjusting resources using the GUI
Adjusting resources using the CLI
Adjusting resources using direct modification
Migrating an LXC container
Accessing an LXC container
The noVNC console
Direct shell through the CLI
Converting OpenVZ to LXC
Summary
Network of Virtual Networks
Exploring virtual networks
Physical networks versus virtual networks
A physical network
A virtual network
Networking components in Proxmox
Virtual Network Interface Cards
Adding/removing vNIC
A virtual bridge
Adding a virtual bridge through the GUI
Name
IP information
Bridge ports
VLAN-aware
Adding a virtual bridge through CLI
Extra bridge options
bridge_stp
bridge_fd
Virtual LAN
Adding a VLAN
Network Address Translation/Translator
Adding NAT/masquerading
Network bonding
Adding a bonding interface
The layer 2 hash policy
The layer 2+3 hash policy
The layer 3+4 hash policy
Multicast
Configuring multicast on Netgear
Open vSwitch
Features of Open vSwitch
Adding an Open vSwitch bridge
Adding the Open vSwitch bond
Adding Open vSwitch IntPort
CLI for Open vSwitch
Practicing Open vSwitch
Configuration requirements
Solutions
Sample virtual networks
Network #1 – Proxmox in its simplest form
Network #2 – the multi-tenant environment
Network #3 – academic institution
A multi-tenant virtual environment
A multi-tenant network diagram
Summary
The Proxmox VE Firewall
Exploring the Proxmox VE firewall
Components of the Proxmox firewall
Zones
Security groups
IPSet
Rules
Protocols
Macros
The pve-firewall and pvefw-logger services
Configuration files of a firewall
Configuring the data center-specific firewall
Configuring the Datacenter firewall through the GUI
Creating the Datacenter firewall rules
Creating the Datacenter IPSet
Creating aliases
Configuring the Datacenter firewall through the CLI
[OPTIONS]
[ALIASES]
[IPSET <name>]
[RULES]
[group <name>]
Configuring a host-specific firewall
Creating host firewall rules
Options for the host zone firewall
Enable a firewall
The SMURFS filter
The TCP flags filter
NDP
nf_conntrack_max
nf_conntrack_tcp_timeout_established
log_level_in/out
tcp_flags_log_level
smurf_log_level
Configuring the host firewall through the CLI
Configuring a VM-specific firewall
Creating VM firewall rules
Creating aliases
Creating IPSets
Options for a VM zone firewall
Enable DHCP
The MAC filter
Input/output policy
Configuring a VM-specific firewall through the CLI
Integrating a Suricata IDS/IPS
Installing/configuring Suricata
Limitations of Suricata in Proxmox
Summary
Proxmox High Availability
Understanding HA
HA in Proxmox
How Proxmox HA works
Requirements for HA setup
At least three nodes
Shared storage
Fencing
BIOS power-on feature
Configuring Proxmox HA
The HA menu
Status
The Resources menu
The Groups menu
ID
Node
The restricted checkbox
The nofailback checkbox
The Fencing menu
Testing Proxmox HA configuration
The Proxmox HA simulator
Configuring the Proxmox HA simulator
Summary
Monitoring the Proxmox Cluster
An introduction to monitoring
Proxmox built-in monitoring
Datacenter Status
Node Status
Zabbix as a monitoring solution
Installing Zabbix
Configuring Zabbix
Configuring a host to monitor
Displaying data using a graph
Configuring the disk health notification
Installing smart monitor tools
Configuring the Zabbix agent
Creating a Zabbix item in the GUI
Creating a trigger in the GUI
Creating graphs in the GUI
Configuring SNMP in Proxmox
Object Identifiers
Management Information Base
Adding an SNMP device in Zabbix
Monitoring the Ceph cluster with the Proxmox GUI
Monitoring a Ceph cluster with third-party options
Summary
Proxmox Production-Level Setup
Defining the production level
Key components
Stable and scalable hardware
Redundancy
Node level
Utility level
Network level
HVAC level
Storage level
Current load versus future growth
Budget
Simplicity
Tracking hardware inventory
Hardware selection
Sizing CPU and memory
Single socket versus multi-socket
Hyper-threading – enable versus disable
Start small with VM resources
Balancing node resources
Ceph cluster production
Forget about hardware RAID
Solid State Drive for Ceph Journal
Network bandwidth
Liquid cooling
Total immersion in oil
Total immersion in 3M Novec
Direct contact liquid cooling
Real-world Proxmox scenarios
Scenario 1 – an academic institution
Scenario 2 – multi-tier storage cluster with a Proxmox cluster
Scenario 3 - Virtual infrastructure for a multi-tenant cloud service provider
Scenario 4 – nested virtual environment for a software development company
Scenario 5 – virtual infrastructure for a public library
Scenario 6 – multi-floor office virtual infrastructure with virtual desktops
Scenario 7 – virtual infrastructure for the hotel industry
Scenario 8 – virtual infrastructure for geological survey organization
Summary
Back Up and Restore Virtual Machines
Proxmox backup options
A full backup
Full backup modes
Snapshot
Suspend
Stop
Backup compression
None
LZO
GZIP
Snapshots
Configuring backup storage
Show VM configuration from backup
Configuring full backup
Creating a schedule for backup
Node
Storage
Day of week
Start Time
Selection mode
Send email to
Email notification
Compression
Mode
Enable
Creating a manual backup
Creating snapshots
Restoring a virtual machine
Backup/restore through the CLI
Backup using the CLI
Restore using the CLI
Unlocking a VM after a backup error
Virtual machine replication
Creating a replication task through the GUI
Target
Schedule
Rate limit (MB/s)
Enabled
Creating a replication task through the CLI
Replication process
Backup configuration file
The bwlimit option
The lockwait option
The stopwait option
The stdexcludes option
The mailto option
The script option
The exclude-path option
The pigz option
Summary
Updating/Upgrading Proxmox
Introducing Proxmox updates
Updating Proxmox through the GUI
Updating Proxmox through the CLI
Difference between upgrade and dist-upgrade
Recovering from the grub2 update issue
Updating after a subscription change
Rebooting dilemma after Proxmox updates
Applying update without reboot
Summary
Proxmox Troubleshooting
Proxmox node issues
Issue – fresh Proxmox install stuck with /dev to be a fully populated error during node reboot
Issue – rejoining a node to a Proxmox node with the same old IP address
Issue – Proxmox installation completed but grub is in an endless loop after reboot
Issue – LSI MegaRAID 9240-8i/9240-4i causes an error during booting of the Proxmox node
Downloading and updating the LSI driver
Updating the Supermicro BIOS
Issue – the Upgrade button is disabled on the Proxmox GUI, which prevents the node upgrade
Issue – Proxmox cannot start due to the getpwnam error
Issue – cannot log in to the GUI as root after reinstalling Proxmox on the same node
The main cluster issues
Issue – Proxmox virtual machines are running, but the Proxmox GUI shows that everything is offline
Issue – kernel panic when disconnecting USB devices, such as a keyboard, mouse, or UPS
Issue – virtual machines on Proxmox will not shut down if shutdown is initiated from the Proxmox GUI
Issue – kernel panic with HP NC360T (Intel 82571EB chipset) only in Proxmox VE 3.2
Issue – the Proxmox cluster is out of quorum and cluster filesystem is in read-only mode
Issue – VM will not respond to shutdown or restart
Issue – Proxmox GUI not responding after Firefox update
Issue – the Proxmox GUI is not showing RRD graphs
Storage issues
Issue – deleting a damaged LVM from Proxmox with the error read failed from 0 to 4096
Issue – Proxmox cannot mount NFS share due to the timing out error
Issue – how to delete leftover NFS shares in Proxmox or what to do when the NFS stale file handle error occurs?
Issue – Proxmox issues --mode session exit code 21 errors while trying to access the iSCSI target
Issue – cannot read an iSCSI target even after it has been deleted from Proxmox storage
Issue – a Ceph node is removed from the Proxmox cluster, but OSDs still show up in PVE
Issue – the no such block device error during creation of an OSD through the Proxmox GUI
Issue – the fstrim command does not trim unused blocks for the Ceph storage
Issue – the RBD couldn't connect to cluster (500) error when connecting Ceph with Proxmox
Issue – changing the storage type from IDE to VirtIO after the VM has been set up and the OS has been installed
Issue – the pveceph configuration not initialized (500) error when you click on the Ceph tab in the Proxmox GUI
Issue – the CephFS storage disappears after a Proxmox node reboots
Issue – VM cloning does not parse in the Ceph storage
Issue – VM disk images stored on ZFS is extremely slow
Network connectivity issues
Issue – no connectivity on Realtek RTL8111/8411 rev. 06 network interfaces
Issue – network performance is slower with the E1000 virtual network interfaces
Issue – patch port for Open vSwitch in Proxmox not working
Issue – trying to add a node to a newly created Proxmox cluster when nodes do not form quorum
Issue – implemented IPv6 but firewall rules do not get applied
KVM virtual machine issues
Issue – Windows 7/XP machine converted to Proxmox KVM hangs during boot
Issue – Windows 7 VM does not reboot, instead it shuts down, requiring a manual boot from Proxmox
Issue – the qemu-img command does not convert the .vmdk image files created with the .ova template in Proxmox VE 5.0
Issue – online migration of a virtual machine fails with a failed to sync data error
Issue – no audio in Windows KVM
Issue – the VirtIO virtual disk is not available during the Windows Server installation
LXC container issues
Issue – a Proxmox node hangs when trying to stop or restart an LXC container
Issue – the noVNC console only shows a cursor for LXC containers
Backup/restore issues
Issue – a Proxmox VM is locked after backup crashes unexpectedly
Issue – how can Proxmox back up only the primary OS virtual disk instead of all the virtual disks for a VM?
Issue – backup of virtual machines stops prematurely with an operation not permitted error
Issue – a backup task takes a very long time to complete, or it crashes when multiple nodes are backing up to the same backup storage
Issue – backup of virtual machines aborts a backup task prematurely
Issue – backup storage has a lot of .dat files and .tmp folders using the storage space
VNC/SPICE console issues
Issue – the mouse pointer is not shared with SPICE (virt-viewer) on Windows 8 VM
Issue – remote viewer is unable to connect to a SPICE-enabled virtual machine on the Windows OS
Firewall issues
Issue – rules are created and a firewall is enabled for vNIC, but rules do not get applied
Issue – a firewall is enabled for a VM and the necessary rules are created, but nothing is being filtered for that VM
Summary
Rescuing Proxmox
Recovering from OS drive failure
Physical drive failure
OS data corruption
Migrating VMs from a faulty node
Reinstalling Proxmox
Recovering from a quorum failure
Recovering from a node failure
Recovering from a network failure
Loss of connectivity between Proxmox nodes
Loss of connectivity between Proxmox nodes and users
Loss of connectivity between Proxmox and storage nodes
Recovering from Ceph failure
Best practices for a healthy Ceph cluster
Stuck inconsistent PGs in Ceph
Stuck inactive incomplete PGs in Ceph
Error while moving a Ceph journal to another drive
Ceph node running out of resources during recovery
Summary
Preface
Based on the foundation laid out by the first edition and second edition, this book, Mastering Proxmox, Third Edition, brings updated information and details of the new features of Proxmox VE 5.0. Since the first edition of this book was published, Proxmox has been through many changes. With this third edition, I am confident that readers will be able to upgrade their skills while building and managing even better Proxmox clusters. This book shows the inner workings of Proxmox, including virtual network components, shared storage systems, the Proxmox firewall, high availability, and other features.
What this book covers
Chapter 1, Understanding Proxmox VE and Advanced Installation, introduces Proxmox VE in general and shows the advanced options available during installation.
Chapter 2, Creating a Cluster and Exploring the Proxmox GUI, explains how to create a cluster and shows the layout of the graphical user interface.
Chapter 3, Proxmox under the Hood, explains the Proxmox directory structure and configuration files.
Chapter 4, Storage Systems, explains how Proxmox interacts with storage and various supported storage systems.
Chapter 5, Installing and Configuring Ceph, shows how to deploy and configure a fully functional Ceph cluster along with Proxmox.
Chapter 6, KVM Virtual Machines, covers creating and managing KVM-based virtual machines.
Chapter 7, LXC Virtual Machines, covers creating and managing LXC containers.
Chapter 8, Network of Virtual Networks, explains the different networking components used in Proxmox to build virtual networks.
Chapter 9, The Proxmox VE Firewall, explains the built-in firewall feature of Proxmox.
Chapter 10, Proxmox High Availability, explains the high availability or redundancy feature of Proxmox and how to configure it.
Chapter 11, Monitoring the Proxmox Cluster, shows how to configure the Zabbix-based network monitoring option.
Chapter 12, Proxmox Production-Level Setup, explains different components in a production-level setup.
Chapter 13, Back Up and Restore Virtual Machines, explains the backup and restore features of Proxmox for disaster planning.
Chapter 14, Updating/Upgrading Proxmox, explains how to keep a Proxmox cluster up to date.
Chapter 15, Proxmox Troubleshooting, lists real incidents that may arise in a Proxmox cluster, with solutions.
Chapter 16, Rescuing Proxmox, shows ways to rescue a Proxmox cluster should a disaster occur.
What you need for this book
Since we will be working with a Proxmox cluster throughout the book, it will be extremely helpful to have a working Proxmox cluster of your own. A very basic cluster of two to three nodes, along with a storage node, will do just fine. If learning to implement Ceph in a Proxmox cluster, then a small cluster of two or three nodes for Ceph will also be extremely helpful.
Who this book is for
This book is for readers who want to build and manage a virtual infrastructure based on Proxmox as the hypervisor. Whether the reader is a veteran in the virtualized industry but has never worked with Proxmox, or somebody is just starting out on a promising career in this industry, this book will serve them well. Due to the advanced nature of this book, prior conceptual knowledge of server virtualization, networking, and hypervisors is required.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning. Code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "The keyring that we need to copy is located in/priv/ceph.client.admin.keyring."
A block of code is set as follows:
allow-vmbr1 ens21 iface ens21 inet manual ovs_type OVSPort ovs_bridge vmbr1
Any command-line input or output is written as follows:
# apt-get install openvswitch-switch
New terms and important words are shown in bold. Words that you see on the screen, for example, in menus or dialog boxes, appear in the text like this: "Open vSwitch bridge and interface under the Create tab of the Network menu of the node."
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply email [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/MasteringProxmoxThirdEdition_ColorImages.pdf.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title. To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at [email protected] with a link to the suspected pirated material. We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Understanding Proxmox VE and Advanced Installation
Virtualization, as we all know today, is a decades-old technology that was first implemented in the mainframes of the 1960s. Virtualization was a way to logically divide the mainframe's resources for different application processing. With the rise in energy costs, running under-utilized server hardware is no longer a luxury. Virtualization enables us to do more with less, thus saving energy and money while creating a virtual green data center without geographical boundaries.
A hypervisor is a piece of software, hardware, or firmware that creates and manages virtual machines. It is the underlying platform or foundation that allows a virtual infrastructure to be built. In a way, it is the very building block of all virtualization. A bare metal hypervisor acts as a bridge between physical hardware and the virtual machines by creating an abstraction layer. Because of this unique feature, an entire virtual machine can be moved over a vast distance over the internet and be made available to function exactly the same. A virtual machine does not see the hardware directly; instead, it sees the layer of the hypervisor, which is the same no matter what hardware the hypervisor has been installed on.
The Proxmox Virtual Environment (VE) is a cluster-based hypervisor and one of the best-kept secrets in the virtualization industry. The reason is simple. It allows you to build an enterprise business-class virtual infrastructure at a small business-class price tag without sacrificing stability, performance, and ease of use. Whether it is a massive data center to serve millions of people, or a small educational institution, or a home serving important family members, Proxmox can handle configuration to suit any situation.
If you have picked up this book, you are no doubt familiar with virtualization, and perhaps well versed with other hypervisors, such as VMware, Xen, Hyper-V, and so on. In this chapter and upcoming chapters, we will see the mighty power of Proxmox from the inside out. We will examine scenarios and create a complex virtual environment. We will tackle some heavy day-to-day issues and show resolutions that might just save the day in a production environment. We will also learn how to deploy a highly redundant storage system using Ceph to store virtual machines. So strap yourself in and let's dive into the virtual world with the mighty hypervisor, Proxmox VE.
Understanding Proxmox features
Before we dive in, it is necessary to understand why one should choose Proxmox over the other mainstream hypervisors. Proxmox is not perfect, but stands out among other contenders with its hard-to-beat features. The following are some of the features that make Proxmox a real game changer.
It is free!
Yes, Proxmox is free! To be more accurate, Proxmox has several subscription levels, among which the community edition is completely free. One can simply download the Proxmox ISO at no cost and raise a fully functional cluster without missing a single hypervisor feature and without paying anything. The main difference between the paid and community subscription level is that the paid subscription receives updates, which go through additional testing and refinement. In a production cluster with a real workload, it is highly recommended to purchase a subscription from Proxmox or Proxmox resellers.
Built-in firewall
Proxmox VE comes with a robust firewall ready to be configured out of the box. This firewall can be configured to protect the entire Proxmox cluster down to a virtual machine. The per-VM firewall option gives you the ability to configure each VM individually by creating individualized firewall rules, a prominent feature in a multi-tenant virtual environment. We will learn about this feature in detail in Chapter 9, The Proxmox VE Firewall.
Open vSwitch
Licensed under Apache 2.0, Open vSwitch is a virtual switch designed to work in a multi-server virtual environment. All hypervisors need a bridge between VMs and the outside network. Open vSwitch enhances the features of the standard Linux bridge in an ever-changing virtual environment. Proxmox fully supports Open vSwitch which allows you to create an intricate virtual environment, all the while reducing virtual network management overhead. For details on Open vSwitch, refer to http://openvswitch.org/.
We will learn about Open vSwitch management in Proxmox in Chapter 8, Network of Virtual Networks.
The graphical user interface
Proxmox comes with a fully functional graphical user interface (GUI) out of the box. The GUI allows an administrator to manage and configure almost all the aspects of a Proxmox cluster. The GUI has been designed keeping simplicity in mind, with functions and features separated into menus for easier navigation. The following screenshot shows an example of the Proxmox GUI dashboard:
We will dissect the Proxmox GUI dashboard in Chapter 2, Creating a Cluster and Exploring the Proxmox GUI.
KVM virtual machines
A Kernel-based Virtual Machine (KVM) is a kernel module that is added to Linux for full virtualization to create isolated, fully independent virtual machines. KVMs are not dependent on the host operating system in any way, but they do require the virtualization feature in BIOS to be enabled. A KVM allows a wide variety of operating systems for virtual machines, such as Linux and Windows. Proxmox provides a very stable environment for KVM-based VMs. We will learn how to create KVM VMs and also how to manage them in Chapter 6, KVM Virtual Machines.
Linux containers, or LXC
Introduced in Proxmox VE 4.0, Linux containers, or LXCs, allow multiple Linux instances on the same Linux host. All the containers are dependent on the host Linux operating system and only Linux flavors can be virtualized as containers. There are no containers for the Windows operating system. LXC replaces prior OpenVZ containers, which were the primary containers in the virtualization method in the previous Proxmox versions. If you are not familiar with LXC or want details on it, refer to https://linuxcontainers.org.
We will learn how to create LXC containers and manage them in Chapter 7, LXC Virtual Machines.
Storage plugins
Out of the box, Proxmox VE supports a variety of storage systems to store virtual disk images, ISO templates, backups, and so on. All plugins are quite stable and work great with Proxmox. Being able to choose different storage systems gives an administrator the flexibility to leverage the existing storage in the network. As of Proxmox VE 5.0, the following storage plugins are supported:
The local directory mount points
LVM
LVM thin
NFS
iSCSI
GlusterFS
Ceph
RADOS Block Devices
(
RBD
)
ZFS over iSCSI
ZFS
We will learn the usage of different storage systems and the types of files they can store in detail in Chapter 4, Storage Systems.
Vibrant culture
Proxmox has a growing community of users who are always helping others learn Proxmox and troubleshoot various issues. With so many active users around the world, and through active participation of Proxmox developers, the community has now become a culture of its own. Feature requests are continuously being worked on, and the existing features are being strengthened on a regular basis. With so many users supporting Proxmox, it sure is here to stay.
The basic installation of Proxmox
The installation of a Proxmox node is very straightforward. Simply accept the default options, select localization, and enter the network information to install Proxmox VE. We can summarize the installation process in the following steps:
Download the ISO from the official Proxmox site and prepare a disc with the image (
http://proxmox.com/en/downloads
)
.
Boot the node with the disc and hit
Enter
to start the installation from the installation GUI, as shown in the following screenshot:
If an optical drive to use the installation disc is unavailable, we can also install Proxmox from a USB drive.
Progress through the prompts to select options or type in information.
After the installation is complete, access the Proxmox GUI dashboard using the IP address, as
https://<proxmox_node_ip>:8006
.
In some cases, it may be necessary to open the firewall port to allow access to the GUI over port 8006.
The advanced installation option
Although the basic installation works in all scenarios, there may be times when the advanced installation option is necessary. Only the advanced installation option provides you the ability to customize the main OS drive.
A common practice for the operating system drive is to use a mirror RAID array using a controller interface. This provides drive redundancy if one of the drives fails. This same level of redundancy can also be achieved using a software-based RAID array, such as ZFS. Proxmox now offers options to select ZFS-based arrays for the operating system drive right at the beginning of the installation. For details on ZFS, if you are not familiar, refer to https://en.wikipedia.org/wiki/ZFS.
Besides ZFS, we can also select other filesystem types, such as ext3, ext4, or xfs, from the same advanced option. We can also set the custom disk or partition sizes through the advanced option. The following screenshot shows the installation interface with the target hard disk selection page:
Click on Options, as shown in the preceding screenshot, to open the advanced options for the hard disk. The following screenshot shows the option window with supported filesystem drop-down menu:
We are going to select the ZFS mirror or RAID1, for the purpose of this book, in order to create a demo cluster from scratch. In the preceding screenshot, we selected zfs (RAID1) for mirroring, and the two drives, Harddisk 0 and Harddisk 1, to install Proxmox. The installer will auto-select the installed disk drive, as shown in the following screenshot:
The Advanced Options include some ZFS performance-related configurations such as compress, checksum, and ashift or alignment shift, as shown in the following screenshot:
For most environments, this configuration can be left as default.
If you are unfamiliar with ZFS advanced tuning, then the following link may be helpful to get some insight on ZFS performance tuning options:
http://open-zfs.org/wiki/Performance_tuning#Alignment_Shift_.28ashift.29
If we pick a filesystem such as EXT3, EXT4, or XFS instead of ZFS, the Harddisk options dialog box will look like the following screenshot, with a different set of options:
Selecting a filesystem gives us the following advanced options:
hdsize
: This is the total drive size to be used by the Proxmox installation.
swapsize
: This defines the swap partition size.
maxroot
: This defines the maximum size to be used by the root partition.
minfree
: This defines the minimum free space that should remain after the Proxmox installation.
maxvz
: This defines the maximum size for the data partition. This is usually
/var/lib/vz
.
From Proxmox VE version 5, we can select the interface that will be used for management. This is very useful when a node has multiple network interfaces and we want to intentionally use a particular interface for cluster management. The following screenshot shows the management network interface selection screen during Proxmox installation:
Debugging the Proxmox installation
Debugging features are part of any good operating system. Proxmox has debugging features that will help you during a failed installation. Some common reasons are unsupported hardware, conflicts between devices, ISO image errors, and so on. Debugging mode logs and displays installation activities in real time. When the standard installation fails, we can start the Proxmox installation in debug mode from the main installation interface, as shown in the following screenshot:
The debug installation mode will drop us in the prompt, as shown in the following screenshot:
To start the installation, we need to press Ctrl + D. If there is an error during the installation, we can simply press Ctrl + C to get back to this console to continue with our investigation. From the console, we can check the installation log using the following command:
# cat /tmp/install.log
At times, it may be necessary to edit the loader information when normal booting does not function. This is a common case when Proxmox is unable to show the video output due to UEFI or a nonsupported resolution. In such cases, the booting process may hang. From the main installation menu, we can press E to enter edit mode to change the loader information, as shown in the following screenshot:
One way to continue with booting is to add the nomodeset argument by editing the loader. The loader should look as follows after the edit:
linux/boot/linux26 ro ramdisk_size=16777216 rw quiet nomodeset
Proxmox subscription and repositories
Proxmox itself is completely free to download and deploy without any cost. But a subscription offers an added level of stability to any node used in a production environment. Both free and subscribed versions have separate repositories and receive updates differently.
Updates or packages released through the subscribed or Enterprise repository go through additional testing and debugging before they are released. This is not to say the updates or packages in the free repository are full of bugs and are released without testing. All Proxmox patches, updates, and packages are taken through the complete development cycle, including testing, before they are released. But Enterprise packages go through much more comprehensive debugging and testing. This level of tests is mandatory for an enterprise-class network environment where a small issue can cost a company a lot of money. A highly stable environment is usually not needed in a home-based platform or small business environment. The subscription menu allows you to activate a purchased subscription on a node. So from a stability point of view, the enterprise version is without a doubt the best choice for any production environment cluster. The price of an enterprise subscription varies depending on the level of Proxmox support provided through tickets, portal, and phone.
Free repository users can only reach out for support through the official Proxmox forum. Proxmox developers quite often lend their expertise to address issues posted on the forum by users. There is no portal or ticket system available for free users. Since this is a free community forum, some issues may not get answered in time.
Both free and enterprise versions can be mixed in the same environment. For example, some critical nodes actively serving users can be on the enterprise version, while any non-critical nodes, such as nodes used for testing, backup, and so on, can be on the free version. Upon logging in through the free non-subscription Proxmox node through the GUI, we will be presented with the following notification:
There are three package repositories for Proxmox:
Proxmox VE Enterprise repository
Proxmox VE No-Subscription or Free repository
Proxmox VE Test repository
Proxmox VE Enterprise repository
As the name suggests, this repository is for nodes with paid subscriptions. By default, the Enterprise Repository is enabled in Proxmox. The repository information is in the file /etc/apt/sources.list.d/pve-enterprise.list. We can disable the Enterprise Repository by simply commenting it out with the #symbol in the following line:
deb https://enterprise.proxmox.com/debian jessie pve-enterprise
When disabling the Enterprise Repository, the No-Subscription Repository must be enabled in order to receive updates, patches, and packages. If you're using the Enterprise Repository on a mission-critical node and a subscription has been purchased, the subscription key can be uploaded through the Proxmox GUI by clicking on the Upload Subscription Key button under the Node | Subscription menu, as shown in the following figure:
Copy and paste the subscription key and then click on OK. Proxmox will automatically check the validity of the key and activate the subscription for the node. A fully subscribed node appears similarly to the following screenshot, under subscriptions in the GUI:
Let's look at the details provided through the Subscription page.
Type
This shows the name of the Proxmox subscription level. There are four levels of subscription available: Community, Basic, Standard, and Premium. The higher the level, the more support add-ons are included.
Subscription key
This is the alphanumeric subscription key the customer receives after purchasing any subscription. The key is formatted in two parts: pveXx-XXXXXXXXXX. The first portion of the key indicates which level of subscription this key belongs to and for how many server sockets. For example, in the previous screenshot, the subscription key is for a Community-level subscription for a server with two sockets. If this were the Premium-level subscription for a server with four sockets, the key would appear as pve4p-XXXXXXXX.
All letters and numbers after the - are unique to each key and should not be shared with unauthorized personnel or made public.
Status
This shows the current status of the subscription key.
Server ID
This uniquely generated ID belongs to one node only. When a subscription key is activated on a particular server, the key gets associated with this unique ID. When a node needs to be reinstalled without any hardware changes in it, the key can be reapplied to the server without being reissued or reactivated. But if the key is to be applied to other server hardware or if any major component (such as the CPU, motherboard, or memory) in the server has been changed, then a new unique ID will be generated. In that case, the key will need to be reissued or reactivated. This reissuing can be done by the user on the Proxmox customer site or by the authorized reseller from whom the subscription key has been purchased.
Sockets
This shows the physical CPU socket count of the server node.
Last checked
This shows the date and time of the last key validation check performed automatically by the node or manually by the user.
Next due date
This shows the expiration date of the subscription key, by which the key needs to be renewed. If the key is not renewed and expires, the Proxmox node will still continue to function properly. But it will not receive any updates from the Enterprise Repository.
Proxmox VE No-Subscription repository
This repository includes updates and packages free of cost. If using this repository, changes must be made to activate it. After disabling the Enterprise Repository, by following the instructions in the previous section, add the following line to the file /etc/apt/sources.list:
deb http://download.proxmox.com/debian jessie pve-no-subscription
Proxmox VE Test repository
This repository largely contains packages for testing purposes only. It is mainly used by Proxmox developers to test new packages and allow interested users to test them as well. Under no circumstances should this repository be used in a production environment. To enable this repository, add the following line to /etc/apt/sources.list:
deb http://download.proxmox.com/debian jessie pvetest
Summary
In this chapter, we looked at why Proxmox is a better option as a hypervisor, what advanced installation options are available during an installation, and why we choose software RAID for the operating system drive. We also looked at different subscription levels and their benefits. We learned about the presence of the debugging features to investigate when an installation does not proceed as usual.
In next chapter, we will take a closer look at the Proxmox GUI and see how easy it is to centrally manage a Proxmox cluster from a web browser.
Creating a Cluster and Exploring the Proxmox GUI
Proxmox VE can be used independently without being part of a cluster. But in order to truly use Proxmox at its full potential, a cluster enables many more advanced features such as centralized management, high availability, and live migration. We will look into the features in later chapters. When multiple Proxmox nodes are in the same cluster, they can all be managed and monitored by logging in to the Proxmox GUI through any member node. There is no master-slave scheme in Proxmox. All nodes works together by sharing the same configuration.
Creating a Proxmox cluster
A cluster is nothing but a group of Proxmox servers or nodes, sharing resources. A Proxmox cluster can contain up to 32 physical nodes. If network latency permits, the number of nodes can be higher. But any number of nodes higher than 32 may cause an unstable situation within the cluster.
As of Proxmox VE 5, we cannot create clusters through the graphical interface. The entire process of cluster creation must be done through the CLI. Proxmox provides a tool to create and add nodes to a cluster called Proxmox VE Cluster Manager or pvecm.
To create a new cluster, log in to any available Proxmox node through SSH and run the following command:
# pvecm create <clustername>
For our first demo cluster, we are going to run the following command to create a cluster named pmx-cluster:
# pvecm create pmx-cluster
After successfully creating the cluster, we can quickly check it through the following command:
# pvecm status
The following screenshot shows the result after running the pvecm command:
As shown in the previous screenshot, we have created a new cluster from node 1. We are now going to add a second node into the cluster. To add a member node, log in to the node through SSH, and then run the following command:
# pvecm add <existing_member_ip>
If there is more than one member node in the cluster already, then the IP address in the command can be any of those nodes. As mentioned earlier, there is no master-slave scheme in a Proxmox cluster. All nodes share the same cluster configuration and information. For our demo cluster, we are going to add our second node into the cluster using the following command, where 172.16.2.1 is the assigned IP address of the first node in the cluster:
# pvecm add 172.16.2.1
The command will initiate the process of adding the node into the cluster and will display results as it progresses. The command also starts or restarts necessary services. The only user prompt that is necessary in the beginning of the process is to enter the destination node's root credentials. The following screenshot shows the command to add a node and the process it progresses through:
Sometimes it may be necessary to rejoin a member node with the same hostname and IP address into the cluster for any number of reasons, such as a hostname change or reinstall. The node-joining command will produce an error, as shown in the following screenshot, if the node has the same network information as it had previously:
The reason this error occurs is the cluster configuration already has a node listed in it with the same hostname and IP address. In such cases, we can add an option at the end of the node-joining command as follows:
# pvecm add <existing_mode_ip> -f
The command will forcefully rewrite the cluster configuration, recreate the SSH authentication key, and join the member node. We can see the list of member nodes in the cluster using the following command:
# pvecm nodes
We can also use the pvecm command to remove or detach a member node from the cluster. This command should be run from any node in the cluster except from the node being detached.
The following command will remove a node from the Proxmox cluster:
# pvecm delnode <hostname/IP>
Exploring the Proxmox GUI
The Proxmox GUI allows users to interact with the Proxmox cluster graphically using menus and a visual representation of the cluster status. Even though all of the management can be done from the CLI, it can be overwhelming at times, and managing a cluster can become a daunting task. To properly utilize a Proxmox cluster, it is very important to have a clear understanding of the Proxmox GUI. The GUI can be accessed through any member nodes in the cluster. From Proxmox VE 4.2, the GUI has been updated to Sencha Ext JS 6, adding a new level of cluster visibility along with aesthetic appeal. We can now gather a lot more, at-a-glance data while managing more details through the GUI.
In this chapter, we are going to explore the different parts of the Proxmox web GUI, such as how the menu system is organized and the menus' functions. The GUI can be easily accessed from just about any browser though a URL similar to https://<node_ip>:8006. For our demo cluster, we are going to access the GUI through the link: https://172.16.2.1:8006.
The following screenshot shows an example of the Proxmox GUI for our demo cluster:
