3D Graphics Rendering Cookbook E-Book

Sergey Kosarevsky

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Veröffentlichungsjahr: 2021

Beschreibung

OpenGL is a popular cross-language, cross-platform application programming interface (API) used for rendering 2D and 3D graphics, while Vulkan is a low-overhead, cross-platform 3D graphics API that targets high-performance applications. 3D Graphics Rendering Cookbook helps you learn about modern graphics rendering algorithms and techniques using C++ programming along with OpenGL and Vulkan APIs.
The book begins by setting up a development environment and takes you through the steps involved in building a 3D rendering engine with the help of basic, yet self-contained, recipes. Each recipe will enable you to incrementally add features to your codebase and show you how to integrate different 3D rendering techniques and algorithms into one large project. You'll also get to grips with core techniques such as physically based rendering, image-based rendering, and CPU/GPU geometry culling, to name a few. As you advance, you'll explore common techniques and solutions that will help you to work with large datasets for 2D and 3D rendering. Finally, you'll discover how to apply optimization techniques to build performant and feature-rich graphics applications.
By the end of this 3D rendering book, you'll have gained an improved understanding of best practices used in modern graphics APIs and be able to create fast and versatile 3D rendering frameworks.

Leseprobe

3D Graphics Rendering Cookbook

A comprehensive guide to exploring rendering algorithms in modern OpenGL and Vulkan

Sergey Kosarevsky

Viktor Latypov

BIRMINGHAM—MUMBAI

3D Graphics Rendering Cookbook

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Associate Group Product Manager: Pavan Ramchandani

Publishing Product Manager: Ashitosh Gupta

Senior Editor: Mark Dsouza

Content Development Editor: Divya Vijayan

Technical Editor: Joseph Aloocaran

Copy Editor: Safis Editing

Project Coordinator: Manthan Patel

Proofreader: Safis Editing

Indexer: Manju Arasan

Production Designer: Nilesh Mohite

First published: July 2021

Production reference: 1290721

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83898-619-3

www.packt.com

To my mom, Irina Leonidovna Kosarevskaya, for your sacrifices; to my granny, Ludmila Fedorovna Sitorkina, for always cheering me on regardless of the hobbies I pursued; to the memory of my grandpa, Leonid Mikhailovich Sirotkin (1936-2013), for bringing me my first computer at the age of nine.– Sergey Kosarevsky

To my daughter, Polina.To my wife, Maria.To my parents, Nikolai and Galina.– Viktor Latypov

Contributors

About the authors

Sergey Kosarevsky is a rendering lead at Ubisoft RedLynx. He worked in the mobile industry at SPB Software, Yandex, Layar and Blippar, TWNKLS, and DAQRI, where he designed and implemented real-time rendering technology. He has more than 18 years of software development experience and more than 10 years of mobile and embedded 3D graphics experience. In his Ph.D. thesis, Sergey employed computer vision to solve mechanical engineering problems. He has co-authored several books on mobile software development in C++.

I wish to thank our reviewers and my friends, Alexander Pavlov, Anton Gerdelan, and Chris Forbes, for putting their time and effort into making this book happen. Thank you for your massive support, guys!

Viktor Latypov is a software engineer specializing in embedded C/C++, 3D graphics, and computer vision. With more than 15 years of software development experience and a Ph.D. in applied mathematics, he has implemented a number of real-time renderers for medical and automotive applications over the last 10 years.

Together with Sergey, he has co-authored two books on mobile software development in C++.

I would like to thank our reviewers for all their time, effort, and invaluable comments to make this book much better. Without you, we wouldn't have made it through the crazy year of 2020.

About the reviewers

Alexander Pavlov is a Senior Software Engineer with Google. He has more than 20 years of experience in various areas of industrial software development. His primary areas of interest are large-scale system design, compiler theory, and 3D graphics. Alexander is also an informal technical reviewer and proofreader behind the Android NDK Game Development Cookbook, published by Packt.

Chris Forbes works as a software developer for Google, working on Vulkan validation support and other ecosystem components. Previously, he has been involved in implementing OpenGL 3 and 4 support in open source graphics drivers for Linux [www.mesa3d.org], as well as rebuilding classic strategy games to run on modern systems [www.openra.net].

Anton Gerdelan is a computer graphics programmer and is most well known for writing an introductory book for graphics students, titled Anton's OpenGL 4 Tutorials, and recently, Professional Programming Tools for C and C++. Anton taught computer graphics and programming at Trinity College Dublin in Ireland, and at Blekinge Tekniska Högskola in Sweden. Anton is the Director of Engineering at Volograms.

Table of Contents

Preface

Chapter 1: Establishing a Build Environment

Technical requirements

OpenGL 4.6 with AZDO and Vulkan

What is the essence of modern OpenGL?

Setting up our development environment on Windows

Getting ready

How to do it...

There's more...

Setting up our development environment on Linux

Getting ready

How to do it...

Installing the Vulkan SDK for Windows and Linux

Getting ready

How to do it...

There's more...

Managing dependencies

Getting ready

How to do it...

How it works...

There's more...

Getting the demo data 

How to do it

There's more...

Creating utilities for CMake projects

Getting ready

How to do it...

There's more...

Chapter 2: Using Essential Libraries

Technical requirements

Using the GLFW library

Getting ready

How to do it...

There's more...

Doing math with GLM

Getting ready

How to do it...

There's more...

Loading images with STB

Getting ready

How to do it...

There's more...

Rendering a basic UI with Dear ImGui

Getting ready

How to do it...

There's more…

Integrating EasyProfiler 

Getting ready

How to do it...

How it works...

There's more...

Integrating Optick

Getting ready

How to do it...

How it works...

There's more…

Using the Assimp library

Getting ready

How to do it...

Getting started with Etc2Comp 

Getting ready

How to do it...

There's more...

Multithreading with Taskflow

Getting ready

How to do it...

There's more...

Introducing MeshOptimizer 

Getting ready

How to do it...

There's more...

Chapter 3: Getting Started with OpenGL and Vulkan

Technical requirements

Intercepting OpenGL API calls

Getting ready

How to do it...

How it works...

Working with Direct State Access (DSA)

Getting ready

How to do it...

There's more...

Loading and compiling shaders in OpenGL

Getting ready

How to do it...

There's more...

Implementing programmable vertex pulling (PVP) in OpenGL

Getting ready

How to do it...

How it works...

There's more...

Working with cube map textures

Getting ready

How to do it...

There's more...

Compiling Vulkan shaders at runtime

Getting ready

How to do it...

How it works...

There's more...

Initializing Vulkan instances and graphical devices

Getting ready

How to do it...

Initializing the Vulkan swap chain

Getting ready

How to do it...

Setting up Vulkan's debugging capabilities

How to do it...

There's more…

Tracking and cleaning up Vulkan objects

Getting ready

How to do it...

Using Vulkan command buffers

How to do it...

See also

Dealing with buffers in Vulkan

Getting ready

How to do it...

How it works...

Using texture data in Vulkan

Getting ready

How to do it...

Using mesh geometry data in Vulkan

Getting ready

How to do it...

Using Vulkan descriptor sets

How to do it...

There's more…

Initializing Vulkan shader modules

Getting ready

How to do it...

Initializing the Vulkan pipeline

Getting ready...

How to do it...

There's more…

See also

Putting it all together into a Vulkan application

Getting ready

How to do it...

Chapter 4: Adding User Interaction and Productivity Tools

Technical requirements

Organizing Vulkan frame rendering code

Getting ready

How to do it...

There's more…

Organizing mesh rendering in Vulkan

Getting ready

How to do it...

Implementing an immediate mode drawing canvas

Getting ready

How to do it...

Rendering a Dear ImGui user interface with Vulkan

Getting ready

How to do it...

Working with a 3D camera and basic user interaction

Getting ready

How to do it...

How it works...

There's more...

Adding a frames-per-second counter

Getting ready

How to do it...

There's more...

Adding camera animations and motion

Getting ready

How to do it...

How it works...

There's more...

Integrating EasyProfiler and Optick into C++ applications

Getting ready

How to do it...

How it works...

Using cube map textures in Vulkan

Getting ready

How to do it...

Rendering onscreen charts

Getting ready

How to do it...

How it works...

Putting it all together into a Vulkan application

Getting ready

How to do it...

Chapter 5: Working with Geometry Data

Technical requirements

Organizing the storage of mesh data

Getting ready

How to do it...

How it works...

There's more...

Implementing a geometry conversion tool

Getting ready

How to do it...

How it works...

There's more...

Indirect rendering in Vulkan

Getting ready

How to do it...

There's more...

Implementing an infinite grid GLSL shader

Getting ready

How to do it...

There's more...

Rendering multiple meshes with OpenGL

Getting ready

How to do it…

Generating LODs using MeshOptimizer

Getting ready

How to do it...

There's more...

Integrating tessellation into the OpenGL graphics pipeline

Getting ready

How to do it...

There's more...

Chapter 6: Physically Based Rendering Using the glTF2 Shading Model

Technical requirements

Simplifying Vulkan initialization and frame composition

How to do it...

Initializing compute shaders in Vulkan

Getting ready

How to do it...

There's more...

Using descriptor indexing and texture arrays in Vulkan

Getting ready

How to do it...

There's more...

Using descriptor indexing in Vulkan to render an ImGui

Getting ready

How to do it...

Generating textures in Vulkan using compute shaders

Getting ready

How to do it...

Implementing computed meshes in Vulkan

Getting ready

How to do it...

There's more...

Precomputing BRDF LUTs

Getting ready

How to do it...

There's more...

Precomputing irradiance maps and diffuse convolution

Getting ready

How to do it...

There's more...

Implementing the glTF2 shading model

Getting ready

How to do it...

There's more...

Chapter 7: Graphics Rendering Pipeline

Technical requirements

How not to do a scene graph

Using data-oriented design for a scene graph

Getting ready

How to do it...

There's more...

Loading and saving a scene graph

Getting ready

How to do it...

There's more...

Implementing transformation trees

Getting ready

How to do it...

There's more...

Implementing a material system

Getting ready

How to do it...

Importing materials from Assimp

Getting ready

How to do it...

There's more...

Implementing a scene conversion tool

Getting ready

How to do it...

There's more...

Managing Vulkan resources

Getting ready

How to do it...

There's more...

Refactoring Vulkan initialization and the main loop

Getting ready

How to do it...

Working with rendering passes

Getting ready

How to do it...

There's more...

Unifying descriptor set creation routines

Getting ready

How to do it...

There's more...

Putting it all together into a Vulkan application

Getting ready

How to do it...

How it works…

There's more...

Chapter 8: Image-Based Techniques

Technical requirements

Implementing offscreen rendering in OpenGL

Getting ready

How to do it…

Implementing fullscreen quad rendering

Getting ready

How to do it…

There's more...

Implementing shadow maps in OpenGL

Getting ready

How to do it…

There's more…

Implementing SSAO in OpenGL

Getting ready

How to do it...

There's more...

Implementing HDR rendering and tone mapping

Getting ready

How to do it...

There's more...

Implementing HDR light adaptation

Getting ready

How to do it...

There's more...

Writing postprocessing effects in Vulkan

Getting ready...

How to do it...

Implementing SSAO in Vulkan

Getting ready...

How to do it...

Implementing HDR rendering in Vulkan

Getting ready…

How to do it...

There's more…

Chapter 9: Working with Scene Graphs

Technical requirements

Deleting nodes and merging scene graphs

How to do it...

Finalizing the scene-converter tool

Getting ready

How to do it...

There's more...

Implementing lightweight rendering queues in OpenGL

Getting ready

How to do it...

There's more...

Working with shape lists in Vulkan

Getting ready

How to do it...

Adding Bullet physics to a graphics application

Getting ready

How to do it...

There's more...

Chapter 10: Advanced Rendering Techniques and Optimizations

Technical requirements

Doing frustum culling on the CPU

Getting ready

How to do it…

There's more...

Doing frustum culling on the GPU with compute shaders

Getting ready

How to do it…

There's more...

Implementing order-independent transparency

Getting ready

How to do it…

Loading texture assets asynchronously

Getting ready

How to do it...

There's more...

Implementing projective shadows for directional lights

Getting ready

How to do it...

There's more...

Using GPU atomic counters in Vulkan

Getting ready

How to do it...

There's more…

Putting it all together into an OpenGL demo

Getting ready

How to do it...

There's more…

Other Books You May Enjoy

Preface

The 3D Graphics Rendering Cookbook is your one-stop-shop practical guide to learning modern graphics rendering algorithms and techniques using the C++ programming language and APIs such as OpenGL and Vulkan. Starting with the configuration of your OpenGL and Vulkan development environment, you will immerse yourself in various common graphics development aspects, including shader development and handling geometrical data, among others. Going further, the book delves into building a 3D rendering engine, guiding you step by step through a number of small yet self-contained recipes. After each recipe, you will be able to incrementally add features to your code base while learning how to integrate numerous possible 3D graphics methods and algorithms into one big project. The book dives deep into the discussion of the relevant graphics techniques, such as physically-based rendering, image-based rendering, and CPU/GPU geometry culling, to name a few. You will be introduced to common techniques and solutions when dealing with large datasets for 2D and 3D rendering and will apply optimization techniques to build high-performance and feature-rich graphics applications. By the end of this book, you will be able to create fast and versatile 3D rendering frameworks, and have a solid understanding of best practices in modern graphics API programming.

This book is about rendering and will be focusing on modern, real-time, GPU-accelerated rendering techniques rather than on just one specific graphics API. Nonetheless, we will use the latest version of OpenGL, a programming interface for creating real-time 3D graphics, with a flavor of the Approaching Zero Driver Overheads philosophy. Some parallels with Vulkan will be drawn to demonstrate the differences between these two APIs and how all the rendering concepts and techniques discussed in the book can be implemented using both programming interfaces.

Who this book is for

We expect our readers to have a good grasp of real-time 3D graphics. The opening chapters explain what you need to get started with OpenGL and Vulkan. However, we will not focus for long on the basics and will quickly move forward to cover more sophisticated content instead. If you are familiar with OpenGL 3 or OpenGL ES 2 and want to learn more about modern rendering approaches and migration paths toward present-day rendering APIs, this book is probably a good fit for you.

Even though graphics programming might seem an easy and fun way to get into software development, it actually requires the mastery of numerous advanced programming concepts. The reader is expected to have a thorough understanding of modern C++ and some math skills, such as basic linear algebra and computational geometry.

What this book covers

This book is divided into distinct chapters. Each chapter covers specific aspects of 3D rendering and contributes more material to build a versatile 3D graphics demo by the end of the book, starting from the basics, then exposing more complicated approaches, and finally adding some advanced rendering techniques to the code:

Chapter 1, Establishing a Build Environment, explains the current state of OpenGL, its place in the computer world, and how it compares to Vulkan. The reader will learn which tools and dependencies are necessary to work with the source code of this book, as well as how to set them up.

Chapter 2, Using Essential Libraries, contains a set of recipes for the rapid building of minimalistic graphical applications in pure OpenGL and Vulkan from scratch using popular open source libraries, such as GLFW, GLM, STB, ImGui, EasyProfiler, Optick, AssImp, Etc2Comp, TaskFlow, and MeshOptimizer.

Chapter 3, Getting Started with OpenGL and Vulkan, covers basic aspects of rendering APIs, such as intercepting API calls, working with buffers and textures, converting between different texture formats, implementing programmable vertex pulling, and compiling Vulkan shaders to SPIR-V at runtime.

Chapter 4, Adding User Interaction and Productivity Tools, focuses primarily on debugging, profiling, and user interaction mechanisms. The reader will learn how to debug and profile graphical applications in different ways, starting with onscreen counters and graphs, then going into open source instrumenting profilers, and finishing with useful helper classes to allow interactive application debugging.

Chapter 5, Working with Geometry Data, teaches you how to deal with geometry data in a modern 3D rendering pipeline and also how to get comfortable with advanced topics such as geometry Level of Detail (LOD) and tessellation. Besides that, some GL shading language techniques will be discussed to show how various utility functions for geometry rendering can be implemented.

Chapter 6, Physically Based Rendering Using the glTF2 Shading Model, presents the glTF2 physically-based shading model and the ways to render it using the GL shading language in OpenGL and Vulkan. Different aspects of data preprocessing techniques will be covered, including the precalculation of Bidirectional Reflectance Distribution Function (BRDF) look-up tables and irradiance maps.

Chapter 7, Graphics Rendering Pipeline, goes into the representation of complex 3D scene data with multiple internal dependencies and cross-references. You will learn how to apply performance-oriented techniques such as data-oriented design to build a high-performance 3D rendering system. This is where the real OpenGL and Vulkan stuff begins to happen.

Chapter 8, Image-Based Techniques, contains a series of recipes on how to improve graphics rendering realism by using image-based techniques, such as screen space ambient occlusion, high dynamic range rendering with light adaptation, and projective shadow mapping.

Chapter 9, Working with Scene Graph, extends the series of scene graph-related recipes by showing how to modify scene data and extend it with components showing potential ways of scaling this approach to build a real-world graphics engine. This is followed by a short discussion on how to use a physics engine with a rendering engine.

Chapter 10, Advanced Rendering Techniques and Optimization, dives deeper into approaches to the construction of GPU-driven rendering pipelines, multithreaded rendering, and other advanced techniques for feature-rich graphics applications. Here, we conclude the book by combining numerous recipes and techniques into a single application.

To get the most out of this book

You will need a machine supporting OpenGL 4.6 and Vulkan 1.2 with the latest GPU drivers. All code examples in this book have been tested using Visual Studio 2019 on Windows 10 and GCC 8 on Linux. macOS users will be able to run only the very first demos from this book due to the lack of API support. To run our Bootstrap dependency downloader script, Python 3.x is required. CMake is used to build the source code.

We advise you to download the source code from the GitHub repository (https://github.com/PacktPublishing/3D-Graphics-Rendering-Cookbook). Doing so will help you avoid any potential errors related to the copying and pasting of code.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/3D-Graphics-Rendering-Cookbook. If there's an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781838986193_ColorImages.pdf.

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "After setting the project name, this macro uses the GLOB_RECURSE function to collect all source and header files into the SRC_FILES and HEADER_FILES variables."

A block of code is set as follows:

macro(SETUP_GROUPS src_files)

  foreach(FILE ${src_files})

    get_filename_component(PARENT_DIR "${FILE}" PATH)

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

macro(SETUP_APP projname)

  set(PROJ_NAME ${projname})

  project(${PROJ_NAME})

Any command-line input or output is written as follows:

sudo apt-get update

Bold: Indicates a new term, an important word, or words that you see on screen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Choose Custom Installation and make sure that the pip box is checked."

Tips or important notes

Appear like this.

Sections

In this book, you will find several headings that appear frequently (Getting ready, How to do it..., How it works..., There's more..., and See also).

To give clear instructions on how to complete a recipe, use these sections as follows:

Getting ready

This section tells you what to expect in the recipe and describes how to set up any software or any preliminary settings required for the recipe.

How to do it…

This section contains the steps required to follow the recipe.

How it works…

This section usually consists of a detailed explanation of what happened in the previous section.

There's more…

This section consists of additional information about the recipe in order to make you more knowledgeable about the recipe.

See also

This section provides helpful links to other useful information for the recipe.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share Your Thoughts

Once you've read 3D Graphics Rendering Cookbook, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.

Chapter 1: Establishing a Build Environment

In this chapter, we will learn how to set up 3D graphics development environment on your computer for Windows and Linux operating systems. We will cover the following recipes:

OpenGL 4.6 with AZDO and VulkanSetting up our development environment on WindowsSetting up our development environment on LinuxInstalling the Vulkan SDK for Windows and LinuxManaging dependenciesGetting the demo dataCreating utilities for CMake projects

Technical requirements

You can find the code files present in this chapter on GitHub at https://github.com/PacktPublishing/3D-Graphics-Rendering-Cookbook/tree/master/Chapter1

OpenGL 4.6 with AZDO and Vulkan

Approaching Zero Driver Overhead (AZDO) is not a single specific OpenGL feature. It is more of a set of techniques, a software development approach with OpenGL, and a way of structuring data access and API usage to make it faster by batching a lot of draw calls together. It was originally presented by Cass Everitt, Tim Foley, John McDonald, and Graham Sellers. Many things possible with Vulkan are also possible with OpenGL/AZDO. However, Vulkan is very different to OpenGL because it can precompile the pipeline state, while in modern OpenGL, a comparable level of performance can be achieved through a completely different set of methods; for example, bindless textures and buffers, programmable vertex pulling, and combining multiple draw calls into a single one.

There is a vendor-specific OpenGL extension, NV_command_list, from NVIDIA, which can compile OpenGL commands into a command list for efficient reuse. We will not be covering it here because it is not part of standard OpenGL. For those who are interested, here is the specification to read: https://www.khronos.org/registry/OpenGL/extensions/NV/NV_command_list.txt.

What is the essence of modern OpenGL?

First, let's focus on some of the modern OpenGL extensions that have made it into OpenGL Core and form the basis of AZDO, making the porting of OpenGL renderers to Vulkan easier, as well as helping to improve their performance in OpenGL:

OpenGL 4.2:

ARB_texture_storage introduces immutable textures whose metadata should be provided up front. For example, the classic glTexImage() function does not have any information on how many MIP levels would be required for a texture. The new glTexStorage/glTextureStorage API is more versatile and allows more driver-side optimizations.

OpenGL 4.3:

ARB_multi_draw_indirect (MDI) is batching "on steroids" and, compared to the classic instancing, can render a number of different geometries at once in a single draw call. This feature plays best when object data, such as geometry and material properties, is stored in large buffers. When combined with persistent mapped buffers, MDI buffers can be filled with commands from different threads. Pointers to the buffers can be passed to threads and do not require any OpenGL context to be available. The buffer content does not have to be regenerated every frame so it can be reused and even changed by the GPU directly. This allows for various GPU-based culling techniques and automatic level-of-detail implemented via compute shaders.

OpenGL 4.4:

ARB_buffer_storage gives better usage hints than glBufferData() and allows applications to pass additional information about the requested allocation to the implementation, which may use this information for further optimizations. Furthermore, this extension introduces the concept of persistent mapped buffers, which allow applications to retain mapped buffer pointers. To implement a buffer update workflow identical to Vulkan, OpenGL app developers should use persistent mapped buffers for all buffers that are expected to be used for read or write access on the CPU. Persistent mapped buffers allow multiple threads to write them without using OpenGL at all. This opens the possibility to do straightforward multi-threaded command buffer generation for MDI buffers.

ARB_enhanced_layouts allows, among other things, the use of compile-time constant expressions in layout qualifiers and specifying explicit byte offsets within a uniform or shader storage block. These features are added on top of the explicit GLSL attribute locations functionality introduced in OpenGL 3.3, which allows us to explicitly specify our binding points. Vulkan uses hardcoded SPIR-V bindings as well and does not allow querying binding information at all.

OpenGL 4.5:

ARB_direct_state_access (DSA) provides a set of API functions to manipulate OpenGL objects, such as textures and buffers, directly rather than by means of the classic bind-to-edit approach. Using DSA allows one to modify an object state without affecting the global OpenGL state shared among all parts of the application. It also makes the API more Vulkan-like, enabling handle-based object access.

OpenGL 4.6:

GL_ARB_indirect_parameters allows us to store some parameters like draw count to MDI drawing commands inside buffers. This approach introduces even more possibilities to create novel GPU-driven rendering pipelines.

GL_ARB_shader_draw_parameters adds new built-in variables to GLSL vertex shader inputs, gl_BaseVertexARB and gl_BaseInstanceARB, which contain the values passed in the baseVertex and baseInstance parameters of the draw call. Besides the two previously mentioned input variables, this extension adds the gl_DrawID variable to store the index of the draw call that is currently being processed by the MDI drawing commands.

ARB_gl_spirv allows a SPIR-V module to be used as a shader stage in OpenGL. This extension is not directly related to the runtime performance or driver overhead per se. However, it allows the usage of the GL shading language compiler to produce binary modules consumable by both OpenGL and Vulkan, which makes porting to Vulkan more straightforward.

Besides the preceding core features of OpenGL, there are a few ARB OpenGL extensions that are considered to be a part of AZDO.

ARB_bindless_texture reinvents the binding of textures by using 64-bit handles instead of texture objects. Texture handles stored in buffers work fairly similarly to Vulkan's DescriptorSets and provide functionality comparable to VK_EXT_descriptor_indexing.

A very important thing to remember when switching from OpenGL/AZDO to Vulkan is that it is not necessarily going to give you any significant performance boost. The GPU hardware is the same and all the graphics rendering functionality exposed by Vulkan is almost identical to that found in OpenGL. One of the major Vulkan design goals is to considerably reduce the CPU time spent in the driver, therefore if your application is limited by the GPU rendering performance, it is highly unlikely that Vulkan will give you better performance.

Furthermore, there is a Vulkan & OpenGL Threaded CAD Scene sample from NVIDIA that compares various rendering code paths using modern OpenGL and Vulkan https://developer.nvidia.com/vulkan-opengl-threaded-cad-scene-sample. The result of this demo is that NV_command_list can deliver better performance than Vulkan on the CPU side, as it benefits from having the API designed directly with the hardware in mind.

Setting up our development environment on Windows

In this recipe, we will get started by setting up our development environment on Windows. We will go through the installation of each of the required tools individually and in detail.

Getting ready

In order to start working with the examples from this book in the Microsoft Windows environment, you will need some essential tools to be installed in your system.

The most important one is Microsoft Visual Studio 2019. Additional tools include the Git version control system, the CMake build tool, and the Python programming language. Throughout this book, we use these tools on the command line only, so no GUI add-ons will be required.

How to do it...

Let's install each of the required tools individually.

Microsoft Visual Studio 2019

Follow these steps to install Microsoft Visual Studio 2019:

Open https://visualstudio.microsoft.com and download the Visual Studio 2019 Community Edition installer.Start the installer and follow the onscreen instructions. For the purposes of this book, you need to have a native C++ compiler for the 64-bit Intel platform. Other components of the Visual Studio development environment are not required to run the sample code.

Git

Follow these steps to install Git:

Download the Git installer from https://git-scm.com/downloads, run it, and follow the on-screen instructions.We assume that Git is added to the system PATH variable. Enable the option shown in the following screenshot during installation:

Figure 1.1 – Git from the command line and also from third-party software

Also, select Use Windows' default console window as shown in the next screenshot. This way, you will be able to run the build scripts from the book seamlessly from any directory on your computer:

Figure 1.2 – Use Windows' default console window

Important note

Git is complex software and a huge topic in itself. We recommend the book Mastering GIT, Jakub Narębski, Packt Publishing, https://www.packtpub.com/application-development/mastering-git, along with the downloadable e-book ProGit, Scott Chacon & Ben Straub, APress at https://git-scm.com/book/en/v2.

CMake

To install CMake, please follow these steps:

Download the latest 64-bit CMake installer from https://cmake.org/download/.Run it and follow the on-screen instructions. If you already have an earlier version of CMake installed, it is best to uninstall it first.Select the Add CMake to system PATH for all users option, as shown here:

Figure 1.3 – Add CMake to the system PATH for all users

Python

To install Python, please follow these steps:

Download the Python 3 installer for 64-bit systems from https://www.python.org/downloads/.Run it and follow the onscreen instructions.During the installation, you also need to install the pip feature. Choose Custom Installation and make sure that the pip box is checked:

Figure 1.4 – Custom Installation options

Once the installation is complete, make sure to add the folder containing python.exe to the PATH environment variable.

There's more...

Besides Git, there are other popular version control systems, such as SVN and Mercurial. While developing large software systems, you will inevitably need to download some libraries from a non-Git repository. We recommend getting familiar with Mercurial.

While working in the command-line environment, it is useful to have some tools from the Unix environment, such as wget, grep, and find. The GnuWin32 project provides precompiled binaries of these tools, which can be downloaded from http://gnuwin32.sourceforge.net.

Furthermore, in the Windows environment, orthodox file managers make file manipulation a lot easier. We definitely recommend giving the open source Far Manager a try. You can download it from https://farmanager.com/index.php?l=en. It is shown in the following screenshot:

Figure 1.5 – The look and feel of Far Manager

Setting up our development environment on Linux

Linux is becoming more and more attractive for 3D graphics development, including gaming technology. Let's go through the list of the tools required to start working with this book on Linux.

Getting ready

We assume our reader has a desktop computer with a Debian-based GNU/Linux operating system installed. We also assume the reader is familiar with the apt package manager.

To start developing modern graphics programs on Linux, you need to have up-to-date video card drivers installed that support OpenGL 4.6 and Vulkan 1.2. To build the examples from this book, a C++ compiler with C++20 support is required. We use the GNU Compiler Collection.

How to do it...

On a Debian-based system, the installation process is straightforward. However, before installing any of the required packages, we recommend running the following command to ensure your system is up to date:

sudo apt-get update

Let's go through the list of essential software and install whatever is missing:

GCC Compiler

Assuming you have a properly configured apt package manager, run the following command to install the GCC compiler and related tools:

sudo apt-get install build-essential

CMake

The CMake build tool is also available in the standard repositories. To install CMake, run the following command:

sudo apt-get install cmake

CMake 3.15 or above is sufficient for the code examples in this book.

Git

To install the Git version control system, run the following command:

sudo apt-get install git

Python 3

To install the Python 3 package, run the following command:

sudo apt-get install python3.7

The exact version of Python may vary between distributions. Any version of Python 3 will suffice for the scripts in this book.

Now we are done with the basic packages, and we can install graphics-related software. Let's jump into the next recipe to learn how to deal with the Vulkan SDK.

Installing the Vulkan SDK for Windows and Linux

In this recipe, we will learn how to get started with the Vulkan SDK. We will describe the requirements and procedure for installing the LunarG Vulkan SDK for Windows and Linux.

In principle, it is possible to write Vulkan applications without the Vulkan SDK using only C/C++ header files provided by Khronos. You can get these header files by cloning the Git repository at https://github.com/KhronosGroup/Vulkan-Headers. However, it is advised to install the complete Vulkan SDK in order to use Vulkan Validation Layers and a standalone GLSL compiler.

Getting ready

Make sure you have the latest video card drivers for your operating system.

How to do it...

To install Vulkan 1.2 on Linux, follow these steps:

Open the https://www.lunarg.com/vulkan-sdk/ page in a browser and download the appropriate Vulkan SDK for Windows or Linux. After the download has finished, run the Windows installer file and follow the onscreen instructions. If you have Ubuntu 20.04 installed, use the following commands provided by LunarG's website:

wget -qO - http://packages.lunarg.com/lunarg-signing-key-pub.asc | \

sudo apt-key add -

sudo wget -qO /etc/apt/sources.list.d/lunarg-vulkan-1.2.176-focal.list https://packages.lunarg.com/vulkan/1.2.176/lunarg-vulkan-1.2.176-focal.list

sudo apt-get update

sudo apt-get install vulkan-sdk

Alternatively, you may need to download the .tar.gz SDK archive and unpack it manually.

There's more...

We do not explicitly describe the development process on macOS because it seems to be a rather turbulent environment with frequent breaking changes in the development infrastructure. However, some general guidance is given along the way.

The installation of well-known frameworks mentioned in the following chapters is similar to Linux. Instead of Synaptic (apt), the installation is performed using the Brew or MacPorts package managers.

The OpenGL support is now deprecated for newer applications in favor of the Metal API, on top of which Vulkan is implemented in the MoltenVK library. The LunarG site https://vulkan.lunarg.com provides the macOS version of the Vulkan SDK, which is the MoltenVK package. Its installation guide can be found at https://vulkan.lunarg.com/doc/sdk/latest/mac/getting_started.html. Make sure you are using CMake version 3.15 or later so that the find_package(Vulkan) macro yields the correct results. Here are the steps required to install the Vulkan SDK on Linux:

Let's download and unpack the Vulkan SDK with the following commands:

cd ~/

wget https://sdk.lunarg.com/sdk/download/1.2.176.1/mac/vulkansdk-macos-1.2.176.1.dmg

hdiutil mount vulkansdk-macos-1.2.176.1.dmg

Then we have to set a few environment variables. To make these settings permanent, add the following lines to $HOME/.bash_profile:

export VULKAN_ROOT_LOCATION="$HOME/"

export VULKAN_SDK_VERSION="1.2.176.1"

export VULKAN_SDK="$VULKAN_ROOT_LOCATION/vulkansdk-macos-$VULKAN_SDK_VERSION/macOS"

export VK_ICD_FILENAMES="$VULKAN_SDK/etc/vulkan/icd.d/MoltenVK_icd.json"

export VK_LAYER_PATH="$VULKAN_SDK/etc/vulkan/explicit_layers.d" 

export PATH="$VULKAN_SDK/bin:$PATH"

This way you can install a new version of the SDK and only change the VULKAN_SDK_VERSION value.

To check that your installation is correct, run Applications/vkcube.app from the SDK. To check whether you can build your Vulkan applications on macOS with CMake, create the following CMakeLists.txt file:

cmake_minimum_required(VERSION 3.15)

project("VulkanTest" CXX C)

find_package(Vulkan)

if (NOT Vulkan_FOUND)

message( FATAL_ERROR "Vulkan not found" )

endif()

If the output of cmake -G "Unix Makefiles" does not contain Vulkan not found, you are ready to use Vulkan in your macOS applications.

Important Note

When developing cross-platform applications, it is good to use similar tools for each platform. Since both macOS and Linux support GCC and Clang compilers, using GCC on Windows ensures that you avoid the most common portability issues. A complete package of C and C++ compilers can be downloaded from http://www.equation.com/servlet/equation.cmd?fa=fortran.

An alternative way to use GCC on Windows is to install the MSys2 environment from https://www.msys2.org. It features the package management system used in Arch Linux, Pacman.

Managing dependencies

This book's examples use multiple open source libraries. To manage these dependencies, we use a free and open source tool called Bootstrap. The tool is similar to Google's repo tool and works both on Windows and Linux, as well as on macOS for that matter.

In this section, we learn how to use Bootstrap to download libraries using the Vulkan Headers repository as an example.

Getting ready

Make sure you have Git and Python installed, as described in the previous recipes. After that, clone the source code bundle repository from GitHub:

git clone https://github.com/PacktPublishing/3D-Graphics-Rendering-Cookbook

How to do it...

Let's look into the source code bundle and run the bootstrap.py script:

bootstrap.py

The script will start downloading all the third-party libraries required to compile and run the source code bundle for this book. The tail of the output should look as follows:

Cloning into 'M:\Projects.CPP\Book_Rendering\Sources\deps\src\assimp'...

remote: Enumerating objects: 25, done.

remote: Counting objects: 100% (25/25), done.

remote: Compressing objects: 100% (24/24), done.

remote: Total 51414 (delta 2), reused 10 (delta 1), pack-reused 51389

Receiving objects: 100% (51414/51414), 148.46 MiB | 3.95 MiB/s, done.

Resolving deltas: 100% (36665/36665), done.

Checking out files: 100% (2163/2163), done.

Once the download process is complete, we are ready to build the project.

How it works...

Bootstrap takes a JSON file as input, opening bootstrap.json from the current directory by default. It contains metadata of libraries we want to download; for example, their names, where to retrieve them from, a specific version to download, and so on. Besides that, each used library can have some additional instructions on how to build it. They can be patches, unpacking instructions, SHA hashes to check archive integrity, and many others.

Source code location for each library can be represented by either a URL of a version control system repository or by an archive file with the library source files.

A typical JSON file entry corresponding to one library looks like in the following snippet. The type field can have one of the following values: archive, git, hg, or svn. The first value corresponds to an archive file, such as .zip, .tar.gz, .tar.bz2, while the last three types describe different version control system repositories. The url field contains a URL of the archive file to be downloaded or a URL of the repository. The revision field can specify a particular revision, tag, or branch to check out:

[

{

    "name": "vulkan",

    "source": {

        "type": "git",

        "url": "https://github.com/KhronosGroup/Vulkan-

               Headers.git",

        "revision": "v1.2.178"

    }

}

]

The complete JSON file is a comma-separated list of such entries. For this recipe, we have got only one library to download. We will add more libraries in the next chapters. The accompanying source code bundle contains a JSON file with all the libraries used in this book.

There's more...

There is comprehensive documentation for this tool that describes other command-line options and JSON fields in great detail. It can be downloaded from https://github.com/corporateshark/bootstrapping.

The Bootstrap tool does not differentiate between source code and binary assets. All the textures, 3D models, and other resources for your application can also be downloaded, organized, and kept up to date automatically.

Getting the demo data 

This book makes use of free 3D graphics datasets as much as possible. The comprehensive list of large 3D datasets is maintained by Morgan McGuire, Computer Graphics Archive, July 2017 (https://casual-effects.com/data). We will use some large 3D models from his archive for demonstration purposes in this book. Let's download and patch one of them.

How to do it

To download the entire dataset, follow these simple steps:

Open the https://casual-effects.com/data/ page in a browser and find the Amazon Lumberyard Bistro dataset.Click on the Download link and allow the browser to download all the data files. The following is a screenshot of Morgan McGuire's site with the download link:

Figure 1.6 – Amazon Lumberyard Bistro as pictured on casualeffects.com as a 2.4-GB download

There's more...

Some of the material properties in the downloaded dataset should be updated. Use the interior.mtl and exterior.mtl files from the https://github.com/corporateshark/bistro_materials repository to replace the corresponding ones from the downloaded dataset.

Creating utilities for CMake projects

In this recipe, we will see how CMake is used to configure all the code examples in this book and learn some small tricks along the way.

Note

For those who are just starting with CMake, we recommend reading the books CMake Cookbook (Radovan Bast and Roberto Di Remigio), Packt Publishing and Mastering CMake (Ken Martin and Bill Hoffman), Kitware.

Getting ready

For a starter, let's create a minimalistic C++ application with a trivial main() function and build it using CMake:

int main()

{

  return 0;

}

How to do it...

Let's introduce two helper macros for CMake. You can find them in the CMake/CommonMacros.txt file of our source code bundle:

The SETUP_GROUPS macro iterates over a space-delimited list of C and C++ files, whether it is a header or source file, and assigns each file into a separate group. The group name is constructed based on the path of each individual file. This way we end up with a nice folder-like directory structure in the Visual Studio Solution Explorer window, as we can see on the right in the following screenshot:

Figure 1.7 – Without groups (left) and with groups (right)

The macro starts by iterating over a list of files passed in the src_files parameter:

macro(SETUP_GROUPS src_files)

  foreach(FILE ${src_files})

    get_filename_component(PARENT_DIR "${FILE}" PATH)

We store the parent directory name as a default group name and replace all the backslash characters with forward slashes:

    set(GROUP "${PARENT_DIR}")

    string(REPLACE "/" "\\" GROUP "${GROUP}")

Then, we can tell CMake to assign the current file into a source group with this name:

    source_group("${GROUP}" FILES "${FILE}")

  endforeach()

endmacro()

The second macro, SETUP_APP is used as a shortcut to create a new CMake project with all the standard properties we want it to have. It is very convenient when you have a number of very similar subprojects, like in this book:

macro(SETUP_APP projname)

  set(PROJ_NAME ${projname})

  project(${PROJ_NAME})

After setting the project name, this macro uses the GLOB_RECURSE function to collect all source and header files into the SRC_FILES and HEADER_FILES variables:

  file(GLOB_RECURSE SRC_FILES LIST_DIRECTORIES

    false RELATIVE

    ${CMAKE_CURRENT_SOURCE_DIR} src/*.c??)

  file(GLOB_RECURSE HEADER_FILES LIST_DIRECTORIES

    false RELATIVE

    ${CMAKE_CURRENT_SOURCE_DIR} src/*.h)

In all our code samples, we use the src directory containing the source files as an include directory as well:

  include_directories(src)

All enumerated source and header files are added to an executable inside the current project:

  add_executable(${PROJ_NAME} ${SRC_FILES}

                 ${HEADER_FILES})

We use the previously mentioned SETUP_GROUP macro to place each source and header file into an appropriate group inside the project:

  SETUP_GROUPS("${SRC_FILES}")

  SETUP_GROUPS("${HEADER_FILES}")

The next three properties set different executable filenames for each supported build configuration. These lines are optional, yet they are really useful when using CMake with the Visual Studio IDE. The reason is that Visual Studio can change build configurations (or build types, as they are called in CMake) dynamically directly from the IDE, and each build configuration can have its own output filename. We add prefixes to these filenames so that they can co-exist in a single output folder:

  set_target_properties(${PROJ_NAME}

    PROPERTIES OUTPUT_NAME_DEBUG ${PROJ_NAME}_Debug)

  set_target_properties(${PROJ_NAME}

    PROPERTIES OUTPUT_NAME_RELEASE

${PROJ_NAME}_Release)

  set_target_properties(${PROJ_NAME}

    PROPERTIES OUTPUT_NAME_RELWITHDEBINFO

${PROJ_NAME}_ReleaseDebInfo)

Since we use C++20 throughout this book, we require CMake to enable it.

  set_property(TARGET ${PROJ_NAME} PROPERTY

CXX_STANDARD 20)

  set_property(TARGET ${PROJ_NAME} PROPERTY

CXX_STANDARD_REQUIRED ON)

To make debugging with Visual Studio easier, we enable console the output by changing the application type to Console. We also set the local debugger working directory to CMAKE_SOURCE_DIR, which will make finding assets a lot more straightforward and consistent:

  if(MSVC)

    add_definitions(-D_CONSOLE)

    set_property(TARGET ${PROJ_NAME} PROPERTY

                 VS_DEBUGGER_WORKING_DIRECTORY

                 "${CMAKE_SOURCE_DIR}")

  endif()

endmacro()

Finally, the top-levelCMakeLists.txt file of our first project will look like this:

cmake_minimum_required(VERSION 3.12)

project(Chapter1)

include(../../CMake/CommonMacros.txt)

SETUP_APP(Ch1_Sample2_CMake "Chapter 01")

You may notice that the project(Chapter1) line is overridden by a call to project() inside the SETUP_APP macro. This is due to the following CMake warning, which will be emitted if we do not declare a new project right from the get-go:

CMake Warning (dev) in CMakeLists.txt:

No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of project(ProjectName) near the top of the file, but after cmake_minimum_required().

To build and test the executable, create the build subfolder, change the working directory to build, and run CMake as follows:For Windows and Visual Studio 2019, run the following command to configure our project for the 64-bit target platform architecture:

cmake .. -G "Visual Studio 16 2019" -A x64

For Linux, we can use the Unix Makefiles CMake generator as follows:

cmake .. -G "Unix Makefiles"

To build an executable for the release build type, you can use the following command on any platform:

cmake --build . --config Release

All the demo applications from the source code bundle should be run from the folder where the data/ subfolder is located.

There's more...

Alternatively, you can use the cross-platform build system Ninja along with CMake. It is possible to do so simply by changing the CMake project generator name:

cmake .. -G "Ninja"

Invoke Ninja from the command line to compile the project:

ninja

[2/2] Linking CXX executable Ch1_Sample1.exe

Notice how quickly everything gets built now, compared to the classic cmake --build command. See https://ninja-build.org for more details.