Augmented Reality with Unity AR Foundation E-Book

Jonathan Linowes

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 443

Veröffentlichungsjahr: 2021

Beschreibung

Augmented reality applications allow people to interact meaningfully with the real world through digitally enhanced content.
The book starts by helping you set up for AR development, installing the Unity 3D game engine, required packages, and other tools to develop for Android (ARCore) and/or iOS (ARKit) mobile devices. Then we jump right into the building and running AR scenes, learning about AR Foundation components, other Unity features, C# coding, troubleshooting, and testing. We create a framework for building AR applications that manages user interaction modes, user interface panels, and AR onboarding graphics that you will save as a template for reuse in other projects in this book. Using this framework, you will build multiple projects, starting with a virtual photo gallery that lets you place your favorite framed photos on your real-world walls, and interactively edit these virtual objects. Other projects include an educational image tracking app for exploring the solar system, and a fun selfie app to put masks and accessories on your face. The book provides practical advice and best practices that will have you up and running quickly.
By the end of this AR book, you will be able to build your own AR applications, engaging your users in new and innovative ways.

Leseprobe

Augmented Reality with Unity AR Foundation


A practical guide to cross-platform AR development with Unity 2020 and later versions


Jonathan Linowes

BIRMINGHAM—MUMBAI

Augmented Reality with Unity AR Foundation

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Group Product Manager: Pavan Ramchandani

Senior Editor: Hayden Edwards

Content Development Editor: Aamir Ahmed

Technical Editor: Saurabh Kadave

Copy Editor: Safis Editing

Project Coordinator: Manthan Patel

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Production Designer: Sinhayna Bais

First published: August 2021

Production reference: 1130821

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-83898-259-1

www.packt.com


Yes, I love my wife, Lisa, and our four amazing children, but this book I dedicate to my dog, Coder, a large, sweet, and boisterous Doberman Shepherd who keeps trying to pull me from my desk to play frisbee outside in the real world.

– Jon

Contributors

About the author

Jonathan Linowes is a VR/AR enthusiast, Unity, and full stack developer, entrepreneur, certified Unity instructor, and the owner of Parkerhill XR Studio, an immersive media, applications, and game developer. Jonathan has a bachelor of fine arts degree from Syracuse University, a master of science degree from the MIT Media Lab, and has held technical leadership positions at Autodesk and other companies. He has authored multiple books on VR and AR published by Packt Publishing.

About the reviewers

David Cantón Nadales is a software engineer from Seville, Spain, with over 15 years of experience. He specializes in Firebase and the development of mobile apps and video games, and is experienced in VR/AR with Oculus, Hololens, HTC Vive, DayDream, and LeapMotion. He was the ambassador of the Samsung development community "Samsung Dev Spain," and the organizer of the "Google Developers Group Seville" community He has worked on, and brought to fruition, more than 50 projects during his professional career. As a social entrepreneur, Grita stands out, a social network that emerged during the period of COVID-19 confinement that allowed people to talk to others in the same circumstances and help each other psychologically.

Rohit Gonsalves is connecting computer graphics, video, media, and devices to create beautiful Extended Reality (XR) workflows for PC and mobiles. He has 17+ years of experience working with computer graphics, specifically, C++, Windows, and DirectX. He is also the author of an in-progress book entitled 3D Game and Graphics Programming using UNIGINE. He is the CTO of BEiT Media.

Table of Contents

Preface

Section 1 – Getting Started with Augmented Reality

Chapter 1: Setting Up for AR Development

Technical requirements

Defining Augmented Reality

Getting started with Unity

Installing Unity Hub

Installing a Unity Editor

Creating and managing Unity projects

Introducing the Unity Editor interface

Basics of using the Unity Editor

Organizing your project assets

Preparing your project for AR development

Installing XR plugins for AR devices

Installing the AR Foundation package

Choosing an input handler

Adding support for the Universal Render Pipeline

Setting up for mobile development

Setting up for Android/ARCore development

Setting up for iOS/ARKit development

Developing for wearable AR glasses

Building and running a test scene

Summary

Chapter 2: Your First AR Scene

Technical requirements

Exploring the AR Foundation example projects from Unity

Building and running the Samples project

Exporting the sample assets for reuse

Building the SimpleAR scene in your own project

Creating a new project

Importing the Sample assets into your own project

Starting a new, basic AR scene

Using AR Session

Using AR Session Origin

Using the AR Camera

Adding Plane and Point Cloud managers

Adding AR Raycast Manager

Adding Light Estimation

Building and running the scene

Placing an object on a plane

Setting up a PlaceObject input action

Introducing Unity C# programming and the MonoBehaviour class

Writing the PlaceObjectOnPlane script

Building and running the scene

Refactoring your script

Creating a prefab for placing

Understanding GameObjects and Transforms

Finding a 3D model

Completing the scene

Summary

Chapter 3: Improving the Developer Workflow

Technical requirements

Troubleshooting with log messages

Using Debug.Log

Using the Console with a mobile device

Simulating a Console window in your app

Debugging with a debugger

Debugging on a remote device

Testing with an editor remote tool

Simulating with Unity MARS

Capturing with the MARS Companion app

Summary

Section 2 – A Reusable AR User Framework

Chapter 4: Creating an AR User Framework

Technical requirements

Understanding AR interaction flow

Installing prerequisite assets

TextMeshPro

DOTween

Serialized Dictionary Lite

Other prerequisite assets

Starting with a new scene

Creating the UI canvas and panels

Creating the screen space canvas

Adding an app title

Creating the UI panels

Creating the UI controller

Creating a Singleton class script

Writing the UIController script

Fading the UI panels

Creating an Interaction Controller mode

Creating the interaction mode hierarchy

Writing the Interaction Controller

Creating the interaction modes behavior

The StartupMode script

The ScanMode script

The MainMode script

The NonARMode script

Testing it out

Using the Unity onboarding UX assets

Introducing the onboarding assets

Preparing the Unity AR onboarding assets

Installing dependency packages

Importing the OnboardingUX package

Writing the AnimatedPrompt script

Integrating the onboarding graphics

Creating a scene template for new scenes

Summary

Chapter 5: Using the AR User Framework

Technical requirements

Planning the project

Starting with the ARFramework scene template

Adding a main menu

Adding PlaceObject-mode with instructional UI

Creating the PlaceObject UI panel

Creating the PlaceObject mode

Wiring the menu buttons

Performing a Building and Run

Hiding tracked objects when not needed

Advanced onboarding issues

Making an AR-optional project

Determining whether the device supports a specific AR feature

Adding localization

Summary

Section 3 – Building More AR Projects

Chapter 6: Gallery: Building an AR App

Technical requirements

Specifying the Art Gallery project UX

Project objectives

Use cases

UX design

User stories

Getting started

Collecting image data

Importing photos to use

Adding image data to the scene

Obtaining the pixel dimensions of an image

Loading the pictures list at runtime

Creating a framed photo prefab

Creating the prefab hierarchy

Writing a FramedPhoto script

Scaling the picture's shape

Hanging a virtual photo on your wall

Detecting vertical planes

Creating the AddPicture UI panel

Writing the initial AddPictureMode script

Creating the AddPicture Mode object

Creating a main menu Add button

Build And Run

Completing the AddPictureMode script

Showing tracked planes in AddPicture mode

Selecting an image to use

Creating the SelectImage mode

Creating the Select Image UI panel

Creating an Image Button prefab

Writing an ImageButtons script

Reroute the Add button

Adjusting for image aspect ratio

Summary

Chapter 7: Gallery: Editing Virtual Objects

Technical requirements

Creating an Edit mode

Creating an edit menu UI

Creating EditPicture mode

Selecting a picture to edit

Defining a SelectObject input action

Replacing the MainMode script

Selecting an object from Main-mode

Wiring the Done edit button

Highlighting the selected picture

Selecting an object from Edit mode

Avoiding intersecting objects

Deleting a picture

Replacing the picture's image

Replacing the frame

Interacting to edit a picture

Ensuring FramedPhoto objects receive Input Action messages

Adding the interaction components

Using our finger to move the picture

Pinching to resize the picture

Summary

Chapter 8: Planets: Tracking Images

Technical requirements

Understanding AR image tracking

Specifying the Planets project

User experience flow

Preparing the planet cards

Collecting planet textures and data

Getting started

Tracking reference images

Adding AR Tracked Image Manager

Creating a reference image library

Configuring the user interaction modes and UI

Scanning for reference images

Build and run

Creating and instantiating a virtual Earth prefab

Creating the generic Planet Prefab

Understanding equirectangular images

Creating the Earth prefab

Adding planet metadata

Animating the planet's rotation

Building the app's Main-mode

Writing the PlanetsMainMode script

Expanding the project with multiple planets

Adding the planet card image to the Reference Image Library

Creating the planet prefab

Responding to detected images

Making a responsive UI

Creating the Main-mode UI

Pointing the camera to show information

Displaying information details

Summary

Chapter 9: Selfies: Making Funny Faces

Technical requirements

Understanding face tracking

Face tracking versus face identification

Tracking a face with AR Foundation

Getting started

Creating a new scene using the ARFramework template

Setting up iOS ARKit for face tracking

Importing assets used in this project

Configuring a new AR scene for face tracking

Setting the AR camera for selfies

Adding an AR Face Manager component

Prompting the user to find a face, or not

Build and run

Tracking the face pose with 3D heads

Making a Mr. Plastic Head prefab

Making a Mr. Facet Head prefab

Building the Main mode and menu

Creating a changeable face prefab

Writing a main mode controller script

Creating scrollable main menu buttons

Adding a reset face button

Attaching 3D Accessories

Wearing a hat

Sporting cool sunglasses

Updating the scripts for accessories

Adding accessories to the main menu

Making dynamic face meshes with a variety of materials

Exploring AR Default Face

Creating face materials

Adding a face mesh visualizer to the changeable face prefab

Controlling the face material

Adding face materials to the main menu

Using eye tracking (ARKit)

Attaching stickers to face regions (ARCore)

Creating the sticker prefabs

Managing attachments' positions

Adding region attachments to the main menu

ARCore-only UI buttons

Tracking expressive face blend shapes (ARKit)

Summary

Other Books You May Enjoy

Section 1 – Getting Started with Augmented Reality

This section provides a basic introduction to developing AR applications and games with Unity, including AR technology concepts and how to use the Unity Editor. We cover the Unity XR plugin architecture, the AR Foundation toolkit, and other productivity tools. By the end of this section, you will be prepared to begin creating your own AR applications with Unity.

This section comprises the following chapters:

Chapter 1, Setting Up for AR DevelopmentChapter 2, Your First AR SceneChapter 3, Improving the Developer Workflow