39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Lebensstil
- Sprache: Englisch
HoloLens revolutionizes the way we work and interact with the virtual world. HoloLens brings you the amazing world of augmented reality and provides an opportunity to explore it like never before.
This is the best book for developers who want to start creating interactive and intuitive augmented reality apps for the HoloLens platform.
You will start with a walkthrough of the HoloLens hardware before creating your first app. Next you will be introduced to the various HoloLens sensors and find out how to program them efficiently so that they can interact with the real world seamlessly. Moving on, you will learn how to create smart animations and add video overlay that implements real-time tracking and motion-sensing abilities to your HoloLens app. Finally, you will learn how to test your app effectively.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 262
Veröffentlichungsjahr: 2017
Ähnliche
Title Page
BIRMINGHAM - MUMBAI
Copyright
HoloLens Beginner’s Guide
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: April 2017
Production reference: 1260417
ISBN 978-1-78646-472-9
www.packtpub.com
Credits
Author
Jason M. Odom
Copy Editors
Dhanya Baburaj Shaila Kusanale
Reviewers
Vangos Pterneas Michael Washington
Project Coordinator
Ulhas Kambali
Commissioning Editor
Amarabha Banerjee
Proofreader
Safis Editing
Acquisition Editor
Larissa Pinto
Indexer
Tejal Daruwale Soni
Content Development Editor
Onkar Wani
Graphics
Abhinash Sahu
Technical Editor
Murtaza Tinwala
Production Coordinator
Deepika Naik
About the Author
Jason Odom is a game developer with nearly 20 years of experience working on titles for companies such as Activision and Take 2 Interactive, along with years of game development teaching and mentoring. He is the founder of Thought Experiment, a start-up mixed reality studio, and he is currently working on HoloQuest and CrossTrainMR properties for HoloLens. When he is not making software and applications, he is a loud and proud mixed reality evangelist with a highly active YouTube channel, and a prolific technology blogger for NextReality. You can connect to him on the following:
h t t p s ://w w w . y o u t u b e . c o m /p l a y l i s t ?l i s t =P L E j p 5C b k s q k 4x - 3r O _ 6G 8I - k 5_ v O E X 25W
h t t p s ://g i t h u b . c o m /S u b e r e 23?t a b =r e p o s i t o r i e s
h t t p ://w w w . t h o u g h t e x p e r i m e n t s t u d i o s . c o m /
http://next.reality.news/
About the Reviewer
Vangos Pterneas is helping innovative companies increase their revenue using motion technology and virtual reality. He is an expert in Kinect, HoloLens, Oculus Rift, and HTC Vive.
Microsoft has awarded him with the title of Most Valuable Professional for his technical contributions to the open source community. Vangos runs LightBuzz Inc, collaborating with clients from all over the world. He’s also the author of the book Getting Started with HTML5 WebSocket Programming, by Packt, and The Dark Art of Freelancing.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
h t t p s ://w w w . p a c k t p u b . c o m /m a p t
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at "Amazon Book URL".
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at h t t p s ://w w w . a m a z o n . c o m /H o l o L e n s - B e g i n n e r s - G u i d e - J a s o n - O d o m - e b o o k /d p /1786464721.
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Dedication
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
Welcome to the New World
What exactly is the HoloLens?
What you need to develop for the HoloLens
Unity
Visual Studio community
HoloToolkit - Unity
HoloLens emulator
Visual Studio tools for Unity
Other useful tools
Installing the software
Installing Visual Studio
Unity HoloLens technical preview
Visual Studio 2015 tools for Unity
HoloLens emulator
HoloToolkit-Unity
Unity3d quick overview
The main view
Summary
HoloWorld
Hello World!, as its customary
The layout of a HoloLens project
Unity directory structure
Saving the scene
Objects and components
Objects
Components
First project
Our cubism period
Adding a little color
Material plane
Creating the text
Physics simulations
Building the project in Unity
Build settings
Player settings
Compiling our project
Setting Up Visual Studio
Using the HoloLens emulator
Compilation time
Summary
I am in Control
Prefabs
Let's save the HelloWorld
Parent/Child relationship
Creating prefabs
Installing the HoloToolkit
User Input overview
InputManager prefab
New Camera prefab updated HoloToolkit
Cursor
Gaze
Gesture
Voice
Animation
A quick history
Humble beginnings
Time to move!
It's easier than it seems
Mecanim
The Mecanim window
Make them interactive
Here is the Game Plan
Our first script
Materials, textures, and shaders
Materials
Search For It
Adding the details
Changing the color
OnFocusEvent component
Speech Input Source
Time to build and test
Summary
User-Friendly Interface
Reloading
Unity C# scripting
What is C#?
A quick example
Where to begin
The breakdown of a Unity C# class
Variables
Access modifiers
Class methods
Understanding scope
A lot to take in
User interface design
General user interface best practices
Something new
Microsoft HoloLens best practices
The Unity camera and HoloLens
Unity UI
Overview
Interactivity
HoloLens UI
A visible canvas
Now it's time to build again
Any object can be an interface object
Summary
Now That Is How It Should Sound
Audio overview
Here is what it is all about
Unity audio components
Audio Listener
Audio Source
Spatial sound and HoloLens
Spatial sound
Setting up Audio Sources
Building a soundscape
Finding sounds
Importing our sounds into Unity
Audio Source options
Using our sounds
Let's listen to our work
Summary
Not So Blank Spaces
Spatial Mapping
Let's get into this
Importing the essentials
Spatial Mapping in practice
Spatial Mapping Observer
Spatial Mapping Manager
Object Surface Observer
Our new Skeeball machine
Application management
Design patterns
There can be only one
Creating our application manager
The ApplicationManager code
Attaching our application manager and wiring it up
So let it be written... so let it be done...
The home stretch
Code matters
Let us enjoy our hard work
Summary
The Tools of the Trade
Persistence
Why are persistent holograms necessary?
Coordinate systems
World Anchors
World Anchor system
Some quick upkeep
World Anchor code
ApplicationManager changes
A few more quick changes
Seeking help
The in-game debug window
The prefab
Let's wire this sucker up
In-game menu
Creating the menu items
Setting up to code
Time to code
Summary
Share What You Have Got
Networking
What is a network?
Making the Skeeball game a shared experience
Quick setup
A very simple first step
Too much information
Spawning our Skeeball machine
It's not really backtracking if it was the plan
Application manager
Enums
Updating the Update() method
Filling in the blanks
The payoff
Summary
Putting It All Together
Our main Prefab
The organization system
The reorganization
Importing the RampColliders
When bounding boxes collide
Who wants to score?
Think globally act locally
Finishing the Prefab
Ball Spawn Point
SkeeSpawner
Speed indicator
GameBall Prefab
Tags and Layers
ThrowBall.cs
Finishing touches
Updating Placeskee ballMachine
ApplicationManager
Action makes sound
Some quick house cleaning
Replacing the main Prefab
Summary
Fixing Problems
The Application Manager
Seeing it all at once
What does our Application Manager do?
Debugging
Bring the black flag
Inspector
Unity Property Attributes
Unity Inspector Debug Mode
Debug.Log/Console window
Visual Studio Debugger
Profiling and fixing performance
HoloLens Developers Portal
HoloToolkit FPS prefab
Unity Profiler
Visual Studio performance profiler
Unity Frame Debugger
Visual Studio Graphics Debugger
Simplygon
Summary
Preface
Microsoft HoloLens is on par with a modern-day wonder. It is the world’s first untethered, head mounted computer capable of projecting holograms into the world around us. This new technology enables forms of interaction that we have not even been able to imagine yet. New ideas are coming to the surface every day.
As this new device and its conceptual successors are further iterated upon, becoming more powerful and smaller, we will see a shift away from the 2D displays that we have had in place for the last 90 years. Even the pocket computers we use daily now will likely fall away into shared experiences that are projected through glasses, and maybe even one day contact lenses.
This book is here to get those interested in being on the front end of the coming wave up to speed with the development process for Windows mixed reality (formerly Windows Holographic), the ecosystem that Microsoft has created in parallel with Windows 10 for their new class of devices.
What this book covers
Chapter 1, Welcome to the New World, will introduce Microsoft HoloLens and its capabilities, as well as introduce and install the tools we will use to develop software for the device.
Chapter 2, HoloWorld, outlines how to build our first basic application. It will include the entire process from start to running on HoloLens.
Chapter 3, I Am in Control, covers the pillars of HoloLens input, Gaze Gesture, and Voice. We will also get into some of the elements of Unity development, such as Prefabs.
Chapter 4, User-Friendly Interface, takes you through C# scripting at a high, and teaches you to use what you have learned to build some interactive user interface elements. We then get to see these elements at work.
Chapter 5, Now That Is How It Should Sound, teaches you to take the knowledge accrued in the earlier chapters and begin your book project. We then learn about sound: AudioSources, AudioListener, and Spatialized Audio.
Chapter 6, Not So Blank Spaces, covers one of the major elements that help HoloLens create the magic that it is capable of--spatial mapping. We then use the spatial map in the context of our project so that we can learn how to move objects around our spatial map.
Chapter 7, The Tools of the Trade, focuses on the World Anchor, World Anchor Manager, and World Anchor Store. These important elements of persistence are a must-know. We then create a simple in-game debug output to help us track down problems.
Chapter 8, Share What You Have Got, explains the Holographic Sharing service that Microsoft provides as part of the HoloToolkit. This service allows us to create multiuser experiences for HoloLens.
Chapter 9, Putting It All Together, takes some of everything we have learned and brings it together to help us round out our project and make a playable game.
Chapter 10, Fixing Problems, showcases what debugging and profiling options exist within the confines of our tools, Unity and Visual Studio, and there are many.
What you need for this book
You require either a Windows 10, compatible PC and a HoloLens, or a PC with Windows 10 Pro and the HoloLens Emulator.
Who this book is for
If you are a developer new to Windows Universal development platform and want to get started with HoloLens development, this is the book for you. No prior experience of C# programming or of the .NET framework is needed to get started with this book.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at h t t p ://w w w . p a c k t p u b . c o m. If you purchased this book elsewhere, you can visit h t t p ://w w w . p a c k t p u b . c o m /s u p p o r t and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
Log in or register to our website using your e-mail address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at h t t p s ://g i t h u b . c o m /P a c k t P u b l i s h i n g /H o l o L e n s - B e g i n n e r s - G u i d e. We also have other code bundles from our rich catalog of books and videos available at h t t p s ://g i t h u b . c o m /P a c k t P u b l i s h i n g /. Check them out!
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from h t t p ://w w w . p a c k t p u b . c o m /s i t e s /d e f a u l t /f i l e s /d o w n l o a d s /H o l o L e n s B e g i n n e r s G u i d e _ C o l o r I m a g e s . p d f.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting h t t p ://w w w . p a c k t p u b . c o m /s u b m i t - e r r a t a , selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to h t t p s ://w w w . p a c k t p u b . c o m /b o o k s /c o n t e n t /s u p p o r t and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Welcome to the New World
We live in very exciting times. Technology is changing at a pace so rapid that it is becoming near impossible to keep up with these new frontiers as they arrive; they seem to arrive on a daily basis now. Moore's Law continues to stand, meaning that technology is getting smaller and more powerful at a constant rate. As I said, very exciting.
One of these new emerging technologies that finally is reaching a place more material than science fiction stories is Augmented or Mixed Reality. Imagine a world where our communication and entertainment devices are worn, and the digital tools we use, as well as the games we play, are holographic projections in the world around us. These holograms know how to interact with our world and change themselves to fit our needs. Microsoft has led the charge by releasing such a device: the HoloLens.
The Microsoft HoloLens changes the paradigm of what we know as personal computing--we can now have our Word window up on the wall (this is how I am typing right now), we can have research material floating around it, and we can have our communication tools, such as Gmail and Skype, in the area as well. We are finally no longer trapped by a virtual desktop, on a screen, sitting at a physical desktop; we aren't even trapped in the confines of a room anymore.
What exactly is the HoloLens?
The HoloLens is a first of its kind head-worn, standalone computer with a sensor array, which includes microphones and multiple types of camera, a spatial sound speaker array, a light projector, and an optical waveguide.
The HoloLens is not only a wearable computer, it is also a complete replacement for the standard 2D display. It has the capability of using holographic projection to create multiple screens throughout an environment and fully 3D-rendered objects. With the HoloLens sensor array, these holograms can fully interact with the environment you are in.
The sensor array allows the HoloLens to see the world around it, to see the input from the user's hands, and for it to hear voice commands. Although Microsoft has been very quiet about what the entire sensor array includes, we have a good general idea about the components used in the sensor array; let's take a look at them:
One IMU
: The
Inertia Measurement Unit
(
IMU
) is a sensor array that includes an accelerometer, a gyroscope, and a magnetometer. This unit handles head orientation tracking and compensates for the
drift
that comes from the Gyroscope's eventual lack of precision.
Four environment understanding sensors
: These together form the spatial mapping that the HoloLens uses to create a mesh of the world around the user.
One depth camera
: This is also known as a structured light 3D scanner. This device is used for measuring the 3D shape of an object using projected light patterns and a camera system. Microsoft first used this type of camera inside the Kinect for the Xbox 360 and Xbox One.
One ambient light sensor
: Ambient light sensors or photosensors are used for ambient light sensing and proximity detection.
2 MP photo/HD video camera
: This is used for taking pictures and video.
Four-microphone array
: These do a great job of listening to the user and not the sounds around them. Voice is one of the primary input types with HoloLens.
Putting all of these elements together forms a Holographic computer that allows the user to see, hear, and interact with the world around them in new and unique ways:
What you need to develop for the HoloLens
The HoloLens development environment breaks down into two primary tools, Unity and Visual Studio. Unity is the 3D environment that we will do most of our work in; this includes adding holograms, creating user interface elements, adding sound and particle systems, and other things that bring a 3D program to life.
Visual Studio on the other hand is the glue that makes everything work. Here, we write scripts or machine code to make our 3D creations come to life and add a level of control and immersion that Unity cannot produce on its own.
Unity
Unity is a software framework designed to speed up the creation of games and 3D-based software. Generally speaking, Unity is known as a game engine but the more apparent the holographic world becomes, the more we will use such a development environment for different kinds of applications.
Unity is an application that allows us to take 3D models, 2D graphics, particle systems, and sound to make them interact with each other and our user. Many elements are dragged and dropped and plugged and played; what you see is what you get. This can simplify the iteration and testing process. As developers, we most likely do not want to build and compile little changes we make in the development process forever. This allows us to see the changes in context to ensure that they work; then once we hit a group of changes, we can test them on the HoloLens ourselves. This does not work for every aspect of HoloLens-Unity development, but it does work for a good 80%-90%.
Visual Studio community
Microsoft Visual Studio Community is a great, free Integrated Development Environment (IDE). Here, we use programming languages, such as C# or JavaScript, to code changes in the behavior of objects, and generally make things happen inside of our programs.
HoloToolkit - Unity
The HoloToolkit-Unity is a repository of samples, scripts, and components to help speed up the process of development. This covers a large selection of areas in HoloLens development, such as the following:
Input
: Gaze, gesture, and voice are the primary ways in which we interact with the HoloLens.
Sharing
: The sharing repository helps allow users to share holographic spaces and connect to each other via the network.
Spatial Mapping
: This is how the HoloLens sees our world. A large 3D mesh of our space is generated and give our holograms something to interact with or bounce off of.
Spatial Sound
: The speaker array inside the HoloLens does an amazing job of giving the illusion of space. Objects behind us seem like they are behind us.
HoloLens emulator
The HoloLens emulator is an extension to Visual Studio that will simulate how a program will run on the HoloLens. This is great for those who want to get started with HoloLens development but do not have an actual HoloLens, yet. This software does require the use of Microsoft Hyper-V, a feature only available inside the Windows 10 Pro operating system. Hyper-V is a virtualization environment, which allows the creation of a virtual machine. This virtual machine emulates the specific hardware, so one can test without the actual hardware.
Visual Studio tools for Unity
This collection of tools adds IntelliSense and debugging features to Visual Studio; if you use Visual Studio and Unity, these are a must have:
IntelliSense
: An intelligent code completion tool for Microsoft Visual Studio. This is designed to speed up many processes when writing code. The version that comes with Visual Studios tools for Unity has Unity-specific updates.
Debugging
: Before this extension existed, debugging Unity apps proved to be a little tedious. With this tool, we can now debug Unity applications inside Visual Studio, thus speeding the bug squashing process considerably.
Other useful tools
The following are some the useful tools that are required:
Image editor
: Photoshop and Gimp both are good examples of programs that allow us to create 2D UI elements and textures for objects in our apps.
3D modeling software
: 3D Studio Max, Maya, and Blender are all programs that allow us to make 3D objects that can be imported in Unity.
Sound editing software
: There are a few resources for free sounds on the Web. With that in mind, Sound Forge is a great tool for editing those sounds, layering sounds together to create new sounds.
Installing the software
Now, we will get our development environment installed and set up so that we can start building the next killer app for HoloLens. We will start with our main tools, Visual Studio and Unity HoloLens technical preview, and then add the extensions that will help speed things along.
Installing Visual Studio
Microsoft has made the installation process for Visual Studio very simple. Just be warned that it can take a while to download and install it:
Navigate to
h t t p ://w w w . v i s u a l s t u d i o . c o m /e n - u s /v i s u a l - s t u d i o - h o m e p a g e - v s . a s p x
.
Click on
Download Community 2015
; this is a free version of Visual Studio that works well with Unity and the HoloLens environment. By default, you should be installing at least Visual Studios 2015 Update 3.
Run
vs_community. exe
and complete the installation process.
Create a Microsoft Developers account (this is a requirement for the HoloLens development).
The first time you run VS2015, you will be prompted to input your account credentials.
