Emerging Extended Reality Technologies for Industry 4.0 E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 442

Veröffentlichungsjahr: 2020

Beschreibung

In the fast-developing world of Industry 4.0, which combines Extended Reality (XR) technologies, such as Virtual Reality (VR) and Augmented Reality (AR), creating location aware applications to interact with smart objects and smart processes via Cloud Computing strategies enabled with Artificial Intelligence (AI) and the Internet of Things (IoT), factories and processes can be automated and machines can be enabled with self-monitoring capabilities. Smart objects are given the ability to analyze and communicate with each other and their human co-workers, delivering the opportunity for much smoother processes, and freeing up workers for other tasks. Industry 4.0 enabled smart objects can be monitored, designed, tested and controlled via their digital twins, and these processes and controls are visualized in VR/AR. The Industry 4.0 technologies provide powerful, largely unexplored application areas that will revolutionize the way we work, collaborate and live our lives. It is important to understand the opportunities and impact of the new technologies and the effects from a production, safety and societal point of view.

Leseprobe

Scrivener Publishing100 Cummings Center, Suite 541JBeverly, MA 01915-6106

Publishers at ScrivenerMartin Scrivener ([email protected])Phillip Carmical ([email protected])

Emerging Extended Reality Technologies For Industry 4.0

Early Experiences with Conception, Design, Implementation, Evaluation and Deployment

Jolanda G. Tromp

State University of New York, Oswego, New York, USA

Dac-Nhuong Le

Haiphong University, Haiphong, Vietnam

Chung Van Le

Duy Tan University, Danang, Vietnam

This edition first published 2020 by John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA and Scrivener Publishing LLC, 100 Cummings Center, Suite 541J, Beverly, MA 01915, USA© 2020 Scrivener Publishing LLCFor more information about Scrivener publications please visit www.scrivenerpublishing.com.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

Wiley Global Headquarters111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchant-ability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials, or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read.

Library of Congress Cataloging-in-Publication Data

ISBN 978-1-119-65463-6

Cover image: Pixabay.ComCover design by Russell Richardson

List of Tables

1.1

Software in use for classrooms and research.

1.2

Anecdotal responses about challenges.

2.1

Organ systems in the human body.

4.1

Results for hardware implementations.

4.2

Results for correlation and entropy.

5.1

Comparison of studies of entrepreneurship activities in social media.

6.1

Scenarios and tasks, as used in the experiment.

6.2

Descriptive statistics for the average TCR.

6.3

Results of independent t-test for TCR.

6.4

Descriptive statistics for the average TCR of individual tasks.

6.5

Descriptive statistics for the average TOT.

6.6

Results of independent t-test for TOT.

6.7

Descriptive statistics for the average TOT of individual tasks.

6.8

Independent t-test results of TOT of each task.

6.9

Descriptive statistics for the average ASQ score.

6.10

Results of independent t-test for average ASQ score.

6.11

Descriptive statistics for the average ASQ score of individual tasks.

6.12

Independent t-test results of the ASQ score of each task.

6.13

Descriptive plot for the average PSSUQ score.

6.14

Results of independent t-test for PSSUQ score.

11.1

Comparison of data mining and data stream mining

11.2

Data-based and task-based approaches.

11.3

Classification of algorithms for concept drift detection.

11.4

Datasets for DSM with concept drift.

Foreword

The 5th International Conference on Communication, Management and Information Technology (ICCMIT’19)1 was jointly organized in Vienna, Austria, on March 26-28, 2019,, by the Universal Society of Applied Research, Prague, Czech Republic, in collaboration with the University of Denver, Colorado, United States of America. The main objective of this conference, which has been running yearly since 2015, was to bring together researchers, societies, new technology experts, and manufacturing professionals interested or already involved in R&D with new technologies and innovative ideas at any scale and create a community spirit and learn from each other. The aim of this yearly conference is to facilitate sharing of research, ideas, and lessons learned by international researchers and explore collaborations to begin working towards achieving the highest standards of ICT. One of the major overall themes of the conference is Industry 4.0 and smart citizens, smart cities, smart factories, etc. These recent and innovative Industry 4.0 technologies are prototypes for the next generation of 21st century production systems. Advancement of information technologies and their convergence with operational technologies paves the way for an evolution of production systems. To remain competitive in the market, enterprises want to utilize these technological advancements in order to solve current challenges and serve customers in new ways which were not imagined before. In order to provide new services and products quickly, new methods and business models are needed. In order to exploit these new technologies they have to be introduced at manufacturing level.

The Fourth Industrial Revolution is emerging and evolving at an exponential rather than linear pace and disrupting almost every industry in every country around the globe. These changes are signaling the transformation of entire systems of production, management, and governance. Industry 4.0 will impact our business, and those businesses which are prepared are already implementing changes to adapt to a future where smart machines will allow them to escalate their business success. The participants of the ICCMIT’19 conference deeply discussed their diverse views on Industry 4.0 based on their expertise, and the major topics of discussion related to the digital divide, how academic institutions can support and advance the digital transformation, how to organize human/robot interactions in the digital transformation era, and how to lead the digital transformation of manufacturing companies. During the conference, researchers and practitioners exchanged their experiences with the different types of 21st century smart methods of monitoring and operating engineering, analytics and servicing activities, including the impacts of automation and smart sensing for the improvement of the quality and accuracy of the entire product or service supply chain.

Ibrahiem M. M. El Emary, PhDProfessor of Computer Science and SystemsFaculty of Arts and Humanities, King Abdulaziz University, Jeddah, Saudi ArabiaOrganizer of ICCMIT 2019

Introduction

Introduction to Key Industry 4.0 Technologies

The broad adoption of seventeen sustainable development goals has strongly emphasized using new emergent technologies for creating new solutions for our 21st century problems. This also calls for new business models and the reassessment of the current modes of government and manufacturing. This will require a global collaborative effort to work out how to employ new technologies to find these solutions, leading to a “Digital Revolution.” The United Nations has identified key sustainable development goals (SDGs) to transform our world that should be part of the Digital Revolution.1 These goals are listed below and in Figure I.1.

No Poverty

Zero Hunger

Good Health and Well-Being

Quality Education

Gender Equality

Clean Water and Sanitation

Affordable and Clean Energy

Decent Work and Economic Growth

Industry, Innovation and Infrastructure

Reduced Inequality

Sustainable Cities and Communities

Responsible Consumption and Production

Climate Action

Life below Water

Life on Land

Peace, Justice and Strong Institutions

Partnerships to Achieve the Goal

The convergence of associated emerging technologies in the form of the Internet of Things along with Artificial Intelligence will create large-scale intelligent networks. In addition, Machine Learning (ML) will facilitate the emergence of a worldwide Internet of Smart Things. These combinations of Artificial Intelligence and the Internet of Things can be called an Artificial Intelligence supported Internet of Things (AIIoT). The networks that implement these converged technologies will be the first major events of the Digital Revolution. It marks the time when users begin to see how vendor components and smart systems implement frictionless economics across integrated Smart Cities.

The exponential growth of AIIoT is based on the numerous configurations of new, smaller, more affordable networked sensors that can communicate with each other and potentially with all other sensors and processes in the supply chain. The configurations and implementation of the networked sensors and the data analytics for business intelligence need to be tailor-made to the requirements of human users, including the entire value chain and supply chain. The estimated 26 million software developers at the end of 2019 is predicted to grow to more than 27 million by 2023. Clearly, new approaches will need to be developed to assure that system professionals are compensated at a level that assures there will be an adequate supply of skilled workers.12

Figure I.1 Seventeen sustainable development goals.

The smooth implementation of automation across a number of industries relies on the coming together of stakeholders. Early successes will translate into rapid adoption and provide the foundation for later intelligent applications. There is a need for international standards in order to facilitate an efficient global collaboration. A number of stakeholders will be involved in collaborative efforts to make this happen. At a minimum these groups will include the following:

Government:

Although governments are not developing the technology, the responsibility for AIIoT systems to meet the needs of society is their purview. In performing this role they can facilitate communication between the various stakeholders and make sure that each voice is heard and collaboration ensures the successful operation of a Smart City and a caring society. It is expected that a government agency is appointed to oversee the implementation processes for AIIoT adoption.

National Laboratories:

Although governments ensure equal access to stakeholders, they will need to rely on the expertise of national laboratories to oversee the complex technical issues that will arise as large systems are assembled and tested. Furthermore, the experts of the Digital Revolution will be called on to identify needs of both the automated systems and their business users and end users. These smart field labs can provide system-level tools for monitoring and diagnostics and modeling and simulations services once component digital twins are populated into the system models. Another role for the labs will be to assist stakeholders in the development of curriculum that will assure that AIIoT professionals are available for system development and implementation. This may include assistance in developing explainable AI, scheme extraction algorithms and ML analytics.

Vendors:

Many AIIoT system components have already been developed and tested against industrial procedures and standards. The integration across public communities will lead to new types of issues that were not foreseen. Public-private partnerships are encouraged because they combine public sector needs with private sector technology and innovation.

Users:

Typically, there will be a core of early adopter companies and others will begin integration at a later time. There is a need to make sure that regardless of when companies begin and independent of their system type of size, that the resources are there to help them in their efforts.

Educators:

The development of curriculum, driven by inputs from the national labs, will assure that all required AIIoT topics are covered for targeted skills. Educators may also be called on to provide certifications based on testing developed by other agencies. It is likely that new cognitive computing systems will be used to rapidly access large libraries of data analytics. New curriculum components will be needed to support this addition. This is also an opportunity to identify student innovators that can aid in new development paradigms.

Voicing Concerns and Digital Twins, Blockchain, Big Data Analytics, Cognitive Computing, and 3D Printing, among others.

Communication between stakeholders is key to realizing the benefits of AIIoT. To facilitate a global conversation forum, there should be a framework for communication that permits conversations in all directions and is capable of addressing any issue. Experts with system-level experience are needed, who can draw on their experiences to avoid pitfalls and minimize risks. The framework may take the form of conferences, meetups or website forums. Moderators working closely with system experts can address issues that are raised by participants. It is important that a solid foundation is put in place that will support additional innovations that will be added at a later date.

The new digital economy is a paradigm shift, towards a data marketplace with many diverse data producers who need a distributed brokering system; a ledger, with seamless insurance and logistics, big data analytics and self-learning systems. The technologies that enable the new digital economy paradigm shift are interconnected, overlapping and converging.

These emergent AIIoT Industry 4.0 pillars currently are: Extended Reality (XR: virtual reality, augmented reality, mixed reality and other new forms still under development) development and deployment education, Sensors, Internet of Things (IoT) and Cybersecurity, Mobile Technologies and Cloud Computing, Machine-to-Machine Communication

The sections in this book are organized according to these various branches of the emergent technologies and the chapters address the evolving research that paves the way and enables solutions for smart cities and smart global solutions. Each chapter provides a time-stamp of current activities towards the paradigm shift and provides the necessary vision statements and use-case descriptions that help steer the adoption of smart city components. These vision statements will be translated into directives or regulations to be enacted by stakeholders. This involves a sequence of examinations and reviews by each participating company. The best general sequence follows the following processes or similar ones:

Policy Statements

: Regulations are distilled to individual directives. These are first reviewed by the national laboratories, perhaps working in a sandbox developed in conversations with vendors.

Vetting

: Once preliminary reviews have been completed the policy statements are provided to local governments for review against their needs. Issues that arise can be communicated with others or a digital regulatory agency. Local governments may also implement their own sandboxes that focus on their specifically unique environment.

Buy-In

: Once participants have tested and acknowledged the usefulness and performance of a policy statement, the parent policy regulation can be put into effect.

Compliance

: Full testing may be used to make sure that regulations perform across the smart city system. During this phase system data compliance and risk analysis reports can be used to address system issues.

The early AIIoT participants will be strategically placed to exponentially grow their productivity through AI and ML analysis and optimization. The superior products and services will rapidly reduce the market demand for other products and services that are outdated and lack functionality or quality, and such operations would systematically shut down due to inefficiency and high costs. Those who are already on the underdeveloped side of the digital divide will increasingly be more rapidly pushed out of competition. The configurations and implementation of the networked sensors and the data analytics for business intelligence need to be tailor-made to the requirements of the human users, and the business and value chains. Human needs for a prosperous, healthy, happy, safe, sustainable environment, are the main drivers for change and innovation. Successful international and intercultural respectful solutions for 21st century global issues can be built, using emergent technologies in novel ways. It is therefore necessarily a human-centered innovation design and development process.

Dr. Jolanda G. TrompDirector of Center for Visualization and SimulationDuy Tan University, Da Nang, Vietnam

John BottomsCEO FirstStar SystemsBoston, Massachusetts, USA

Preface

In Industry 4.0, extended reality (XR) technologies, such as virtual reality (VR) and augmented reality (AR), are creating location-aware applications to interact with smart objects and smart processes via cloud computing strategies enabled with artificial intelligence (AI) and the Internet ofThings (IoT). Factories and processes can be automated and machines can be enabled with self-monitoring capabilities. Smart objects are given the ability to analyze and communicate with each other and their human coworkers, delivering the opportunity for much smoother processes, and freeing up workers for other tasks. Industry 4.0-enabled smart objects can be monitored, designed, tested and controlled via their digital twins, and these processes and controls are visualized in VR/AR. The Industry 4.0 technologies provide powerful, largely unexplored application areas that will revolutionize the way we work, collaborate and live our lives. It is important to understand the opportunities and impacts of the new technologies and the effects from a production, safety and societal point of view.

This book presents empirical research results from user-centered qualitative and quantitative experiments on these new applications, and facilitates a discussion forum to explore the latest trends in XR applications for Industry 4.0. Additional contributions were collected via a public call to raise the number and quality of the chapters to the highest standard.

The selected best papers in this book are from the International Conference on Communication, Management and Information (ICCMIT’19), www.icmit.net (International Conference on Communication, Management and Information, 26-28 March 2019, Vienna, Austria) plus an open call for contributions showcasing the state-of-the-art of these new technologies and applications in terms of design challenges, evaluations and long-term use implications.

As we have entered the Industrial Revolution 4.0, XR applications, in combination with AI/IoT technologies, are fundamentally changing the way we work and live, generally referred to as Industry 4.0 or IR 4.0. Developments in these fields are very important because the novel combinations of these technologies can help improve and save lives, improve the work and collaboration processes and create smart objects in smart systems and smart cities. This in turn has far-reaching effects for educational, organizational, economic and social improvements to the way we work, teach, learn and care for ourselves and each other.

This book aims to combine the early explorations and discussions of Industry 4.0 key features that need to be addressed on a global scale:

The latest trends in new XR Industry 4.0 application developments.

Powerful, largely unexplored application areas that will revolutionize the way we work and live.

Combinations of XR technologies with artificial intelligence (AI) and the Internet of Things (IoT), showcasing the effect this has on Industry 4.0.

Practical use cases and evaluations of new XR technologies and applications that can help improve work processes and the way we live our lives.

Overview of the economic, psychological, educational and organizational impacts of the new XR applications on the way we work, teach, learn and collaborate in Industry 4.0 use cases.

Overview of the design, evaluation and long-term use implications for the development, assessment and use of XR applications.

Dac-Nhuong Le, PhDAssociate Professor of Computer ScienceDeputy Head, Faculty of Information TechnologyHai Phong University, Hai Phong, Vietnam

Acknowledgments

First of all, I would like to thank the authors for contributing their excellent chapters to this book. Without their contributions, this book would not have been possible. Thanks to all my colleagues and friends for sharing my happiness at the start of this project and following up with their encouragement when it seemed too difficult to complete.

I would like to acknowledge and thank the most important people in my life, my father, my mother and my partner, for their support. This book has been a long-cherished dream of mine which would not have been turned into reality without the support and love of these amazing people, who encouraged me despite my not giving them the proper time and attention. I am also grateful to my best friends for their blessings, unconditional love, patience and encouragement.

Dac-Nhuong Le, PhDAssociate Professor of Computer ScienceDeputy Head, Faculty of Information TechnologyHai Phong University, Hai Phong, Vietnam

Acronyms

5G

The next (5th) Generation

AI

Artificial Intelligence

AIIOT

Artificial Intelligence and Internet of Things

ADWIN

Adaptive Windowing

ADT

Active Drawing Time

AES

Advanced Encryption Standard

API

Application Programming Interface

AR

Augmented Reality

AUE2

Accuracy Updated Ensemble

AWS

Amazon Web Services

ASQ

After-Scenario Questionnaire

AGV

Automated Guided Vehicle

BPMN

Business Process Management Notation

B2B

Business-to-Business

B2C

Business-to-Consumer

B2G

Business-to-Government

B2E

Business-to-Employee

CA

Cellular Automaton

C2C

Consumer-to-Consumer

C2G

Consumer-to-Government

CoP

Communities of Practice

CRM

Customer Relationship Management

CPU

Central Processing Unit

CalTo

Calibration Timeouts

CR

Common Rail

CMD

Charge Motion Design

CRI

CR Rail Injector

CVS

Center of Visualization and Simulation

DDM

Drift Detection Method

DNS

Domain Name System

DaaM

Drawing as a Matrix

DST

Drawing Start Time

DET

Drawing End Time

DAnim

Drawing Animation

DWM

Dynamic Weighted Majority

DWCDS

Double-Window-Based Classification Algorithm

DSM

Data Stream Mining

EFT

Electronic Funds Transfer

EDI

Electronic Data Interchange

E2E

Employee-to-Employee

EIPM

Enterprise Innovation Processes Management

EDDM

Early Drift Detection Method

FPDD

Fisher Proportions Drift Detector

FTDD

Fisher Test Drift Detector

FSDD

Fisher Square Drift Detector

FFS

Fuel Feed System

FHDDM

Fast Hoeffding Drift Detection Method

GUI

Graphical User Interface

GUID

Global Unique Identification

GTM

Google Transactions Model

GPS

Global Positioning System

GIS

Geographic Information System

GPRS

General Packet Radio Service

HTTP

Hypertext Transfer Protocol

HTML

Hypertext Markup Language

HCI

Human Computer Interaction

HMD

Head-Mounted Display

HVAC

Heating, Ventilating, and Air Conditioning

ICT

Information and Communications Technology

IoT

Internet of Things

IP

Internet Protocol

IPv6

Internet Protocol version 6

IT

Information Technology

ISO

International Organization for Standardization

iLRN

Immersive Learning Research Network

LO

Learning Object

LoWPAN

Low-Power Wireless Personal Area Networks

LMS

Learning Management Systems

LPG

Liquefied Petroleum Gas

LPWAN

Low Power WANs

MOS

Mean Opinion Score

MOA

Massive Online Analysis

MQ6

LPG Gas Sensor

MDDM

McDiarmid Drift Detection Method

MRI

Magnetic Resonance Imaging

ML

Machine Learning

M2M

Machine to Machine

MPI

Message Passing Interface

NLP

Natural Language Processing

NB

Naive Bayes

OLS

Ordinary Least Squares

OS

Operating System

OpenGL

Open Graphics Library

OT

Operational Technology

ISO

International Organization for Standardization

PKI

Public Key Infrastructure

PHP

Hypertext Preprocessor

PC

Personal Computer

P2P

Peer to Peer

PLS

Partial Least Squares

PESQ

Perceptual Evaluation Speech Quality

PIR

Passive Infrared Sensor

PL

Paired Learner

PSSUQ

Post-Study System Usability Questionnaire

PLC

Powerline Connections

QS

Queuing System

QGD

Quasigasdynamic

RTW

Response Time Window

RS

Real Student

RFID

Radio Frequency Identification

RSSI

Received Signal Strength Indication

RDDM

Reactive Drift Detection Method

RC

Rivest Cipher

RefD

Reference Drawing

SAC

Strict Avalanche Criterion

SABI

Simple Algorithm for Boredom Identification

SAMOA

Scalable Advanced Massive Online Analysis

SEM

Structural Equation Modeling

SEO

Search Engine Optimization

SDM

Server Data Model

SIT

Secure IoT

SLA

Service Level Agreement

SME

Small and Medium-Sized Enterprise

SMS

Short Message Service

SNA

Social Network Analysis

SNR

Signal-to-Noise Ratio

SSL

Secure Sockets Layer

SQL

Structured Query Language

STEPD

Statistical Test of Equal Proportions

TCR

Task Completion Rate

TOT

Time on Task

TSL

Transport Layer Security

TCP

Transmission Control Protocol

TiAPI

TELECI input from Application Programming Interface

ToAPI

TELECI output to Application Programming Interface

ToITC

TELECI input from Initial Test Component

ToITC

TELECI output to Initial Test Component

TiPSC

TELECI input from Preliminary Survey Component

ToPSC

TELECI output to Preliminary Survey Component

TixAPI

TELECI input from Experience API

ThT

Threshold Time

TEA

Tiny Encryption Algorithm

URL

Uniform Resource Locator

UX/UI

User Experience/User Interface

UX

User Experience

UI

User Interaction

VR

Virtual Reality

VFDT

Very Fast Decision Tree

W3C

World Wide Web Consortium

WSN

Wireless Sensor Network

XRDC

eXtended Reality Developer Conference

XR

Extended Reality

XSS

Cross-Site Scripting

XML

Extensible Markup Language

PART IEXTENDED REALITY EDUCATION

The stakeholders in the AIIoT simulation-based optimization of planning, processing and delivery of operations, are the following three human user groups: human society, human operators, and human developers of the systems. Based on the exponential growth and all-pervasiveness of the AIIoT technologies that are embedded throughout our processes and will be driving our systems, it is rapidly clearly becoming more urgent to prepare a labor force with the required digital skills at all levels of education and training in order to be able to harness and benefit from the digital AIIoT transformations.

New job categories will arise with tasks that require technical capabilities and soft skills – essential human skills to manage the errors and problem solving that machines cannot handle. Governments and companies must plan to accelerate the creation of industrial engineering jobs dedicated to 3D modeling, 3D simulations, big data analytics, ML, robotics and development and customization of integrations of AIIoT-driven simulations and robotics solutions. Chapter 1 presents the results from an international survey regarding the use of XR technologies in the classroom to deliver classes, to teach the development of XR technologies and to research XR technologies, and a summary of the lessons learned.

There is a global need for skilled engineers and operators in order to research, build, test, deploy and maintain these new AIIoT-driven products, services, machines and platforms. To achieve positive economics for investment, robots must replace humans on the work floor, rather than support them. Routine manual activities can become fully automated. Routine and non-routine human activities will change, and the share of non-routine activities will increase for the human operator. Manual work will shift towards non-routine tasks, which means that workers must acquire more advanced skills. Chapter 2 presents a use-case study of a XR e-Health, e-Learning application for teaching anatomy, showing how disruptive new technologies can be to traditional education and accelerating opportunities for learning.

Chapter 1

: Mixed Reality Use in Higher Education: Results from an International Survey

Chapter 2

: Using 3D Simulation in Medical Education

CHAPTER 1MIXED REALITY USE IN HIGHER EDUCATION: RESULTS FROM AN INTERNATIONAL SURVEY

J. Riman1, N. Winters2, J. Zelenak3, I. Yucel4, J. G. Tromp5,*

1 SUNY Fashion Institute of Technology, New York, New York, USA

2 SUNY Delhi College of Technology, Delhi, New York, USA

3 University at Albany - State University of New York, Albany, New York, USA

4 SUNY Polytechnic Institute, Utica, New York, USA

5 Duy Tan University, Da Nang, Vietnam

* Corresponding author: [email protected]

Abstract