Handbook of Human-Machine Systems E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Serie: Wiley-IEEE Press Book Series on Human-Machine Systems
• Sprache: Englisch

Seitenzahl: 1068

Veröffentlichungsjahr: 2023

Beschreibung

Handbook of Human-Machine Systems Insightful and cutting-edge discussions of recent developments in human-machine systems In Handbook of Human-Machine Systems, a team of distinguished researchers delivers a comprehensive exploration of human-machine systems (HMS) research and development from a variety of illuminating perspectives. The book offers a big picture look at state-of-the-art research and technology in the area of HMS. Contributing authors cover Brain-Machine Interfaces and Systems, including assistive technologies like devices used to improve locomotion. They also discuss advances in the scientific and engineering foundations of Collaborative Intelligent Systems and Applications. Companion technology, which combines trans-disciplinary research in fields like computer science, AI, and cognitive science, is explored alongside the applications of human cognition in intelligent and artificially intelligent system designs, human factors engineering, and various aspects of interactive and wearable computers and systems. The book also includes: * A thorough introduction to human-machine systems via the use of emblematic use cases, as well as discussions of potential future research challenges * Comprehensive explorations of hybrid technologies, which focus on transversal aspects of human-machine systems * Practical discussions of human-machine cooperation principles and methods for the design and evaluation of a brain-computer interface Perfect for academic and technical researchers with an interest in HMS, Handbook of Human-Machine Systems will also earn a place in the libraries of technical professionals practicing in areas including computer science, artificial intelligence, cognitive science, engineering, psychology, and neurobiology.

Leseprobe

IEEE Press445 Hoes LanePiscataway, NJ 08854

IEEE Press Editorial BoardSarah Spurgeon, Editor in Chief

Jón Atli BenediktssonAnjan BoseJames DuncanAmin MoenessDesineni Subbaram Naidu

Behzad RazaviJim LykeHai LiBrian Johnson

Jeffrey ReedDiomidis SpinellisAdam DrobotTom RobertazziAhmet Murat Tekalp

Handbook of Human-Machine Systems

Edited by

Giancarlo Fortino

University of Calabria

Italy

David Kaber

University of Florida

USA

Andreas Nürnberger

Otto-von-Guericke-Universität Magdeburg

Germany

David Mendonça

MITRE Corporation

USA

Copyright © 2023 by The Institute of Electrical and Electronics Engineers, Inc. All rights reserved.

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, [phone], fax [phone], or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, [phone], fax [phone], or online at http://www.wiley.com/go/permission.

Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates in the United States and other countries and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at [phone], outside the United States at [phone] or fax [phone].

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data applied for:

Hardback ISBN: 9781119863632

Cover Design: WileyCover Images: © bergamont/Shutterstock; SviatlanaLaza/Shutterstock; barleyman/Getty Images; 3alexd/Getty Images; Olemedia/Getty Images; Westend61/Getty Images; 3alexd/Getty Images; Monty Rakusen/Getty Images; Witthaya Prasongsin/Getty Images; Luca Sage/Getty Images; Nisian Hughes/Getty Images; Serhii Brovko/Getty Images

Editors Biography

Giancarlo Fortino (IEEE Fellow '22) is full professor of Computer Engineering in the Department of Informatics, Modeling, Electronics, and Systems at the University of Calabria (Unical), Italy. He has a PhD in Systems and Computer Engineering from University of Calabria in 2000. His research interests include wearable computing systems, Internet of Things, and cybersecurity. He is named Highly Cited Researcher 2002–2022 by Clarivate in Computer Science. He has authored more than 650 papers in international journals, conferences, and books. He is (founding) series editor of the IEEE Press Book Series on Human–Machine Systems and of the Springer Internet of Things series, and is Associate Editor of premier IEEE Transactions. He is cofounder and CEO of SenSysCal S.r.l., a Unical spinoff focused on innovative IoT systems, and cofounder of BigTech S.r.l., a startup focused on AI-driven systems and Big Data. Fortino is currently a member of the IEEE SMCS BoG and co-chair of the SMCS TC on IWCD.

David Kaber is currently the Dean's Leadership Professor and Chair of the Department of Industrial and Systems Engineering at the University of Florida (UF). Prior to joining UF, Kaber was a distinguished professor of industrial engineering at North Carolina State University where he also served as the Director of Research for the Ergonomics Center of North Carolina. Kaber's primary area of research interest is human-systems engineering with a focus on human–automaton interaction, including design and analysis for situation awareness in complex human in-the-loop systems. Domains of study for his research have included physical work systems, industrial safety systems, robotic systems, transportation systems, and healthcare. Kaber is a fellow of IEEE and previous editor-in-chief of the IEEE Transactions on Human–Machine Systems. He is a fellow of Institute of Industrial Engineers and a fellow of the Human Factors and Ergonomics Society. Kaber is also a Certified Human Factors Professional (BCPE) and a Certified Safety Professional (BCSP).

Andreas Nürnberger is professor of Data and Knowledge Engineering at the Otto-von-Guericke Universität Magdeburg (OVGU), Germany. His research focuses on adaptivity in human–machine systems, considering aspects such as user behavior analysis and intelligent user assistance. He was involved in the organization of many conferences and workshops in related areas and the development of new scientific events, among others, the IEEE SMCS sponsored international conference series on Human–Machine Systems (IEEE ICHMS). Andreas was visiting researcher at the University of Melbourne, Australia; postdoc at UC Berkeley, United States; and visiting professor at Université Pierre et Marie Curie, Paris. Andreas is an Emmy Noether Fellow of the German Science Foundation (DFG).

David Mendonça (Senior Member, 2012) is Senior Principal Decision Scientist at MITRE Corporation. He previously held the rank of professor in the Department of Industrial and Systems Engineering and in the Department of Cognitive Science at Rensselaer Polytechnic Institute. He served as a Program Director at the National Science Foundation from 2015 to 2017. He was a visiting scholar at the University of Lisbon (Portugal) and at Delft University of Technology (The Netherlands). He is currently a member of the Board of Governors of the IEEE Systems, Man and Cybernetics Society, and of the IEEE Boston (Massachusetts) Section. He holds a PhD in Decision Sciences and Engineering Systems from Rensselaer Polytechnic Institute, an MS from Carnegie Mellon University, and a BA from University of Massachusetts/Amherst.

List of Contributors

Marie-Hélène AbelHeudiasyc LaboratoryUniversity of Technology of CompiègneCompiègneFrance

Roohallah AlizadehsaniInstitute for Intelligent Systems Research and Innovation (IISRI)Deakin UniversityWaurn Ponds, VICAustralia

Terry AmoreseDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItaly

Abdelkader Nasreddine BelkacemDepartment of Computer and Network EngineeringCollege of Information TechnologyUnited Arab Emirates UniversityAl AinUAE

Emanuele BelliniDipartimento di Matematica e FisicaUniversitá degli Studi della Campania “Luigi Vanvitelli”CasertaItaly

Kawtar BenghaziUniversidad de GranadaGranadaSpain

Luigi BianchiDepartment of Civil Engineering and Computer Science Engineering“Tor Vergata” University of RomeRomeItaly

Maria Stella De BiaseDipartimento di Matematica e FisicaUniversitá degli Studi della Campania “Luigi Vanvitelli”CasertaItaly

Thomas BohnéDepartment of EngineeringInstitute for ManufacturingUniversity of CambridgeCambridgeUK

Stefanie S. BradleyIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Valentina BreschiDepartment of Electrical EngineeringEindhoven University of TechnologyEindhovenNetherlands

Zoraida CallejasUniversidad de GranadaGranadaSpain

Maximiliano CancheFaculty of MathematicsUniversidad Autónoma de YucatánMérida, YucatánMexico

Filippo CantucciTrust Theory and Technology GroupInstitute of Cognitive Sciences and TechnologiesNational Research Council of ItalyRomeItaly

Tom CarlsonAspire CreateUniversity College LondonStanmore, MiddlesexUK

Cristiano CastelfranchiTrust Theory and Technology GroupInstitute of Cognitive Sciences and TechnologiesNational Research Council of ItalyRomeItaly

Tom ChauIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Ricardo ChavarriagaCentre for Artificial IntelligenceSchool of EngineeringZurich University of Applied Sciences ZHAWWinterthurSwitzerlandandCLAIRE Office Switzerland, ZHAW digitalZurich University of Applied Sciences WinterthurZürichSwitzerland

Jessie Y. C. ChenU.S. Army Research LaboratoryAberdeen Proving Ground, MarylandUSA

Min ChenSchool of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina

Wei ChenState Key Lab of CAD&CGZhejiang UniversityHangzhouChina

Gennaro CordascoDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItaly

Alessandro Correa-VictorinoHeudiasyc LaboratoryUniversity of Technology of CompiègneCompiègneFrance

Gennaro CostagliolaDipartimento di InformaticaUniversità di SalernoVia Giovanni Paolo II, 132, 84084Fisciano (SA)Italy

Marialucia CucinielloDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItaly

Giuseppe D'AnielloDepartment of Information and Electrical Engineering and Applied MathematicsUniversity of SalernoFisciano (SA)Italy

Fabrizio DabbeneIstituto di Elettronica e Ingegneria dell'Informazione e delle Telecomunicazioni – IEIITCentro Nazionale delle RicercheTorinoItaly

Arwen H. DeCostanzaU.S. DEVCOMArmy Research LaboratoryAberdeen Proving Ground, MDUSA

Yulin DengCepheid Human Factors and Engineering TeamSunnyvale, CAUSA

Birsen DonmezDepartment of Mechanical and Industrial EngineeringUniversity of TorontoToronto, ONCanada

Anna EspositoDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItaly

Rino FalconeTrust Theory and Technology GroupInstitute of Cognitive Sciences and TechnologiesNational Research Council of ItalyRomeItaly

Tiago H. FalkInstitut national de la recherche scientifiqueUniversity of QuebecMontreal, QuebecCanada

Scott FangToronto Research CenterDefence Research and Development CanadaTorontoCanada

Philip S. E. FarrellToronto Research CenterDefence Research and Development CanadaTorontoCanada

Federico FaruffiniHeudiasyc LaboratoryUniversity of Technology of CompiègneCompiègneFranceandDIBRISUniversity of GenoaGenoaItaly

Francesco FlamminiSchool of Innovation, Design, and EngineeringMälardalen UniversityEskilstunaSweden

Erica D. FloreaniI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Domenico FormicaSchool of EngineeringNewcastle UniversityUK

Giancarlo FortinoDepartment of Informatics, ModelingElectronics and SystemsUniversity of CalabriaRendeItaly

Vittorio FuccellaDipartimento di InformaticaUniversità di SalernoFisciano (SA)Italy

Matteo GaetaDepartment of Information and Electrical Engineering and Applied MathematicsUniversity of SalernoFisciano (SA)Italy

Milad GeravandDeep Care GmbHWaiblingenGermany

Raffaele GravinaDepartment of Informatics, ModelingElectronics and SystemsUniversity of CalabriaRendeItaly

David GriolUniversidad de GranadaGranadaSpain

Stephanie GrossAustrian Research Institute for Artificial IntelligenceViennaAustria

Christoph Gugerg.tec medical engineering GmbHg.tecAustria

Bin GuoDepartment of intelligent computing systemSchool of Computer ScienceNorthwestern Polytechnical UniversityXi'anChina

Lydia HabibDepartment of Automation and ControlUniv. Polytechnique Hauts-de-France, CNRSUMR 8201 – LAMIHValenciennesFrance

Niloufaralsadat HashemiIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Klaus HauerAgaplesion Bethanien-Hospital, Geriatric CentreUniversity of HeidelbergHeidelbergGermany

Dengbo HeIntelligent Transportation Thrust, Systems HubHong Kong University of Science and Technology (Guangzhou)GuangzhouChinaandDepartment of Civil and Environmental EngineeringHong Kong University of Science and TechnologyHong Kong SARChina

Ina HeineOrganizational DevelopmentLaboratory for Machine Tools and Production EngineeringRWTH Aachen UniversityAachen, NRWGermany

Thomas HellebrandtOrganizational DevelopmentLaboratory for Machine Tools and Production EngineeringRWTH Aachen UniversityAachen, NRWGermany

Christine HornerIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Ming HouToronto Research CenterDefence Research and Development CanadaTorontoCanada

Louis HuebserOrganizational DevelopmentLaboratory for Machine Tools and Production EngineeringRWTH Aachen UniversityAachen, NRWGermany

Iztok HumarFaculty of Electrical EngineeringUniversity of LjubljanaLjubljanaSlovenia

Brian IrvineII Department of PediatricsAlberta Children's HospitalCalgaryCanadaandIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Zeanna JadavjiI BCI4KidsUniversity of CalgaryCalgaryCanada

Radmila JuricIndependent Researcher LondonUK

David B. KaberUniversity of FloridaDepartment of Industrial and Systems EngineeringGainesville, FLUSA

Dion KellyI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Abbas KhosraviInstitute for Intelligent Systems Research and Innovation (IISRI)Deakin UniversityWaurn Ponds, VICAustralia

Eli Kinney-LangI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Adam KirtonI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Brigitte KrennAustrian Research Institute for Artificial IntelligenceViennaAustria

Abderrahmane LakasDepartment of Computer and Network EngineeringCollege of Information TechnologyUnited Arab Emirates UniversityAl AinUAE

Qingyang LiDepartment of intelligent computing systemSchool of Computer ScienceNorthwestern Polytechnical UniversityXi'anChina

Bangli LiuSchool of Computer Science and InformaticsFaculty of Computing, Engineering, and MediaDe Montfort UniversityLeicester, EnglandUK

Honghai LiuSchool of Computing, Faculty of TechnologyUniversity of PortsmouthPortsmouth, EnglandUK

Jia LiuSchool of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina

Elliot LohDefence Research and Development CanadaNational Defence/Government of CanadaOttawa, ONCanada

Monika LohaniUniversity of UtahDepartment of PsychologySalt Lake City, UTUSA

Stefano MarroneDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItalyandDipartimento di Matematica e FisicaUniversitá degli Studi della Campania “Luigi Vanvitelli”CasertaItaly

Carlo MassaroniDepartmental Faculty of EngineeringUniversità Campus Bio-Medico di RomaRomeItaly

Zach McKinneyIEEE Standards AssociationPiscataway, NJUSA

David MendonçaMITRE CorporationBedford, MAUSA

Mirco MoencksDepartment of EngineeringInstitute for ManufacturingUniversity of CambridgeCambridgeUK

Darius NahavandiInstitute for Intelligent Systems Research and Innovation (IISRI)Deakin UniversityWaurn Ponds, VICAustralia

Martijn de NeelingDepartment of NeurologyAmsterdam University Medical Centers (UMC)AmsterdamNetherlands

Andreas NürnbergerFaculty of Computer ScienceOtto-von-Guericke-Universität MagdeburgMagdeburgGermany

Sergio F. OchoaComputer Science DepartmentUniversity of ChileSantiago, RMChile

Marie-Pierre Pacaux-LemoineDepartment of Automation and ControlUniv. Polytechnique Hauts-de-France, CNRSUMR 8201 – LAMIHValenciennesFrance

Marcos PadrónOrganizational DevelopmentLaboratory for Machine Tools and Production EngineeringRWTH Aachen UniversityAachen, NRWGermany

Junho ParkWm Michael Barnes ‘64 Department of Industrial and Systems EngineeringTexas A&M UniversityCollege Station, TXUSA

Angelika PeerFaculty of EngineeringFree University of BolzanoBolzanoItaly

Daniel PerovichComputer Science DepartmentUniversity of ChileSantiago, RMChile

Daniela lo PrestiDepartmental Faculty of EngineeringUniversità Campus Bio-Medico di RomaRomeItaly

Xueliang QuanSchool of Artificial Intelligence and AutomationHuazhong University of Science and TechnologyWuhanChina

Luigi RaianoDepartmental Faculty of EngineeringUniversità Campus Bio-Medico di RomaRomeItaly

Chiara RavazziIstituto di Elettronica e Ingegneria dell'Informazione e delle Telecomunicazioni – IEIITCentro Nazionale delle RicercheTorinoItaly

María Jesús Rodríguez-SánchezUniversidad de GranadaGranadaSpain

Mattia De RosaDipartimento di InformaticaUniversità di SalernoFisciano (SA)Italy

Elisa RothDepartment of EngineeringInstitute for ManufacturingUniversity of CambridgeCambridgeUK

Danette RowleyI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Ilyas SadybekovI BCI4KidsUniversity of CalgaryCalgaryCanadaandII Department of PediatricsAlberta Children's HospitalCalgaryCanada

Rodrigo SantosElectrical Engineering and Computers Department – ICICUniversidad Nacional del Sur – CONICETBahia Blanca, Buenos AiresArgentina

Alessandro SapienzaTrust Theory and Technology GroupInstitute of Cognitive Sciences and TechnologiesNational Research Council of ItalyRomeItaly

Emiliano SchenaDepartmental Faculty of EngineeringUniversità Campus Bio-Medico di RomaRomeItaly

Olga ShevalevaDepartment of PsychologyUniversità della Campania “L. Vanvitelli”CasertaItaly

Charlene K. StokesU.S. Army Futures CommandDEVCOMAberdeen Proving Ground, MDUSA

Silvia StradaDipartimento di ElettronicaInformazione e BioingegneriaPolitecnico di MilanoMilanoItaly

Mara TanelliIstituto di Elettronica e Ingegneria dell'Informazione e delle Telecomunicazioni – IEIITCentro Nazionale delle RicercheTorinoItalyandDepartment of Electrical EngineeringEindhoven University of TechnologyEindhovenNetherlands

Joshua Di ToccoDepartmental Faculty of EngineeringUniversità Campus Bio-Medico di RomaRomeItaly

Si Long Jenny TouIII Bloorview Research InstituteHolland Bloorview Kids Rehabilitation HospitalTorontoCanada

Lucas R. TrambaiolliMcLean HospitalHarvard Medical SchoolBelmont, MAUSA

Holland M. VasquezDepartment of Mechanical and Industrial EngineeringUniversity of TorontoToronto, ONCanada

Carl VogelTrinity Centre for Computing and Language StudiesSchool of Computer Science and StatisticsTrinity College DublinThe University of DublinDublinIreland

Ivan VolosyakFaculty of Technology and BionicsRhine-Waal University of Applied SciencesKleveGermany

Fei-Yue WangThe State Key Laboratory for Management and Control of Complex SystemsInstitute of AutomationChinese Academy of SciencesBeijingChina

Wenbi WangToronto Research CenterDefence Research and Development CanadaTorontoCanada

Xumeng WangCollege of Computer ScienceNankai UniversityTianjinChina

Andreas WendemuthDepartment of Electrical Engineering and Information TechnologyInstitute for Information Technology and CommunicationsOtto-von-Guericke-UniversityMagdeburgGermany

Christian WernerAgaplesion Bethanien-Hospital, Geriatric CentreUniversity of HeidelbergHeidelbergGermany

Dongrui WuSchool of Artificial Intelligence and AutomationHuazhong University of Science and TechnologyWuhanChina

Jinfeng XuSchool of Computer Science and TechnologyHuazhong University of Science and TechnologyWuhanChina

Zhiwen YuDepartment of intelligent computing systemSchool of Computer ScienceNorthwestern Polytechnical UniversityXi'anChina

Maryam ZahabiWm Michael Barnes ‘64 Department of Industrial and Systems EngineeringTexas A&M UniversityCollege Station, TXUSA

Andrea Maria ZanchettinPolitecnico di MilanoDipartimento di ElettronicaInformazione e BioningegneriaMilanItaly

Ephrem ZewdieI BCI4KidsUniversity of CalgaryCalgaryCanada

Preface

Human-Machine Systems (HMS) refer to integrated systems where the functions of human operators and machines are combined. These systems can be seen as a unified entity that interacts with the external environment. In an HMS, humans and machines work together, leveraging their respective capabilities to achieve specific goals or perform tasks.

HMS can take various forms and exist in different domains. Examples include aircraft flight control systems, industrial automation systems, medical robotic systems, and even virtual assistants or chatbots. In these systems, humans contribute their cognitive abilities, decision-making skills, and expertise, while machines provide computational power, automation, and precision.

The integration of humans and machines in HMS can be achieved through different levels of automation. It can range from fully manual systems, where humans are responsible for all tasks, to partially automated systems, where machines assist humans in specific functions, to fully autonomous systems, where machines take over the majority of tasks with minimal human intervention.

The field of HMS engineering focuses on understanding, designing, and optimizing these integrated systems. It involves studying human capabilities, limitations, and interactions, as well as developing technologies and interfaces that facilitate effective collaboration between humans and machines. The goal is to create systems that enhance human performance, improve efficiency, and ensure safety and reliability in various operational contexts.

Overall, HMS represents the synergy between humans and machines, harnessing the strengths of both to create powerful and efficient systems that can tackle complex tasks and challenges across different domains.

This book aims to be a manifesto of HMS research and development, delineating the state-of-the-art in the field, also by means of representative use cases, as well as future research challenges.

It is organized in seven areas covering all aspects of HMS: Brain–machine interfaces and systems (BMIS), Collaborative intelligent systems and applications (CISA), Companion technology (CT), Human–AI interaction and cognitive computing & engineering (HAICCE),

Human factors engineering (HFE), Interactive and wearable computing and systems (IWCS), and Hybrid Technologies (HT). Apart from the introduction chapter, the book includes 36 chapters contextualized in the aforementioned areas.

The chapters 2–6 are related to BMIS and deal with BCI (Brain-Computer Interface) methods and technologies. In particular, chapter 2 provides a general background on BCIs, chapter 3 focuses on BCI-enabled affective neurofeedback, chapter 4 introduces the adaptation of BCI techniques for children living with severe physical disability, chapter 5 proposes an application of BCI for controlling drones, and chapter 6 exploits BCI to allow people with disability to remotely control robots.

The chapters 7–13 refer to CISA and deal with advances on the development of scientific and engineering foundations, innovative technologies, and solutions for technology and data-driven collaborative intelligent systems. In particular, chapter 7 focuses on human–machine social systems such as human–machine teaming, chapter 8 introduces mechanisms for enabling human–robot interaction, chapter 9 explores people-driven collaborative processes enabled by human-machine units, chapter 10 reviews theoretical frameworks and recent advancements in human–machine transparency research and development, chapter 11 presents an overview of existing technology and methods for developing conversational human–machine interfaces, chapter 12 proposes an enduring strategy and methodology for complex socio-technical systems, and chapter 13 introduces a new computing paradigm of human–machine integration.

The chapters 14–17 belong to CT and deal with trans-disciplinary research in fields such as computer science and artificial intelligence, cognitive science, engineering, psychology, and neurobiology. In particular, chapter 14 overviews some background aspects of CT, chapter 15 focuses on mobility assistance robots of rollator-type and reviews their platforms and functionalities, chapter 16 focuses on the innovative concept of wearable affective robots, another instance of CT, and chapter 17 reviews the state-of-the-art on visual human–computer interaction in intelligent vehicles.

The chapters 18–22 focus on important topics in HAICCE and concern with human cognition in intelligent/AI system designs or/and development of AI algorithms, technology, methods for human beings. Specifically, chapter 18 addresses the problem of synchronization between human and robot actions, chapter 19 explores the new frontier of intelligent systems and their ability to enter into advanced collaboration with humans and other more or less intelligent artificial systems, chapter 20 focuses on decoding user emotional expressions as an important factor influencing interaction between humans and virtual assistants, chapter 21 reviews the intelligent computational edge, and chapter 22 addresses the important issue of implementing context awareness in autonomous vehicles.

The chapters 23–28 concern HFE and focus on the advancements in theory and practice related to human interaction with intelligent agents in a wide variety of environments. In particular, chapter 23 offers a systematic review on operator assistance systems, chapter 24 reviews cognitive performance modelling approaches in the human-systems engineering area, chapter 25 addresses advanced driver assistance systems, chapter 26 focuses on RGB-D based human action recognition, chapter 27 promotes the exploitation of AI in industry, and chapter 28 presents human factors in driving.

The chapters 29–33 are contextualized in IWCS and deal with advances and developments in interactive and/or wearable computing and systems. Specifically, chapter 29 presents a brief yet effective review of sensors, devices, and systems enabling wearable computing, chapter 30 concentrates on a technical use case about monitoring respiratory rate and activities using smart garments, chapter 31 highlights issues correlated with huge amounts of data generated by wearables, chapter 32 surveys the main methods and techniques for gesture-based computing, and chapter 33 reviews recent progress in EEG-based affective computing.

Finally, the chapters 34–37 deals with transversal topics related to HT. In particular, chapter 34 provides an overview of cyber-physical security applied to HMS, chapter 35 outlines the need and the role of standards in HMS, chapter 36 provide an overview of models, methods, and techniques of situation awareness measurement in the context of HMS, and chapter 37 defines a general and widely usable data-driven framework to perform human-centered service and process analysis.

1Introduction

Giancarlo Fortino1, David Kaber2, Andreas Nürnberger3, and David Mendonça4

1Department of Informatics, Modeling, Electronics and Systems, University of Calabria, Rende, Italy

2Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, USA

3Faculty of Computer Science, Otto-von-Guericke-Universität Magdeburg, Magdeburg, Germany

4MITRE Corporation, Bedford, MA, USA

1.1 Book Rationale

Human–machine systems (HMS) [6] are systems in which the functions of a human operator (or a group of operators) and a machine are integrated. This terminology can also be used to identify a system as a single entity that interacts with an external environment. HMS engineering is different from the more general and well-known fields such as human–computer interaction (HCI) [3] and sociotechnical engineering in that it focuses on complex, dynamic control systems that often are partially automated (such as flying an airplane). It also studies human problem-solving in naturalistic settings or in high-fidelity simulation environments. HMS area therefore includes subareas such as human/machine interaction; cognitive ergonomics and engineering; assistive/companion technologies; human/machine system modeling, test and evaluation; and fundamental issues of measurement and modeling of human-centered phenomena in engineered systems. More specifically, HMS inform theory and improve engineering practice by Bass [1]:

Taking into account human sensory, motor, and cognitive capabilities, knowledge, skills, preferences, emotions, limitations, biases, learning, and adaptation;

Considering human synchronous and asynchronous interactions among humans and with intelligent agents, computational support, and assistive devices via associated input and output technologies within the person's operational, organizational, cultural, and regulatory contexts;

Developing, instantiating, testing, and refining measures, methods, models, and apparatus that address (i) and (ii) and that can provide insights given real world imprecision, uncertainty, and constraints that impact human characteristics, performance, behavior, and learning; and

Supporting operational concept development, architecture, design, implementation, and evaluation of dynamic, complex systems that include human participants in their multifaceted roles (such as analyst, decision-maker, operator, collaborator, communicator, and learner).

The aim of this handbook is to be a manifesto of HMS research and development, delineating the state-of-the-art in the field, also by means of representative use cases, as well as future research challenges. Thus, the main target is the worldwide scientific and technical communities of HMS, specifically those associated with technical committees as part of important scientific/technical societies such as IEEE [7], IFAC [8], HFES [12], etc.

The book is the first volume addressing all the main issues of HMS and, as such, is a comprehensive handbook. This handbook has therefore a scope broader than a vertical book, which is to provide the “big picture” of the state-of-the-art of research (and technology) of HMS. With this aim, the handbook is organized around the most important well-established six areas of HMS (listed here in alphabetical order):

Brain–machine interfaces and systems

(

BMIS

)

include assistive technology and research allowing locked-in individuals to communicate and control exoskeletons/devices to improve locomotion

[9]

.

Collaborative intelligent systems and applications

(

CISA

)

address advances on the development of scientific and engineering foundations, innovative technologies, and solutions for technology- and data-driven collaborative intelligent systems

[10]

.

Companion technology

(

CT

)

covers and combines trans-disciplinary research in fields such as computer science and artificial intelligence, cognitive science, engineering, psychology, and neurobiology

[2]

.

Human–AI interaction and cognitive computing & engineering

(

HAICCE

)

encompasses human cognition in intelligent/AI system designs or/and development of AI algorithms, technology, methods for human beings

[11]

.

Human factors engineering

(

HFE

)

focus on the advancements in theory and practice related to human interaction with intelligent agents in a wide variety of environments

[4]

.

Interactive and wearable computing and systems

(

IWCS

)

focus on advances and developments in all aspects of interactive and/or wearable computing and systems

[5]

.

Moreover, the handbook also includes the Hybrid Technologies(HT) area focusing on aspects that cut across the above six areas (e.g. security, interoperability, and awareness).

1.2 Chapters Overview

The chapters of the handbook, which are summarized in the following list, were purposely collected and organized according to the aforementioned HMS areas and are ordered as follows: BMIS, CISA, CT, HAICCE, HFE, IWCS, and HT. For each chapter, we detail title, author/s, and a brief summary. The first chapter/s of each HMS area will also provide a brief literature review to give a big picture of research in the area. The other chapters are either more vertical reviews or technical use cases.

Brain–machine interfaces and systems (BMIS)

:

“Brain–Computer Interfaces: Recent Advances, Challenges, and Future Directions,” Tiago H. Falk, Christoph Guger, Ivan Volosyak

. In this chapter, the authors provide a general background on

Brain–computer interface

s (

BCI

s), from their introduction to recent findings and beyond. Specifically, they focus on active, passive, and hybrid BCI configurations. Some representative BCI applications are also discussed.

“Brain–computer interfaces for affective neurofeedback applications,” Lucas R. Trambaiolli, Tiago H. Falk

. Affective neurofeedback is a growing subfield of BCIs, as introduced in the second chapter. Participants are trained to achieve volitional control over neural patterns related to emotion regulation. The authors of this chapter present basic concepts necessary for neurofeedback protocols, including commonly used neuroimaging techniques and the main differences in targeted features. Two interesting applications for neurofeedback in psychiatry are also discussed, including depressive disorder and posttraumatic stress disorder.

“Pediatric Brain–Computer Interfaces: An Unmet Need,” Eli Kinney-Lang et al

. In this chapter, the authors discuss the adaptation of BCI techniques (described in the

Chapters 2

and

3

) for children living with severe physical disability. The authors state that it involves understanding fundamental differences in the neurophysiology, signal processing, and task design among other considerations for adults versus pediatric cases. This chapter therefore provides the necessary context and considerations to help stakeholders drive forward the full realization of pediatric BCI.

“Brain–Computer Interface-based Predator–Prey Drone Interactions,” Abdelkader Nasreddine

. In this chapter, the author proposes an application of BCI (described in

Chapter 2

) for control of drones. Specifically, the BCI method determines the fight path in a predator–prey situation to seek enemy drones and track them. The applications were evaluated with different testbeds where subjects were equipped with EEG systems. The results show that BCI subjects were able to accurately generate near-optimal trajectories by reacting quickly to a target's movements.

“Toward the Definition of Levels of Cooperation according to Human–Machine System Objectives and Constraints: example with Human-BCI-Robot cooperation,” Marie-Pierre Pacaux-Lemoine, Lydia Habib, Tom Carlson

. In this chapter, the authors focus on the challenge of designing safe and efficient human–machine systems. Specifically, they introduce the concept of levels of cooperation, which is a generic and flexible approach to support the designers in identifying: human–machine common goals, individual and cooperative abilities, and criteria for dynamic task allocation. The proposed method is exemplified via a use case showing how people with a high level of disability might use only a BCI interface (see

Chapters 2

and

5

for basics on BCI and BCI-controlled devices) to control a remote robot.

Collaborative Intelligent Systems and Applications (CISA):

“Human–Machine Social Systems,” Charlene K. Stokes, Arwen H. DeCostanza, Elliot Loh

. The focus of this chapter is on human–machine social systems (such as human–machine teaming) enabling intelligent, bidirectional interactions between humans and technologies that involve complex social interactions. The findings are applied in the military domain and examined in situ with real users in collaboration with the US military.

“The Role of Multimodal Data for Modeling Communication in Artificial Social Agents,” Stephanie Gross, Brigitte Krenn

. In this chapter, the authors introduce mechanisms for enabling human–robot interaction. The mechanisms are based on verbal and nonverbal communication cues that play a crucial role in interaction. In order to study human communication behavior in task-oriented scenarios, the authors present a selection of collected multimodal datasets and discuss the implications of their findings for modeling communicative behavior in artificial social agents.

“Modeling Interactions Happening in People-Driven Collaborative Processes,” Maximiliano Canche, Sergio F. Ochoa, Daniel Perovich, Rodrigo Santos

. This chapter explores people-driven collaborative processes, and the way to support interactions among participants using human–machine units. Specifically, the chapter surveys and analyzes the state-of-the-art on interaction (visual) modeling languages and techniques used to represent scenarios of interaction among human–machine units.

“Transparent Communications for Human-Machine Teaming,” Jessie Y.C. Chen

. This chapter reviews theoretical frameworks and recent advancements in human–machine transparency research and development. The analysis covers implementation of transparent human–machine interfaces across a wide spectrum of human–machine systems involving human interactions; for example, small ground robots; multiagent systems and human–swarm interaction; automated/autonomous driving systems; and Explainable AI systems.

“Conversational human–machine interfaces,” Marıa Jesus Rodrıguez-Sanchez, Kawtar Benghazi, David Griol, Zoraida Callejas

. In this chapter, the authors present an overview of existing technology and methods for developing conversational human–machine interfaces. In specific, current approaches for the development of dialog systems are described along with a focus on the different objectives that dialogs may have, either to solve particular tasks or to hold open chit-chat conversations.

“Interaction-Centered Design: An Enduring Strategy and Methodology for Complex Socio-Technical Systems,” Ming Hou, Scott Fang, Wenbi Wang, Philip S. E. Farrell, and Renee Chow

. With collective human–machine intelligence, human–machine symbiosis technologies are prevalent in society and capable of solving complex problems. To provide guidance to understand and mitigate potential risks associated with employing such technologies, the evolution of design strategy and methodology for intelligent adaptive systems was reviewed in this chapter. In specific, an

interaction-centered design

(

ICD

) framework, an associated set of methodologies and roadmap and a related trust model called IMPACTS were then introduced as an enduring strategy and appropriate solution. The ICD approach was validated via experimental studies for utility and effectiveness involving real-world technology demonstrations and evaluation activities.

“Human-Machine Computing: Paradigm, Challenges and Practices,” Zhiwen Yu, Qingyang Li, Bin Guo

. This chapter introduces a new computing paradigm of human–machine integration, which is called Human–Machine Computing. It combines capabilities of humans and machines in the computing process. In specific, a general platform named hmOS is constructed for managing and scheduling humans and machines that interactively work together in a task in a continuous development environment. It is worth noting that during the human–machine cooperation and collaboration procedure, the performance of machines and humans may gradually improve through mutual influence.

Companion technology (CT):

“Companion technology,” Andreas Wendemuth

. The central concept of the contribution of this chapter is Companion technology that is fundamental to a new class of cognitive systems which are correspondingly called Companion systems. Such systems are characterized by capabilities to adapt to requirements of their users, which change with time and situation. In particular, companion systems respond appropriately to user capabilities, preferences, and current needs. Their technical functionalities reflect changes in the environment or users' disposition. This chapter therefore presents the concepts of companion technology and illustrates the wide spectrum of its concepts and applications.

“A Survey on Rollator-Type Mobility Assistance Robots,” Milad Geravand, Christian Werner, Klaus Hauer, Angelika Peer

. This review chapter focuses on mobility assistance robots of rollator-type (a companion technology, see

Chapter 14

) and reviews their platforms and functionalities. Specifically, the reviewed systems are grouped according to their actuation, kinematic structure as well as employed sensors and human–machine interfaces. Reviewed functionalities include the following categories: sit-to-stand and stand-to-sit assistance, walking assistance, locomotion, and navigation assistance.

“Wearable Affective Robots,” Jia Liu, Yingying Jiang, Min Chen, Iztok Humar