Artificial Intelligence for Autonomous Vehicles E-Book
168,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Serie: Advances in Data Engineering and Machine Learning
- Sprache: Englisch
With the advent of advanced technologies in AI, driverless vehicles have elevated curiosity among various sectors of society. The automotive industry is in a technological boom with autonomous vehicle concepts. Autonomous driving is one of the crucial application areas of Artificial Intelligence (AI). Autonomous vehicles are armed with sensors, radars, and cameras. This made driverless technology possible in many parts of the world. In short, our traditional vehicle driving may swing to driverless technology. Many researchers are trying to come out with novel AI algorithms that are capable of handling driverless technology. The current existing algorithms are not able to support and elevate the concept of autonomous vehicles. This addresses the necessity of novel methods and tools focused to design and develop frameworks for autonomous vehicles.
There is a great demand for energy-efficient solutions for managing the data collected with the help of sensors. These operations are exclusively focused on non-traditional programming approaches and depend on machine learning techniques, which are part of AI. There are multiple issues that AI needs to resolve for us to achieve a reliable and safe driverless technology.
The purpose of this book is to find effective solutions to make autonomous vehicles a reality, presenting their challenges and endeavors. The major contribution of this book is to provide a bundle of AI solutions for driverless technology that can offer a safe, clean, and more convenient riskless mode of transportation.
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 442
Veröffentlichungsjahr: 2024
Ähnliche
Table of Contents
Cover
Table of Contents
Series Page
Title Page
Copyright Page
Preface
1 Artificial Intelligence in Autonomous Vehicles—A Survey of Trends and Challenges
1.1 Introduction
1.2 Research Trends of AI for AV
1.3 AV-Pipeline Activities
1.4 Datasets in the Literature of Autonomous Vehicles
1.5 Current Industry Standards in AV
1.6 Challenges and Opportunities in AV
1.7 Conclusion
References
2 Age of Computational AI for Autonomous Vehicles
2.1 Introduction
2.2 Autonomy
2.3 Classification of Technological Advances in Vehicle Technology
2.4 Vehicle Architecture Adaptation
2.5 Future Directions of Autonomous Driving
2.6 Conclusion
References
3 State of the Art of Artificial Intelligence Approaches Toward Driverless Technology
3.1 Introduction
3.2 Role of AI in Driverless Cars
3.3 Conclusion
References
4 A Survey on Architecture of Autonomous Vehicles
4.1 Introduction
4.2 A Study on Technologies Used in AV
4.3 Analysis on the Architecture of Autonomous Vehicles
4.4 Analysis on One of the Proposed Architectures
4.5 Functional Architecture of Autonomous Vehicles
4.6 Challenges in Building the Architecture of Autonomous Vehicles
4.7 Advantages of Autonomous Vehicles
4.8 Use Cases for Autonomous Vehicle Technology
4.9 Future Aspects of Autonomous Vehicles
4.10 Summary
References
5 Autonomous Car Driver Assistance System
5.1 Introduction
5.2 Related Work
5.3 Methodology
5.4 Results and Analysis
5.5 Conclusion
References
6 AI-Powered Drones for Healthcare Applications
6.1 Introduction
6.2 Kinds of Drones Used by Medical Professionals
6.3 Medical and Public Health Surveillance
6.4 Potential Benefits of Drones in the Healthcare Industry
6.5 Conclusion
References
7 An Approach for Avoiding Collisions with Obstacles in Order to Enable Autonomous Cars to Travel Through Both Static and Moving Environments
7.1 Introduction
7.2 Related Works
7.3 Methodology of the Proposed Work
7.4 Experimental Results and Analysis
7.5 Results and Analysis
7.6 Conclusion
References
8 Drivers’ Emotions’ Recognition Using Facial Expression from Live Video Clips in Autonomous Vehicles
8.1 Introduction
8.2 Related Work
8.3 Proposed Method
8.4 Results and Analysis
8.5 Conclusions
References
9 Models for the Driver Assistance System
9.1 Introduction
9.2 Related Survey
9.3 Proposed Methodology
9.4 Experimental Study
9.5 Conclusion
References
10 Control of Autonomous Underwater Vehicles
10.1 Introduction
10.2 Literature Review
10.3 Control Problem in AUV Control System
10.4 Methodology
10.5 Results
References
11 Security and Privacy Issues of AI in Autonomous Vehicles
11.1 Introduction
11.2 Development of Autonomous Cars with Existing Review
11.3 Automation Levels of Autonomous Vehicles
11.4 The Architecture of an Autonomous Vehicle
11.5 Threat Model
11.6 Autonomous Vehicles with AI in IoT-Enabled Environments
11.7 Physical Attacks Using AI Against Autonomous Vehicles
11.8 AI Cybersecurity Issues for Autonomous Vehicles
11.9 Cyberattack Defense Mechanisms
11.10 Solution Based on Machine Learning
11.11 Conclusion
References
Index
Also of Interest
End User License Agreement
List of Tables
Chapter 5
Table 5.1 Indian traffic police hand gestures.
Chapter 8
Table 8.1 Shallow version of emotion accuracy.
Table 8.2 Deep version of emotion accuracy.
Chapter 9
Table 9.1 Input parameters with results.
List of Illustrations
Chapter 1
Figure 1.1 Representation of the AV system.
Figure 1.2 Traffic signs from the GTSRB dataset [16].
Figure 1.3 Straight lane line.
Figure 1.4 Curved lane line.
Chapter 2
Figure 2.1 Summary of fatalities due to traffic accidents.
Figure 2.2 Primary components in driverless technology.
Figure 2.3 Schematic block diagram for autonomous vehicles.
Figure 2.4 Autonomous levels.
Figure 2.5 Fundamental navigation functions.
Figure 2.6 An insight of a self-driving car.
Figure 2.7 Training the model using deep learning techniques.
Figure 2.8 Communications between V2V and V2R.
Figure 2.9 Automated control system.
Figure 2.10 Identification of ambulance locations.
Figure 2.11 Alternate routes to reach the target.
Chapter 3
Figure 3.1 System architecture for selecting the leader car.
Chapter 4
Figure 4.1 An autonomous vehicle simulator in a traffic environment is propose...
Figure 4.2 Levels of vehicle autonomy.
Chapter 5
Figure 5.1 Aim of the proposed research.
Figure 5.2 Challenges for gesture rule recognition.
Figure 5.3 Vehicle features.
Figure 5.4 Challenges for vehicle light detection.
Figure 5.5 Road markings along the road.
Figure 5.6 Road lane marks affected by shadow.
Figure 5.7 Road lane marks affected by low illumination.
Chapter 6
Figure 6.1 Architecture of a complete drone.
Figure 6.2 Drug delivery flow diagram.
Figure 6.3 Block diagram of first aid delivery system.
Chapter 7
Figure 7.1 Driverless car.
Figure 7.2 Flow diagram of ground surface detection.
Figure 7.3 Input and the obstacle detection.
Figure 7.4 Image processing stage 2.
Figure 7.5 Image processing stage 3.
Figure 7.6 Image with the car detection.
Figure 7.7 Image processing background reduction.
Figure 7.8 Final result analysis.
Chapter 8
Figure 8.1 Flowchart for the testing phase.
Figure 8.2 Different types of facial emotions.
Figure 8.3 Segmentation of the Facial Expression Recognizer (FER) method. Resp...
Figure 8.4 Preprocessing and extraction/classifier of frame using CNN.
Figure 8.5 System architecture for real emotion recognition using movie clips....
Figure 8.6 Collections of grayscale images.
Figure 8.7 CNN architecture.
Figure 8.8 Matrix waveform for the sets of six global expressions.
Chapter 9
Figure 9.1 Input flowchart of the proposed work.
Figure 9.2 Input image.
Figure 9.3 Sample preprocessed images of NTHU drowsy driver detection dataset....
Figure 9.4 (a) Preprocessed image, and panels (b), (c), and (d) are the
Figure 9.5 (a) Face image, (b) eye blink, (c) yawning, and (d) head
Figure 9.6 Output image of histogram of oriented gradients (HOG).
Figure 9.7 Graphical representation of the proposed system performance.
Figure 9.8 Graphical comparison of the proposed system with dissimilar classif...
Chapter 10
Figure 10.1 Input image.
Figure 10.2 Log Gabor filtered image.
Figure 10.3 Autonomous underwater vehicle.
Figure 10.4 Detected number plate.
Figure 10.5 Recognized number.
Chapter 11
Figure 11.1 The layered architecture of the autonomous vehicle.
Figure 11.2 Automated vehicle.
Figure 11.3 Automation levels of autonomous cars.
Figure 11.4 Threat model (Karthik
et al.
, 2016).
Figure 11.5 IoT infrastructure for autonomous vehicles using AI.
Figure 11.6 Cyberattack defense mechanisms.
Guide
Cover Page
Series Page
Title Page
Copyright Page
Preface
Table of Contents
Begin Reading
Index
Also of Interest
WILEY END USER LICENSE AGREEMENT
Pages
ii
iii
iv
xi
xii
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
Scrivener Publishing
100 Cummings Center, Suite 541J Beverly, MA 01915-6106
Publishers at ScrivenerMartin Scrivener ([email protected])
Phillip Carmical ([email protected])
Artificial Intelligence for Autonomous Vehicles
Edited by
Sathiyaraj Rajendran
Munish Sabharwal
Yu-Chen Hu
Rajesh Kumar Dhanaraj
and
Balamurugan Balusamy
This edition first published 2024 by John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA and Scrivener Publishing LLC, 100 Cummings Center, Suite 541J, Beverly, MA 01915, USA© 2024 Scrivener Publishing LLCFor more information about Scrivener publications please visit www.scrivenerpublishing.com.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.
Wiley Global Headquarters111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.
Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials, or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read.
Library of Congress Cataloging-in-Publication Data
ISBN 9781119847465
Front cover images supplied by Pixabay.comCover design by Russell Richardson
Preface
With the advent of advanced technologies in AI, driverless vehicles have elevated curiosity among various segments of the society. The automotive industries are at the technological uprising with Autonomous vehicle concepts. Autonomous driving is one of the crucial application areas of Artificial Intelligence (AI). Autonomous vehicles are armed with sensors, radars and cameras. This made driverless technology possible in many parts of the world. In short, our traditional vehicle driving may swing to driverless technology.
The goal of this book is to highlight the efforts as well as obstacles involved in bringing self-driving cars to fruition while offering solutions that are feasible. The book’s primary accomplishment is the collection of AI-based notions for driverless technology, which can give a risk-free, risk-clean, and more convenient means of transportation. The book begins with a brief overview of autonomous vehicles, which serves as a survey of the latest developments and the need of AI-based solutions in the present and the future. The architecture of AI-based systems for monitoring and managing autonomous cars is covered in the next part, along with topics like AI drones, using sensors to detect obstructions, and identifying driver emotions to prevent collisions. The final portion covers a variety of AI-based models, autonomous underwater vehicle control, and security and privacy concerns related to these models.
This book is perfect for professionals, academicians, researchers, data scientists, data analysts, engineers, marketers, robotic scientists, e-commerce, and students. This book also outlines the future paths towards research into artificial intelligence on driverless technology, which has the potential to give a risk-free, risk-clean, and more convenient means of transportation.
1Artificial Intelligence in Autonomous Vehicles—A Survey of Trends and Challenges
Umamaheswari Rajasekaran, A. Malini* and Mahalakshmi Murugan
Thiagarajar College of Engineering, Madurai, India
Abstract
The potential for connected automated vehicles is multifaceted, and automated advancement deals with more of Internet of Things (IoTs) development enabling artificial intelligence (AI). Early advancements in engineering, electronics, and many other fields have inspired AI. There are several proposals of technologies used in automated vehicles. Automated vehicles contribute greatly toward traffic optimization and casualty reduction. In studying vehicle autonomy, there are two categories of development available: high-level system integrations like new-energy vehicles and intelligent transportation systems and the other involves backward subsystem advancement like sensor and information processing systems. The Advanced Driver Assistance System shows results that meet the expectations of real-world problems in vehicle autonomy. Situational intelligence that collects enormous amounts of data is considered for high-definition creation of city maps, land surveying, and quality checking of roads as well. The infotainment system of the transport covers the driver’s gesture recognition, language transaction, and perception of the surroundings with the assistance of a camera, Light Detection and Ranging (LiDAR), and Radio Detection And Ranging (RADAR) along with localization of the objects in the scene. This chapter discusses the history of autonomous vehicles (AV), trending research areas of artificial intelligence technology in AV, state-of-the-art datasets used for AV research, and several Machine Learning (ML)/Deep Learning (DL) algorithms constituting the functioning of AV as a system, concluding with the challenges and opportunities of AI in AV.
Keywords: Vehicle autonomy, artificial intelligence, situational intelligence, datasets, safety standards, network efficiency, algorithms
1.1 Introduction
An autonomous vehicle in simple terms is that its movements are from the start to predecided stop in “autopilot” mode. Autonomous vehicle technology is developed to provide several pros in comparison with human-driven transport. Increased safety on the road is one such potential primacy—connected vehicles could drastically decrease the number of casualties every year. The automated driving software is the most comfortable transport system that highly supports the class of people who could not drive because of age and physical constraints. Autonomous vehicles help them to find a new smart ideas, and it is predicted that it could provide them with different opportunities to work in fields that require driving. Automated decisions are taken without any human intervention, and the necessary actions are implemented to ensure stability of the system. The smart connected vehicles are supported by an AI-based self-driving system that responds to external conditions through the ability to sense their surroundings using Adaptive Driver Control (ADC) that uses lasers and radars enabled with ML and DL technologies. To extract object information from noisy sensor data, these components frequently employ machine learning (ML)-based algorithms. An understandable representation of autonomous vehicles as a system is shown in Figure 1.1.
The history of driverless cars took the spark from the year 1478 with Da Vinci creating the first driverless automobile prototype. In Da Vinci’s vehicle, there was a self-propelled robot that was driven by springs. It had programmable steering and could output predetermined routes. This brief history shows where the roots of artificial intelligence may be found in philosophy, literature, and the human imagination. Autonomous vehicles (AVs) were first conceptualized in the 1930s. It was the Houdina Radio Control project that demonstrated a radio-controlled “driverless” car. In the mid-20th century, General Motors took the initiative to develop a concept car called Firebird II. This was considered the basis for the first cruise control car named “Imperial” designed by the Chrysler company in 1958. Back in the 1990s, organizations slowly proceeded by considering safety measures including cruise and brake controls. After the turn of the century, blind-spot detection and electronic stability controls through sensors were made available in self-driven vehicles. One of the biggest achievements in the year 1995 was the VaMP-designed autonomous vehicle that drives (almost) by itself for 2,000 km. Between 1995 and 1998, the National AHS Consortium was held for cars followed by the PATH organization that conducted the automated bus and truck demos. In the year 2009, the Google Self-Driving Car Project began; eventually, Tesla Multinational Automotive released the update in autopilot software. Self-driving cars designed by Google, having met with a test drive accident in the year 2016, were considered a major damage to the development of AVs. At the end of 2016, the Bolt and Super Cruise in Cadillac have autonomous controls. At the start of January 2022, Volvo unveiled Ride Pilot, a new Level 3 autonomous driving technology, at the CES consumer electronics expo. Without human input, the system navigates the road using LiDAR, radar, ultrasonic sensors, and a 360-degree camera setup.
Figure 1.1 Representation of the AV system.
As of 2019, self-driven vehicles possess the following features: Free-hand handles the steering, but monitoring the system is also needed. With additional improvement in free-hand steering, the adaptive cruise control (ACC) keeps the vehicle at a predetermined displacement from the surrounding objects. When an interruption is encountered like the motorist crossing the lane, the car insists that the system slows down and changes the lane. One of the main problems with AV mobility is how to combine sensors and estimate circumstances to distinguish between risky scenarios and less dangerous ones. AV’s mobility is quite tedious, but continuous accepted releases in this field give better mobility to the vehicles. AI gives a fantastic change in the technological revolution. The Department of Transportation (DOT) and NHTSA deal with safety in addition to automation protocols. Section 1.2 of this chapter presents a detailed survey of machine learning and deep learning algorithms used in the AV literature where the important AV-pipeline activities are analyzed individually. Section 1.3 presents a survey of the state-of-the-art datasets used in autonomous vehicles, and Section 1.4 discusses the industry standards, risks, challenges, and opportunities, with Section 1.5 summarizing the analysis.
1.2 Research Trends of AI for AV
To provide readers a comprehensive picture of the status of the literature in AI on AV, the current research trends of ML/DL algorithms in the design of autonomous vehicles are presented.
The systems rely on several technological advancements in AI technicalities. When looking into the automation ideology in vehicles, it comprises six different levels. Starting from level 0, the human driver operates with no self-control in the cars. We could say that there is no self-working at level 0. At the first level, the Advanced Driver Assistance System (ADAS) in vehicles supports the driver with either accelerating or steering controls. In addition to level 1, the ADAS provides brake controls to the system such as Automated Emergency Braking System (AEBS). However, the driver must pay complete attention to the surroundings—level 2. At level 3, almost all automation processes are performed by the system, but driver’s access is required in certain operations. Real-time high-definition maps must be accessible, complete, and detailed. These maps are necessary, and they are used to decide their path and trajectory. Further development of level 3 is that the vehicle’s advanced driving system (ADS) provides complete automation without a human even if a person does not respond to a request in some circumstances. A vehicle’s six degrees of freedom are indicated by the stance, which is monitored by AV pose sensors: x, y, z, ϕ, θ, and ψ; here, x, y, and z depicts actual positions of the system at level 4. Eventually, at level 5, virtual chauffer concept is introduced by ADS along with ACC and does all the driving in all circumstances. Autonomous vehicles can be successfully implemented in a range of use case situations combining the complete working of sensors, actuators, maps, detectors, etc. It is necessary to demonstrate the functionalities in real-life environments like urban and rural developments combining smart vehicles.
LiDAR technology ensures the functional and safe operation of an autonomous vehicle. LiDAR is more efficient in creating 3D images of the surroundings that is considered critical in urban settings [1]. A rotating roof-mounted LiDAR sensor creates and maintains a live 3D map of a 60-m range of surroundings and thus generates a specific route for travel to the destination specified by the driver. Radars are mounted to measure the distance between obstructions and are placed at the front and rear. To determine the car’s position in the lane concerning the 3D map, a sensor on the left rear wheel tracks and signals the system’s sideway movement. All of the sensors are connected to the AI program, which receives the dataset in accordance. Thirdly, collaborative mapping efforts gather massive amounts of raw data at once utilizing GPS technologies to recover information on traffic signals, landmarks, and construction projects that cause deviations. By acquiring more knowledge on the NVH behavior of the entire vehicle, the vehicle’s performance systems can be improved. Utilizing Simcenter NVH systems has additional advantages, such as AI-based strategies for improving the effectiveness of NVH testing, and NVH performance prediction without the need for intricate simulation models. After the development of the 3D models, the developers focused on substances in the surroundings.
A revolutionary YOLOv2 vehicle architecture has been developed to address accuracy-based concerns, enhance sluggishness identification, and solve the lagging in classification criteria. An autonomous driving system was overlaid with augmented reality (AR), displayed on the windshield. AR vehicle display systems are necessary for navigation. Situational awareness has been considered as the command-line interface that was developed using AR technology. The INRIA dataset was used to identify vehicles and pedestrians as part of the demonstration of an AR-HUD-based driving safety instruction based on an article released by SAE International. SVM and HOG algorithms have been used to build the identification method, which detected partial obstacles with 72% and 74% accuracy in frames per second [2]. Since improvement in the above system has been demanded, technologists modeled a better version through the use of stereo cameras and augmented reality. The forward collision alert system detected vehicles and people and showed early warnings while applying an SVM classifier for apprehension that received 86.75% for identifying cars and 84.17% for identifying pedestrians [3]. Although it was challenging to establish the motion-detecting strategy, the model was able to control the car in confusion situations. This method established a rule base to handle illogical situations. The method might complete 45.6 m of designing a path in 50.2 s [4]. The study suggested the use of R-tree data structure and the continuous nearest neighbor search [5]. To simulate 3D rain, stereo pictures are used. The method can be used for already existing photographs, preventing the need to retake. The Recurrent Rolling Convolution (RRC) for object prediction and KITTI dataset were employed in the study’s tests at the Karlsruhe Institute of Technology. The results of testing 450 photographs reveal that the algorithm’s measured average precision decreases by 1.18% [6]. The most likely to happen 5G-enabled vehicle autonomy says that through C-RAN, autonomous and connected vehicles running on 5G will be able to access cloud-based ML/AI to make self-drive decisions. In high-mobility settings, edge computing and 5G can greatly speed up and improve data decryption. In these cases, edge computing-based data analysis and processing are automatically executed in the servers that are nearest to the cars. This might significantly lessen data traffic, remove Internet constraints for handling greater speed in data transfer, and lower the expense of transmission [7]. The researchers confirm that high-speed Internet access can improve data transaction, and eventually, autonomous driving is highly benefited by the same.
The stochastic Markov decision process (MDP) has been used to simulate how an autonomous vehicle interacts with its surroundings and learns to drive like an expert driver. The MDP model considers the road layout to account for a wider range of driving behaviors. The autonomous vehicle’s desired expert-like driving behavior is achieved by choosing the best driving strategy for the autonomous car. A deep neural network (DNN) has been used to make an approximation of the expert driver’s unknown reward function. The maximum entropy principle (MEP) has been used to train DNN. The simulations show the ideal driving characteristics of an autonomous vehicle. The use of the MEP has been modeled to train a DNN’s reward function. Additionally, essential derivations for applying the MEP to know the applications and workings of a reward function have also been offered (certain functionalities) [8].
1.3 AV-Pipeline Activities
Each domain was examined by looking at various approaches and techniques and comparing their benefits, drawbacks, results, and significance. To hasten the construction of a level 4 or 5 AVS, the examination of each domain is summarized as follows.
1.3.1 Vehicle Detection
This section analyzes deep learning methods, which are used for quicker and more precise vehicle identification and recognition in a variety of unpredictable driving situations. This paper suggested an online network architecture for identifying and tracking vehicles. Using the KITTI data-set [9] and a trajectory approximation of LSTM, the framework changed 3D poses when instances were moved around in a global coordinate system, outperforming long-range LiDAR results. LiDAR obtained 350.50 false negatives in a 30-m range. LiDAR-based false-negative scores for 50-m and 10-m tests have been mentioned as 857.08 and 1,572.33 [10]. To improve recognition speed, and as a way of resolving it, a novel YOLOv2 vehicle architecture was proposed [11]. One of the main problems with the real-time traffic monitoring approach was the low resolution of the images, which may be attributed to factors like poor weather. Vehicles in low-resolution photographs and videos were examined for this issue in terms of CNN’s effectiveness. The architecture utilized by the neural network operated in two stages: first, it detected high-level features, and then it detected low-level attributes. It evaluated the model’s capability to recognize automobiles at various degrees of input resolution. Results state that CNN is surprisingly positive in the identification even with low resolution [12]. Liu et al. demonstrated the least complex approach for AVS by combining a lightweight YOLO network. The technique was applied to a dataset that was created by themselves and, within 44.5 ms, achieved 90.38% precision. This could be a turning point for AVS, a quicker and more precise solution for diverse fields of vision and implementation during the day or at night [13].
1.3.2 Rear-End Collision Avoidance
Collision negatively influences road safety and the assurance for a safe ride. Wide and varied publications emphasize extensively for proposing methods to reduce collisions based on different approaches such as collision avoidance, collision warning, and collision risk assessment (VANET). These algorithms were implemented in vehicular ad hoc network. In this paper [14], the authors proposed a collision avoidance theory model. This model works by giving weights to the inputs. Certain characteristics like driver’s age, visual power, mental health, and physical health conditions are provided to the algorithm. The algorithm computes the weights for the characteristics, processes the input, and generates the output. Generally, the output is always a warning. The implementation of the algorithm was done in MATLAB, and the other platform of implementation was Vissim. The above paper was completely based on human factors. Whereas the other model called collision warning that was discussed by the authors considers human factors and weather situations and timings. This algorithm highly focuses on the visual power of humans and puts forth a visuality-based warning system. The MATLAB software and PreScan commercial software were used for testing the visuality-based warning system. This system is accepted to provide better performance than the other warning systems. To prevent crashes, a Multi-Factor-Based Road Accident Prevention System (MFBRAPS) was suggested. The implementation made use of MATLAB and NetLogo. Based on a previous study, the authors are retrieving new significant elements that might aid collision avoidance algorithms in a more efficient manner in this work. They added these fresh parameters to MFBRAPS to improve the V2V collision warning system. The suggested algorithm activates the various functions of the driver assistance system to control the speed and apply the brake after estimating the likelihood of an accident. The alarm is produced as necessary by the driver assistance system [15].
1.3.3 Traffic Signal and Sign Recognition
The literature on traffic sign recognition is very vast, and there are so many unsolved questions yet to be addressed by researchers from all over the world. Traffic sign detection is indispensable in the design of the autonomous vehicle, as the signs communicate commands to the driver in place to take appropriate actions. The results of traffic sign recognition using the GTSRB dataset is shown in Figure 1.2. A novel structure named Branch Convolutional Neural Network (B-CNN) is now changed into site visitors’ signal identification. A branch-output mechanism was introduced to the architecture and inserted between the pooling and convolutional layers to increase the recognition along with the machine’s speed and accuracy. The B-CNN-based technique performed exceptionally well in complicated visual settings [17]. The LeNet-5 CNN architecture was used to train the 16 varietal Korean traffic. The practice set consisted of true positives—25,000—and false positives—78,000 [18]. Additionally, a convolutional traffic light identification feature for AVS based on Quicker R-CNN is appropriate for traffic light identification as well as categorization. They applied their method to a sizable dataset called DriverU traffic light and got an average precision of 92%. However, restrictions on false positives will be depreciated by using a strategy or an RNN [19]. Traffic sign recognition based on the HSV color model by the LeNet-5 architecture with Adam optimizer was proposed and tested with the German technology-based traffic identification and yielded a median accuracy of 99.75 with 5.4 ms per frame [20]. DeepTLR traffic light recognition and classification system are a real-time vision-dependent, intricate, and very complex system that did not want positional information or temporal principles [14]. The GTSRB dataset served as the training set for majority of the deep learning techniques. In the LetNet-5-based CNN on a self-created dataset with spatial threshold segmentation using the HSV, the GTSRB dataset scored the best for traffic sign identification, despite a decline in performance in a complicated environment and the detection of separated signs. Despite a drop-off act in a composite setting and the apprehension of divided indications on account of the submitted extent, the GTSRB dataset outperformed the LetNet-5-based CNN on a designed dataset accompanying geographical opening separation adopting the HSV. Despite relying on prelabeled data, the YOLO method-based technique for traffic light detection with recognition of interior signs obtained the highest accuracy in the shortest amount of time from the above discussed [21].
Figure 1.2 Traffic signs from the GTSRB dataset [16].
1.3.4 Lane Detection and Tracking
Lane detection and tracking have been considered as essential modules in a complete autonomous vehicle system. In various literatures, it has been considered as an important field of research in autonomous vehicles, and many algorithms have been proposed to solve the encountered problems of lane detection and tracking. It is an important module that solves the perception problems of autonomous vehicles. In comparison with straight lane detection, some literatures introduce curved road lane detection tracking as challenging and important to avoid accidents. Figure 1.3 and Figure 1.4 shows straight lane line and curved lane line respectively. This paper categorized the lane detection methods into three as feature-based, model-based, and methods based on other trending technologies and discussed a novel curved road detection algorithm, where the road images were initially divided into ROI and the background. The ROI has been further divided into linear and curved portions. The proposed approach evolved as a series of mathematical equations and Hough transformation. It has been concluded that the proposed algorithm effectively identifies lane boundaries and to be effective enough to assist drivers, improving safety standards [22].
Figure 1.3 Straight lane line.
Figure 1.4 Curved lane line.
This paper presented a lane detection and tracking approach based on Gaussian sum particle filter. The proposed feature extraction approach in this paper was based on the linear propagation of boundary and lane points while zooming in. Three types of algorithms used for tracking, namely, SIR particle filter, Gaussian particle filter, and the used GSPF, are compared, and it has been concluded that GSPF outperforms the other two widely used tracking algorithms [23]. This paper presented a survey of trends and methods in road and lane detection, highlighting the research gaps. Seven different modalities, namely, monocular vision, LiDAR, stereo imaging, radar, vehicle dynamics, GPS, and GIS, have been utilized for perception of roads and lanes. The survey highlighted the gap in the literature of perception problems of multiple-lane roads and nonlinear lanes. Detection of lane split and merges has been proposed as an unnoticed perception problem to be solved in the near future for the development of fully autonomous vehicles. Lack of established benchmarks has also been discussed as an important issue in order to compare the currently efficient methods in different scenarios [24]. This paper compared the computer vision-based lane detection techniques with sensor-based algorithms. The traditional modules in conventional computer vision-based lane detection algorithms as highlighted in the survey are image preprocessing, feature extraction, fitting the lane, and the tracking module. Popular edge detection operators such as Canny, Sobel, Prewitt, Line detection operators, and thresholding techniques have been highlighted as the popularly used feature extraction techniques based on the assumptions in the structure, texture, and color of the road. The potential of deep learning for automatic feature extraction and to learn the environment with most variance conditions has been highlighted. The survey also criticized the lack of bench-marked datasets and standard evaluation metrics to test the efficiency of the algorithms [25]. In this paper [26], the authors presented a detailed survey of deep learning methods used in lane detection. The survey categorized the state-of-the art methods into two based on the number of modules as two-step and single-step methods based on feature extraction and postprocessing. Classification, object detection, and segmentation-based architectures have been detailed as three types of model architectures used for lane detection. The paper also discussed the evaluation metrics of different algorithms based on the TuSimple dataset, which has been regarded as the largest dataset for lane detection. Based on the analysis, the CNNLSTM(SegNet+) architecture has been reported to have the highest accuracy of 97.3%. The computational complexity of deep learning models like CNN and RNN has been emphasized as a disadvantage for implementation on low-memory devices. In this paper [27], the authors applied a new CNN learning algorithm for lane detection, namely, Extreme Learning Machine (ELM), to reduce the training time, also by using a relatively smaller data-set for training. The training time of the ELCNN has been reported to be 800 times faster, and the experimental results established that incorporating ELM into CNN also increases the performance of the model based on accuracy. This paper emphasized an edge-cloud computing-based CNN architecture for lane detection, where the concepts of edge computing and cloud computing have been introduced to increase the data-processing efficiency for real-time lane detection. Their approach achieved fair accuracy than other state-of-the art algorithms in challenging scenarios of insufficient light, shadow occlusion, missing lane line, and curved lane line. The accuracy achieved on normal road conditions was 97.57, and all of the evaluation metrics have been derived based on the TuSimple data-set [28]. In this paper [29], the author proposed a multiple-frame-based deep learning lane detection architecture using CNN and RNN to avoid the ambiguities that arise due to detections based on a single frame in the presence of shadow, occlusion of vehicles, etc. The continuous scene-based architecture has CNN layers, where the multiple-frame input-outputs constitute time series data that are fed into the recurrent neural network. The CNN architecture acts as the encoder decoder circuitry for reducing the size of the input, and the LSTM architecture determines the next scene, thus detecting the lane. The work laid a strong foundation for using CNN-RNN architectures for detecting lanes in challenging scenarios. It established the idea that the situations that cannot be analyzed using a single-image input can be improved by using a multiple-frame input architecture. In this paper [30], the authors introduced YOLO-based obstacle and lane detection algorithms based on video inputs and TuSimple datasets. The inverse perspective transform has been used for constructing the bird’s-eye view of the lane. The deep learning models are criticized for their poor performance, and their YOLO-based architecture achieved an accuracy of 97.9% for lane detection within a time span of 0.0021 s. This increased speed of processing has been claimed advantageous for implementation in real-time autonomous vehicle scenarios.
1.3.5 Pedestrian Detection
One of the main vision-based issues for AVS is accurately recognizing and localizing pedestrians on roads in a variety of settings. The study of vehicle autonomy aims at increasing the accuracy in identifying and localizing the pedestrians to prevent fatalities. Numerous studies have been successful in lowering accident rates and developing a more accurate and long-lasting approach to autonomous driving technologies. The approaches used for pedestrian detection can be broadly categorized into two as those that use man-in-the-loop feature extraction and deep learning-based methods. For pedestrian detection, a most sought deep learning model is the convolutional neural network, which has been used in many studies to detect pedestrians. In that way, a deep grid named large-field-of-view (LFOV) was imported to uniformly skim perplexing representations of walkers. The submitted structure was created to analyze and create categorization judgments across a broad extent of neighborhoods concurrently and efficiently. The LFOV network can implicitly reuse calculations since it analyzes enormous regions at higher speed that are substantially faster than those of standard deep networks. This pedestrian detection system demonstrated a convincing performance for practical application, taking 280 ms per image on GPU and having an average miss rate of 35.85% [31]. For passersby discovery, a new CNN structural version was proposed [32]. To annotate the positions of pedestrians in the model, they used synthetic photographs and transfer learning, together with a bounding box suggestion for an uncovered region. When crowded scenarios were taken into account, it got a 26% lost count in the CUHK08 dataset and a 14% missing proportion in the Caltech walker dataset. The major benefit was that it did not need a region proposal method and did not require explicit detection during training.
In this paper [33], the authors presented a detailed survey of the state-ofthe-art pedestrian detection algorithms in the domain of automobiles and safety. The position of humans on the lane, pose, color, environment, weather, texture, etc., increase the complexity of the pedestrian detection problem, making the classical template matching approach to be inefficient for real-time detections. The feature-based detection approach requires man-in-theloop feature extraction methods, whereas deep learning models self-learn features during training. This has been highlighted as an advantage of deep learning-based detection methods where the models can achieve fair accuracy with a huge training set; however, both the approaches have been considered to be implemented on par with each other. CNN has been regarded as the most widely used algorithm for pedestrian detection. Bounding box, missing rate, TP, TN, FP, FN, False Positives per Window, AMR, and IoU are regarded as the most commonly used evaluation metrics. It has been concluded that accuracy and cost are still a trade-off to be balanced in pedestrian detection, as the robust solution to this complex problem is yet to be proposed more efficiently. In this paper [34], the authors exemplified the similarities between pedestrian detection pipeline and object detection pipeline and conducted experiments to optimize a CNN-based detection pipeline for pedestrian detection. Sliding Window, Selective Search, and LDCF algorithms have been tested for best efficiency in the selection of candidate regions. Selective Search has been concluded not suitable, the Sliding Window approach achieves better recall, and the LDCF algorithm has been concluded as the best region proposal algorithm. Their approach has been evaluated based on the Caltech-Pedestrian dataset using error metrics such as MR and FPPI. The entire experiment has been tested on the NVIDIA Jetson TK1, the most widely used hardware in smart cars. In this paper [35], the authors proposed a hybrid approach by combining HOV, LUV, and CNN classifier, namely, the multilayer-channel feature framework. It has been discussed that by eliminating the overlapping windows, the cost of CNN has been reduced. The proposed approach has been evaluated in the Caltech, KITTI, INRIA, TUD-Brussels, and ETH datasets. The performance of MCF has been considered fair in comparison with the state-of-the-art methods. In this paper [36], the authors proposed a feature sharing-based deep neural network architecture for pedestrian detection, thereby reducing the computational cost of feature extraction during the model training phase. Feature sharing has been emphasized among the different ConvNet-based detectors that differ in their model window sizes. INRIA and Caltech have been used for the evaluation of their approach, and it has been reported using four ConvNet detector units; the time for detection decreased almost by 50%. In this paper [37], the authors criticized the slow detection speed of conventional deep learning algorithms and used YOLO architecture for pedestrian detection. The experiments have been trained using the INRIA dataset and a few selected images from PASCAL VOC. YOLOv2 has been concluded to have a higher detection speed than the presented approach in the paper. In this paper [38], the authors proposed a two-modular pedestrian detection architecture based on convolutional neural networks. In the region generation module, contrasting to image pyramids, feature pyramids are used to capture features of different resolutions. The Modified ResNet-50 architecture has been used as the background network of both modules. The deep supervision-based region prediction module thus achieved a precision score of 88.6% and an MR of 8.9%. It has been concluded that a fair trade-off balance has been achieved between real-time predictions and accuracy. In this paper [39], the authors proposes a convolutional neural network architecture for real-time pedestrian detection tasks. The approach signified a lightweight architecture for real-time detection and warning to avoid accidents. A regression-based problem-solving approach has been implemented in contrast to the conventional classification modules to increase the inference speed. Pruning and quantization techniques have been highlighted as the model compression techniques employed. The observed inference speed was 35 frames per second with a maP of 87% in the specifically designed hardware for autonomous vehicles. Hazy weather pedestrian detection has been considered even more challenging than the clear-day pedestrian detection. In this paper [40], the authors proposed a hazy weather pedestrian detection framework based on a modified YOLO architecture, reducing cost, thereby improving accuracy. The contributions made in the research work also include a hazy weather pedestrian detection dataset. They present three YOLO-based detection models, namely, Simple-YOLO, VggPrioriboxes-YOLO, and MNPrioriboxes-YOLO. The MNPrioriboxes-YOLO that uses depth-wise convolutions based on the MobileNetV2 architecture has the least number of parameters among all of the other methods. It has been concluded that the strategies employed reduced the number of parameters, thereby decreasing the running time of the algorithm. In this paper [41], the authors presented an approach to pedestrian detection in nighttime scenarios using multispectral images based on CNN. The authors studied different image fusion and CNN fusion methods. The pixel-level image fusion technique has been considered superior to CNN fusion methods. All of the experimental conclusions presented in the research work were based on the KAIST multispectral database. In this paper [42], the authors considered the multispectral approach to pedestrian detection as a solution to solve challenges encountered in cases of detection in poor illumination, occlusion, etc. The R-FCN-based detector has been used, and information on both the color and thermal images is fused using a network in the network model. Their approach evaluated based on the KAIST dataset revealed that the proposed approach has better performance than the other state-of-the-art methods implemented for multispectral images.
1.4 Datasets in the Literature of Autonomous Vehicles
Datasets contain advanced and to a certain extent upgraded information in many scholarly studies by benefiting to solve legitimate-experience models of issues. They enable quantitative analysis of methods, revealing important information about their strengths and weaknesses. The performance of algorithms must be ensured on real-world datasets. Algorithms used, for instance, must deal with complicated objects and settings while contending with difficult environmental factors like direct illumination, shadows, and rain. This section surveys the current datasets used in the literature of autonomous vehicles.
1.4.1 Stereo and 3D Reconstruction
To evaluate the effectiveness of stereo matching algorithms, the Middlebury stereo was developed in 2002 that created the Middlebury Multi-View Stereo (MVS). Being only two scenes in size, the benchmark was crucial in the development of MVS methods. In comparison, 124 diverse scenarios were captured in a controlled laboratory environment. Light scans from each camera position are combined to create reference data, and the resulting scans are extremely dense, with an average of 13.4 million points per scan. The whole 360-degree model for 44 sceneries were created by rotating and scanning in contrast to the datasets so far. High-resolution data were captured. DSLR images and synced stereo videos in various interior and outdoor settings were also included. With the use of a reliable procedure, all images can be registered using a high-precision laser scanner. The evaluation of a thorough 3D reconstruction is made possible by high-resolution photographs [43].
1.4.2 Optical Flow
The mobility concept in AI has a huge capability and upgrades the working efficiency by collecting enormous amount of data, and norms will direct the system to the output platform; thereby, it meets the goal of vehicle autonomy. A toothbrush is used to track concealed fluorescent presence on the objects in all non-fixed sequences to decide the base reality progress. The dataset is made up of eight distinct sequences with a frame count of 8 each. For each sequence, a pair receives the ground truth flow [44]. Contrary to other datasets, the Middlebury dataset offers extremely accurate and deep ground truth, which enables the evaluation of sub-pixel precision. Accurate reference data are generated by monitoring pixels over extensively sampled space-time volumes using high-speed video cameras. With this technique, optical flow ground truth may be automatically acquired in difficult everyday settings, and realistic elements like motion blur can be added to compare approaches under various circumstances.
1.4.3 Recognition and Segmentation of Objects
The Microsoft COCO dataset was introduced in 2014 for object detection. They offer visual representations with intricate scenarios with typical items established in their normal surroundings. The count of situations by means of division for Microsoft COCO is indeed above in consideration of the PASCAL VOC target separation standard. The dataset contains 328k photographs, 2.5 million commented details, and 91 object classes. In a significant crowd, all objects have per-instance segmentations tagged on them. The intersection-over-union measure is employed for evaluation, much like PASCAL VOC [45].
1.4.4 Tracking Datasets
Multi-object tracking benchmarks consists of 14 difficult video sequences shot in unrestricted contexts with both stationary and moving cameras. The trio groups were annotated: moving or stationary walkers, humans who are not standing up straight, and others. They assess the approaches using two well-known tracking standards, MOTA—Multiple Object Tracking Accuracy, and the other one is MOTP—Multiple Object Tracking Precision [46].
1.4.5 Datasets for Aerial Images
Data from air sensors for the detection of urban objects as well as the reconstruction and segmentation of 3D buildings were indicated by the ISPRS benchmark. It is made up of the Vaihingen and Downtown Toronto databases. The scene in a road setup is the object classes taken into account in the object detection process. Three locations with different object classes and sizable road detection test methods are available in the Vaihingen data-set. There are two smaller sections for building reconstruction and object extraction, similar to Vaihingen.
1.4.6 Sensor Synchronization Datasets
A camera’s sync connects different levels and angles of photographs. The exposure trigger time is represented by the time stamp of the image, and the full rotation of the current LiDAR frame is represented by the time stamp of the LiDAR scan. This method typically produces good data alignment since the camera’s sync is instantaneous [47].
1.5 Current Industry Standards in AV
Innovative technologies like AVs come with risks and consequences that could reduce society’s acceptance. These risks include those related to the environment, market, society, organizations, politics, economy, technology, and turbulence. The examination of 86 documents produced by 29 leading AV technology firms summarizes the industry standards. The main topic of this essay is a technological risk that is defined as potential adverse social, economic, and physical effects linked to individuals’ worries about utilizing cutting-edge technologies. The consequence is that since no single framework can adequately account for the wide range of potential machine decisional applications, we should not try to create a generic “one size fits all” model of artificial intelligence application. AVs are related to five different categories of technological risk: security, privacy, cybersecurity, liability, and industry impact. Governments must implement new policies and laws to resolve the risks connected with AVs to guarantee the society with benefits as much as possible from the developing AV sector. According to estimates, human error is to blame for at least 90% of automobile collisions. By surpassing drivers (humans) in perception, decision-making, and execution, AV adoption potentially reduces or eliminates the main cause of auto accidents. According to authors Collingwood and Litman, very less drivers use seatbelts and pedestrians may become less cautious as a result of feeling safer. This means that a nonlinear relational model of classification notions must be constructed and that the technology must be accepted at face value to appropriately frame each unique choice situation. Additionally, the absence of human mistakes does not imply the absence of mechanical error. The possibility of scientific weaknesses risking vehicle security hikes in addition to the ramification of telecommunications. The catastrophic Tesla automatic steering system accident in 2016 emphasizes the obstructions of components’ skill in order to avoid accidents and discovered the vagueness of the developed arrangement understanding. Concerns are raised about how “crash algorithms” should be used to program AVs to react in the event of inevitable accidents. Rules to control AVs’ responses to moral standards are necessary because the harm produced by AVs in accidents cannot be subjectively assessed due to the “absence of responsibility.” The overall tone of the AV reports examined for this study is predominantly favorable. However, this finding must be taken in the context of the fact that these reports are prepared for a specific group of stakeholders, including investors, consumers, and regulatory bodies. The AV industry papers make numerous references to ethical dilemmas, even though deficient in the definiteness and insight of the descriptions in the experimental essay. It manifests that prevalent reliability, sustainability, comfort, human listening, control, and scrutinizing, as well as the science-policy relationship, are the ethical issues that were handled by most organizations, with safety and cybersecurity coming in second.
1.6 Challenges and Opportunities in AV
Without a doubt, the driverless car has many benefits, such as providing a means of transportation for people who cannot drive and reducing the driver’s stress on roads. However, in addition to these positive outcomes, numerous obstacles were encountered and they have to addressed to implement a successful AV model. The subsequent are few of the predominant difficulties alongside driverless jeeps:
