Artificial Intelligent Techniques for Wireless Communication and Networking E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 509

Veröffentlichungsjahr: 2022

Beschreibung

ARTIFICIAL INTELLIGENT TECHNIQUES FOR WIRELESS COMMUNICATION AND NETWORKING The 20 chapters address AI principles and techniques used in wireless communication and networking and outline their benefit, function, and future role in the field. Wireless communication and networking based on AI concepts and techniques are explored in this book, specifically focusing on the current research in the field by highlighting empirical results along with theoretical concepts. The possibility of applying AI mechanisms towards security aspects in the communication domain is elaborated; also explored is the application side of integrated technologies that enhance AI-based innovations, insights, intelligent predictions, cost optimization, inventory management, identification processes, classification mechanisms, cooperative spectrum sensing techniques, ad-hoc network architecture, and protocol and simulation-based environments. Audience Researchers, industry IT engineers, and graduate students working on and implementing AI-based wireless sensor networks, 5G, IoT, deep learning, reinforcement learning, and robotics in WSN, and related technologies.

Leseprobe

Table of Contents

Cover

Title Page

Copyright

Preface

1 Comprehensive and Self-Contained Introduction to Deep Reinforcement Learning

1.1 Introduction

1.2 Comprehensive Study

1.3 Deep Reinforcement Learning: Value-Based and Policy-Based Learning

1.4 Applications and Challenges of Applying Reinforcement Learning to Real-World

1.5 Conclusion

References

2 Impact of AI in 5G Wireless Technologies and Communication Systems

2.1 Introduction

2.2 Integrated Services of AI in 5G and 5G in AI

2.3 Artificial Intelligence and 5G in the Industrial Space

2.4 Future Research and Challenges of Artificial Intelligence in Mobile Networks

2.5 Conclusion

References

3 Artificial Intelligence Revolution in Logistics and Supply Chain Management

3.1 Introduction

3.2 Theory—AI in Logistics and Supply Chain Market

3.3 Factors to Propel Business Into the Future Harnessing Automation

3.4 Conclusion

References

4 An Empirical Study of Crop Yield Prediction Using Reinforcement Learning

4.1 Introduction

4.2 An Overview of Reinforcement Learning in Agriculture

4.3 Reinforcement Learning Startups for Crop Prediction

4.4 Conclusion

References

5 Cost Optimization for Inventory Management in Blockchain and Cloud

5.1 Introduction

5.2 Blockchain: The Future of Inventory Management

5.3 Cost Optimization for Blockchain Inventory Management in Cloud

5.4 Cost Reduction Strategies in Blockchain Inventory Management in Cloud

5.5 Conclusion

References

6 Review of Deep Learning Architectures Used for Identification and Classification of Plant Leaf Diseases

6.1 Introduction

6.2 Literature Review

6.3 Proposed Idea

6.4 Reference Gap

6.5 Conclusion

References

7 Generating Art and Music Using Deep Neural Networks

7.1 Introduction

7.2 Related Works

7.3 System Architecture

7.4 System Development

7.5 Algorithm-LSTM

7.6 Result

7.7 Conclusions

References

8 Deep Learning Era for Future 6G Wireless Communications—Theory, Applications, and Challenges

8.1 Introduction

8.2 Study of Wireless Technology

8.3 Deep Learning Enabled 6G Wireless Communication

8.4 Applications and Future Research Directions

Conclusion

References

9 Robust Cooperative Spectrum Sensing Techniques for a Practical Framework Employing Cognitive Radios in 5G Networks

9.1 Introduction

9.2 Spectrum Sensing in Cognitive Radio Networks

9.3 Collaborative Spectrum Sensing for Opportunistic Access in Fading Environments

9.4 Cooperative Sensing Among Cognitive Radios

9.5 Cluster-Based Cooperative Spectrum Sensing for Cognitive Radio Systems

9.6 Spectrum Agile Radios: Utilization and Sensing Architectures

9.7 Some Fundamental Limits on Cognitive Radio

9.8 Cooperative Strategies and Capacity Theorems for Relay Networks

9.9 Research Challenges in Cooperative Communication

9.10 Conclusion

References

10 Natural Language Processing

10.1 Introduction

10.2 Conclusions

References

11 Class Level Multi-Feature Semantic Similarity-Based Efficient Multimedia Big Data Retrieval

11.1 Introduction

11.2 Literature Review

11.3 Class Level Semantic Similarity-Based Retrieval

11.4 Results and Discussion

Conclusion

References

12 Supervised Learning Approaches for Underwater Scalar Sensory Data Modeling With Diurnal Changes

12.1 Introduction

12.2 Literature Survey

12.3 Proposed Work

12.4 Results

12.5 Conclusion and Future Work

References

13 Multi-Layer UAV Ad Hoc Network Architecture, Protocol and Simulation

13.1 Introduction

13.2 Background

13.3 Issues and Gap Identified

13.4 Main Focus of the Chapter

13.5 Mobility

13.6 Routing Protocol

13.7 High Altitude Platforms (HAPs)

13.8 Connectivity Graph Metrics

13.9 Aerial Vehicle Network Simulator (AVENs)

13.10 Conclusion

References

14 Artificial Intelligence in Logistics and Supply Chain

14.1 Introduction to Logistics and Supply Chain

14.2 Recent Research Avenues in Supply Chain

14.3 Importance and Impact of AI

14.4 Research Gap of AI-Based Supply Chain

References

15 Hereditary Factor-Based Multi-Featured Algorithm for Early Diabetes Detection Using Machine Learning

15.1 Introduction

15.2 Literature Review

15.3 Objectives of the Proposed System

15.4 Proposed System

15.5 HIVE and R as Evaluation Tools

15.6 Decision Trees

15.7 Results and Discussions

15.8 Conclusion

References

16 Adaptive and Intelligent Opportunistic Routing Using Enhanced Feedback Mechanism

16.1 Introduction

16.2 Related Study

16.3 System Model

16.4 Experiments and Results

16.5 Conclusion

References

17 Enabling Artificial Intelligence and Cyber Security in Smart Manufacturing

17.1 Introduction

17.2 New Development of Artificial Intelligence

17.3 Artificial Intelligence Facilitates the Development of Intelligent Manufacturing

17.4 Current Status and Problems of Green Manufacturing

17.5 Artificial Intelligence for Green Manufacturing

17.6 Detailed Description of Common Encryption Algorithms

17.7 Current and Future Works

17.8 Conclusion

References

18 Deep Learning in 5G Networks

18.1 5G Networks

18.2 Artificial Intelligence and 5G Networks

18.3 Deep Learning in 5G Networks

Conclusion

References

19 EIDR Umpiring Security Models for Wireless Sensor Networks

19.1 Introduction

19.2 A Review of Various Routing Protocols

19.3 Scope of Chapter

19.4 Conclusions and Future Work

References

20 Artificial Intelligence in Wireless Communication

20.1 Introduction

20.2 Artificial Intelligence: A Grand Jewel Mine

20.3 Wireless Communication: An Overview

20.4 Wireless Revolution

20.5 The Present Times

20.6 Artificial Intelligence in Wireless Communication

20.7 Artificial Neural Network

20.8 The Deployment of 5G

20.9 Looking Into the Features of 5G

20.10 AI and the Internet of Things (IoT)

20.11 Artificial Intelligence in Software-Defined Networks (SDN)

20.12 Artificial Intelligence in Network Function Virtualization

20.13 Conclusion

References

Index

Also of Interest

End User License Agreement

List of Tables

Chapter 11

Table 11.1 Experimental details.

Table 11.2 Analysis on clustering accuracy vs no. of classes.

Table 11.3 Performance analysis in retrieval accuracy vs no. of classes.

Table 11.4 Performance on false classification ratio vs no. of classes.

Table 11.5 Performance analysis on time complexity vs no. of classes.

Table 11.6 Performance on clustering accuracy vs no. of terms/relations.

Table 11.7 Performance analysis in retrieval accuracy vs no. of terms/relations.

Table 11.8 Analysis on false classification ratio vs no. of terms/relations.

Table 11.9 Performance analysis on time complexity vs no. of terms/relations.

Chapter 12

Table 12.1 SeaOne temperature data considered during day time.

Table 12.2 SeaOne temperature considered during night time.

Table 12.3 SeaOne temperature data considered during day time.

Table 12.4 SeaOne temperature considered during night time.

Table 12.5 SeaOne temperature data considered during day time.

Table 12.6 SeaOne temperature considered during night time.

Table 12.7 SeaOne temperature data considered during day time.

Table 12.8 SeaOne temperature considered during night time.

Table 12.9 Network information in statistical analysis for sample-1.

Table 12.10 Model summary for calculating relative errors in testing data.

Table 12.11 Network information in statistical analysis for sample-1 at night.

Table 12.12 Model summary for calculating relative errors in testing data at nig...

Table 12.13 Network information in statistical analysis for sample-2.

Table 12.14 Model summary for calculating relative errors in testing data at nig...

Table 12.15 Case processing summary for sample 1 data obtained from Table 12.1.

Table 12.16 ACF function for Table 12.1 data.

Table 12.17 Case processing summary for sample 1 and Table 12.2 data.

Table 12.18 ACF function for Table 12.2 data.

Table 12.19 PACF function for Table 2 data.

Table 12.20 Absorption loss in underwater environment for input of real time tem...

Table 12.21 Transmission loss in underwater environment for input of real time t...

Chapter 15

Table 15.1 Tools and techniques in healthcare.

Table 15.2 Attributes and distinct values.

Chapter 19

Table 19.1 Rate of production for GETUS, plain AODV, ETUS, IDSEM and EIDR.

Table 19.2 Communication overhead for GETUS, plain AODV, ETUS, IDSEM and EIDR.

List of Illustrations

Chapter 1

Figure 1.1 Reinforcement learning process.

Figure 1.2 Markov process.

Figure 1.3 Raw images of State.

Figure 1.4 Value based learning.

Figure 1.5 Policy based learning.

Chapter 2

Figure 2.1 Growth of 5G Connections worldwide.

Figure 2.2 5G Market analysis.

Figure 2.3 AI in next generation networks.

Figure 2.4 Service providers achieving benefits through AI.

Chapter 3

Figure 3.1 AI in supply chain and logistics market.

Figure 3.2 Growth rate ranking of AI in logistics and supply chain market ecosys...

Chapter 4

Figure 4.1 General neural network framework for crop yield analysis [7].

Figure 4.2 Reinforcement learning [5].

Figure 4.3 Time series forecast of agricultural reinforcement learning [8].

Figure 4.4 Deep Q Networks in RL [13].

Figure 4.5 Deep Q Network for crop prediction [11].

Chapter 5

Figure 5.1 Blockchain based inventory management in cloud.

Figure 5.2 Inventory financing scenario.

Figure 5.3 Blockchain example.

Chapter 6

Figure 6.1 Plant disease identification and localization.

Figure 6.2 Convolution block of proposed system.

Figure 6.3 (a) Leaf which has brown spot. (b) Leaf blast with background noise. ...

Figure 6.4 (a) Leaf which has identified brown spot using Yolo. (b) Identified l...

Chapter 7

Figure 7.1 System architecture.

Figure 7.2 Art module.

Figure 7.3 Music module.

Figure 7.4 (a) Content image (b) Style image.

Figure 7.5 Transformation module.

Figure 7.6 (a) VGG16 data set image. (b) VGG16 architecture.

Figure 7.7 (a) L content squared-error loss function. (b) Derivative of loss wit...

Figure 7.8 Working of LSTM with RNN.

Figure 7.9 (a) Sample input. (b) Sample output.

Chapter 8

Figure 8.1 6G vision [9].

Figure 8.2 Wireless technologies—Overview.

Figure 8.3 History of wireless technology [12].

Figure 8.4 AI enabled 6G.

Figure 8.5 Deep learning enabled intelligent 6G networks.

Figure 8.6 Drone of IoT in 6G.

Figure 8.7 Deep learning in 6G—Future [3].

Chapter 9

Figure 9.1 Spectrum sensing requirements.

Figure 9.2 Spectrum agile radio.

Figure 9.3 Fundamental limits on cognitive radio.

Figure 9.4 Research challenges in cooperative communication.

Chapter 11

Figure 11.1 Architecture of proposed multimedia big retrieval system.

Figure 11.2 Comparison on clustering performance vs no. of classes.

Figure 11.3 Comparison on retrieval accuracy vs. no. of classes.

Figure 11.4 Comparison on false classification ratio vs no. of classes.

Figure 11.5 Comparison on time complexity vs no. of classes.

Figure 11.6 Comparison on clustering performance vs no. of terms/relations.

Figure 11.7 Comparison on retrieval accuracy vs. no. of terms/relations.

Figure 11.8 Comparison on false classification ratio vs no. of terms/relations.

Figure 11.9 Comparison on time complexity vs no. of terms/relations.

Chapter 12

Figure 12.1 Multi-layer perceptron model using different depth coordinates.

Figure 12.2 Multi-layer perceptron model using different depth coordinates at ni...

Figure 12.3 Multi-layer perceptron model using different depth coordinates at ni...

Figure 12.4 Autocorrelation function and lag for temperature data in Table 12.1.

Figure 12.5 Partial Autocorrelation function and lag for temperature data in Tab...

Figure 12.6 Autocorrelation Function and lag for temperature data in Table 12.2.

Figure 12.7 Partial Autocorrelation function and lag for temperature data in Tab...

Chapter 13

Figure 13.1 Flying-ad hoc-networks.

Figure 13.2 Multi-layer UAV ad hoc network [6].

Figure 13.3 A group of five mobile node movements using the RPGM model [7].

Figure 13.4 Data-centric routing.

Figure 13.5 FANET augmented with a HAP station.

Figure 13.6 Aerial vehicle network simulator integration.

Chapter 14

Figure 14.1 Elements of logistics and supply chain.

Figure 14.2 Transportation network model.

Figure 14.3 Inventory routing problem.

Figure 14.4 Inventory routing problem using JADE.

Figure 14.5 Inventory routing problem using multi-agent model.

Figure 14.6 Overview of reverse logistics.

Figure 14.7 Elements of green supply chain.

Figure 14.8 Healthcare supple chain network.

Figure 14.9 Hospital outpatient simulation model.

Figure 14.10 System Dynamic (SD) simulation model of replenishment quantity.

Figure 14.11 Networked manufacturing of supply chain network.

Figure 14.12 Humanitarian supply chain network.

Chapter 15

Figure 15.1 Architecture of the proposed system.

Figure 15.2 Decision tree derived with updated attributes.

Figure 15.3 Formation of decision trees.

Figure 15.4 Significant order dataset formation.

Figure 15.5 Decision tree with hereditary factor.

Figure 15.6 (a) Comparison of classifiers on proposed method vs ENORA vs NGSA. (...

Chapter 16

Figure 16.1 Typical architecture of wireless mesh networks.

Figure 16.2 Feedback frame format.

Figure 16.3 Throughput comparison POR Vs conventional opportunistic routing prot...

Figure 16.4 Packet loss ratio.

Figure 16.5 Average transmit power—POR.

Figure 16.6 Node test bed.

Chapter 17

Figure 17.1 Characteristics, design principles and enabling technology defining ...

Figure 17.2 New models, means, and forms of intelligent manufacturing.

Figure 17.3 Four industrial revolutions.

Figure 17.4 Several sub-problems in Artificial intelligence.

Figure 17.5 Technical architecture of a typical knowledge graph [25].

Figure 17.6 3DES structure.

Figure 17.7 Framework of current and future works.

Chapter 18

Figure 18.1 Evolution of 5G networks.

Figure 18.2 5G network architecture.

Figure 18.3 Applications of 5G.

Figure 18.4 Requirements of 5G.

Figure 18.5 AI and 5G technology.

Figure 18.6 AI for RAN optimization.

Figure 18.7 Integrated Access Backhaul (IAB).

Figure 18.8 Research challenges identified in 5G network.

Figure 18.9 Recurrent neural network (RNN) model for traffic prediction.

Figure 18.10 3D CNN model for traffic prediction.

Figure 18.11 RNN and 3D CNN combined model for traffic prediction.

Figure 18.12 The hyper parameters for RNN and 3D-CNN.

Chapter 20

Figure 20.1 The cognitive radio concept architecture.

Figure 20.2 A simple Artificial Neural Network.

Figure 20.3 Some uses of the Internet of Things.

Figure 20.4 SDN architecture.

Guide

Cover

Table of Contents

Title Page

Copyright

Preface

Begin Reading

Index

Also of Interest

End User License Agreement

Pages

v

ii

iii

iv

xvii

xviii

xix

xx

xxi

xxii

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

351

Scrivener Publishing100 Cummings Center, Suite 541JBeverly, MA 01915-6106

Artificial Intelligence and So Computing for Industrial Transformation

Series Editor: Dr. S. Balamurugan ([email protected])

Scope: Artificial Intelligence and So Computing Techniques play an impeccable role in industrial transformation. The topics to be covered in this book series include Artificial Intelligence, Machine Learning, Deep Learning, Neural Networks, Fuzzy Logic, Genetic Algorithms, Particle Swarm Optimization, Evolutionary Algorithms, Nature Inspired Algorithms, Simulated Annealing, Metaheuristics, Cuckoo Search, Fire˝y Optimization, Bio-inspired Algorithms, Ant Colony Optimization, Heuristic Search Techniques, Reinforcement Learning, Inductive Learning, Statistical Learning, Supervised and Unsupervised Learning, Association Learning and Clustering, Reasoning, Support Vector Machine, Di˙erential Evolution Algorithms, Expert Systems, Neuro Fuzzy Hybrid Systems, Genetic Neuro Hybrid Systems, Genetic Fuzzy Hybrid Systems and other Hybridized So Computing Techniques and their applications for Industrial Transformation. The book series is aimed to provide comprehensive handbooks and reference books for the benefit of scientists, research scholars, students and industry professional working towards next generation industrial transformation.

Publishers at ScrivenerMartin Scrivener ([email protected])Phillip Carmical ([email protected])

Artificial Intelligent Techniques for Wireless Communication and Networking

Edited by

R. Kanthavel

K. Ananthajothi

S. Balamurugan

R. Karthik Ganesh

This edition first published 2022 by John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA and Scrivener Publishing LLC, 100 Cummings Center, Suite 541J, Beverly, MA 01915, USA

© 2022 Scrivener Publishing LLC

For more information about Scrivener publications please visit www.scrivenerpublishing.com.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

Wiley Global Headquarters

111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials, or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read.

Library of Congress Cataloging-in-Publication Data

ISBN 978-1-119-82127-4

Cover image: Pixabay.Com

Cover design by Russell Richardson

Set in size of 11pt and Minion Pro by Manila Typesetting Company, Makati, Philippines

Printed in the USA

10 9 8 7 6 5 4 3 2 1

Preface

In the current digital era, artificial intelligence (AI) resembling human intelligence is enabling superior inventions for the advancement of the world. Broadening the scientific scope of AI has made it possible to change fundamentals and modulate everlasting facts in the wireless communication and networking domains. This breakthrough in AI miraculously preserves the inspired vision of communication technology.

Wireless communication and networking based on AI concepts and techniques are explored in this book, specifically focusing on the current research in the field by highlighting empirical results along with theoretical concepts. The possibility of applying AI mechanisms towards security aspects in the communication domain is also elaborated. Moreover, the application side of integrated technologies are also explored to enhance AI Revolution innovations, insights, intelligent predictions, cost optimization, inventory management, identification processes, classification mechanisms, cooperative spectrum sensing techniques, ad-hoc network architecture, and protocol and simulation-based environments.

This book allows both practitioners and researchers to share their opinions and recent research on the convergence of these technologies with those in academia and industry. The contributors have presented their technical evaluation and comparative analysis of existing technologies; and theoretical explanations and experimental case studies related to real-time scenarios are also included. Furthermore, this book will connect IT professionals, researchers, and academicians working on 5G communication and networking technologies.

The book is organized into 20 chapters that address AI principles and techniques used in wireless communication and networking. It outlines the benefits, functions, and future role of AI in wireless communication and networking. In addition, AI applications are addressed from a variety of aspects, including basic principles and prominent methodologies that offer researchers relevant instructions to follow in their research. The editing team and expert reviewers in various disciplines have thoroughly reviewed the information included.

– In

Chapter 1

, “Comprehensive and Self-Contained Introduction to Deep Reinforcement Learning,” P. Anbalagan, S. Saravanan and R. Saminathan present a brief guide to the deep reinforcement learning process and its detailed applications and research directions in order to enhance the basics of reinforcement mechanisms.

– In

Chapter 2

, “Impact of AI in 5G Wireless Technologies and Communication Systems,” A. Sivasundari and K. Ananthajothi present an in-depth overview of the implementation of AI to improve 5G wireless communication systems, discuss the role and difficulties faced, and highlight suggestions for future studies on integrating advanced AI into 5G wireless communications.

– In

Chapter 3

, “Artificial Intelligence Revolution in Logistics and Supply Chain Management,” P.J. Satish Kumar, Ratna Kamala Petla, Elangovan K and P.G. Kuppusamy give a brief description of recent developments and some relevant impacts concerning logistics and supply chain related to AI.

– In

Chapter 4

, “An Empirical Study of Crop Yield Prediction Using Reinforcement Learning,” M. P. Vaishnnave and R. Manivannan use reinforcement learning (RL) data technologies and high-performance computing to create new possibilities to activate, calculate, and recognize the agricultural crop forecast.

– In

Chapter 5

, “Cost Optimization for Inventory Management in Blockchain Under Cloud,” C. Govindasamy, A. Antonidoss and A. Pandiaraj investigate some of the most prominent bases of blockchain for inventory management of blockchain and cost optimization methods for inventory management in cloud.

– In

Chapter 6

, “Review of Deep Learning Architectures Used for Identification and Classification of Plant Leaf Diseases,” G. Gangadevi and C. Jayakumar describe the deep learning architectures for plant disease prediction and classification using standard techniques like artificial neural network (ANN), k-means classifier (K-means), recurrent neural network (RNN), k-nearest neighbor (K-NN) classifier, and support vector machine (SVM).

– In

Chapter 7

, “Generating Art and Music Using Deep Neural Networks,” A. Pandiaraj, Lakshmana Prakash, R. Gopal and P. Rajesh Kanna develop a model using deep neural nets that can imagine like humans and generate art and music of their own. This model can also be used to increase cognitive efficiency in AGI (artificial general intelligence), thereby improving the agent’s image classification and object localization.

– In

Chapter 8

, “Deep Learning Era for Future 6G Wireless Communications – Theory, Applications, and Challenges,” S. K. B. Sangeetha, R. Dhaya, S. Gayathri, K. Kamala and S. Divya Keerthi encapsulate the background of 6G wireless communication with details on how deep learning has made a contribution to 6G wireless technology and also highlight future research directions for deep learning-driven wireless technology.

– In

Chapter 9

, “Robust Cooperative Spectrum Sensing Techniques for a Practical Framework Employing Cognitive Radios in 5G Networks,” J. Banumathi, S.K B. Sangeetha and R. Dhaya discuss the use of cooperative spectrum sensing in cognitive radio systems to improve the efficiency of detecting primary users.

– In

Chapter 10

, “Natural Language Processing,” S. Meera and B. Persis Urban Ivy elucidate natural language processing (NLP), which is a branch of computer science and artificial intelligence that studies how computers and humans communicate in natural language, with the aim of computers understanding language as well as humans do.

– In

Chapter 11

, “Class-Level Multi-Feature Semantic Similarity-Based Efficient Multimedia Big Data Retrieval,” Sujatha D, Subramaniam M and A. Kathirvel present an efficient class-level multi-feature semantic similarity measure-based approach. The proposed method receives the input query and estimates class-level information similarity, class-level texture similarity, and class-level semantic similarity measures for different classes.

– In

Chapter 12

, “Supervised Learning Approaches for Underwater Scalar Sensory Data Modeling with Diurnal Changes,” J. V. Anand, T. R. Ganesh Babu, R. Praveena and K. Vidhya describe an inter-depth variation profile in line with latitude and longitude in a particular area, calculate attenuations by the temperature coefficient of real data sets, and attribute attenuation to a frequency-dependent loss.

– In

Chapter 13

, “Multi-Layer UAV Ad-Hoc Network Architecture, Protocol and Simulation,” Kamlesh Lakhwani, Tejpreet Singh Orchu and Aruna discuss the use of flying ad hoc networks (FANETs) to provide communication among the UAVs. In FANET, the selection of a suitable mobility model is a critical task for researchers.

– In

Chapter 14

, “Artificial Intelligence in Logistics and Supply Chain,” Jeyaraju Jayaprakash explains the source of activities in any white product manufacturing and service industry. This logistics and supply chain network will become more complex over the next few decades as a result of pandemic situations, natural disasters, increasing population, and other side entities developing smart strategies.

– In

Chapter 15

, “Hereditary Factor-Based Multi-Feature Algorithm for Early Diabetes Detection Using Machine Learning,” S. Deepajothi, R. Juliana, Aruna S K and R. Thiagarajan indicate that the predominance of diabetes mellitus among the global population ultimately leads to blindness and death in some cases. The proposed model attempts to design and deliver an intelligent solution for predicting diabetes in the early stages and address the problem of late detection and diagnosis.

– In

Chapter 16

, “Adaptive and Intelligent Opportunistic Routing Using Enhanced Feedback Mechanism,” V. Sharmila, Mandal K, Shankar Shalani and P. Ezhumalai discuss opportunistic routing of an intercepted packet to provide an effective wireless mesh network. Traditional opportunistic routing algorithms are being used to provide high-speed use batching of packets, which is a complex task. Therefore, an enhanced opportunistic feedback-based algorithm is proposed in this chapter in which individual packet forwarding uses a new route calculation in the proposed work that takes into consideration the cost of transmitting feedback and the capacity of the nodes to choose appropriate rates for monitoring operating conditions.

– In

Chapter 17

, “Enabling Artificial Intelligence and Cyber Security in Smart Manufacturing,” R. Satheesh Kumar, G. Keerthana, L. Murali, S. Chidambaranathan, C.D. Premkumar and R. Mahaveerakannan propose an efficient green manufacturing approach in SM systems with the aid of AI and cyber security frameworks. The proposed work employs a dual-stage artificial neural network (ANN) to find the design configuration of SM systems in industries. Then, for maintaining data confidentiality while communicating, the data are encrypted using the 3DES approach.

– In

Chapter 18

, “Deep Learning in 5G Networks,” G. Kavitha, P. Rupa Ezhil Arasi and G. Kalaimani discuss a 3D-CNN model combined with RNN model for analyzing and classifying the network traffic into three various classes, such as maximum, average and minimum traffic, which proves that the combined 3D-CNN and RNN model provides better classification of network traffic.

– In

Chapter 19

, “EIDR Umpiring Security Models for Wireless Sensor Networks,” A. Kathirvel, S. Navaneethan and M. Subramaniam provide an overview of WSNs with their classification, as well as comparisons between different routing algorithms and the proposed EIDR (enhanced intrusion detection and response). Soundness of proposed EIDR is tested using QualNet 5.0.

– In

Chapter 20

, “Artificial Intelligence in Wireless Communication,” Prashant Hemrajani, Vijaypal Singh Dhaka, Manoj Kumar Bohra and Amisha Kirti Gupta describe the applications of AI techniques in wireless communication technologies and networking, which can bring these changes through new research. Also, AI/ML techniques can improve the current state of network management, operations and automation. They can further support software-defined networking (SDN) and network function virtualization (NFV), which are considered important wireless communication technology components for the deployment of 5G and higher generation communication systems.

All of the contributors to this book deserve our heartfelt gratitude. With the fervent cooperation of the editorial director and production editor at Scrivener Publishing, we aspire to cross many more milestones to glory in the future academic year.

Prof. Dr. R. Kanthavel

Department of Computer Engineering, King Khalid UniversityAbha, Saudi Arabia

Dr. K. Ananthajothi

Department of Computer Science and Engineering Misrimal Navajee Munoth Jain Engineering CollegeChennai, India

Dr. S. Balamurugan

Founder and Chairman, Albert Einstein Engineering and Research Labs (AEER Labs)Vice-Chairman, Renewable Energy Society of India, India

Dr. R. Karthik Ganesh

Department of Computer Science and EngineeringSCAD College of Engineering and Technology Cheranmahadevi, IndiaJanuary 2022

1Comprehensive and Self-Contained Introduction to Deep Reinforcement Learning

P. Anbalagan*, S. Saravanan and R. Saminathan

Department of Computer Science and Engineering, Annamalai University, Annamalai Nagar, India

Abstract

Deep reinforcement learning is a type of machine learning and artificial intelligence in which smart robots, similar to the way people make good decisions, can learn from their actions. Implicit in this form of machine learning is that, depending on their behavior, an agent is rewarded or punished. Including unsupervised machine learning and supervised learning, reinforcement learning is yet another common type of artificial intelligence development. Deep reinforcement learning can lead to incredibly impressive results beyond normal reinforcement learning, due to the fact that it incorporates the core qualities of both deep learning and reinforcement learning. Since this is becoming a very broad and rapidly growing field, the entire application landscape will not be explored, but mainly based on comprehensive and self contained introduction to deep reinforcement learning. The goal of this chapter is twofold: (i) to provide a brief guide to the deep reinforcement learning process; (ii) to present detailed applications and research directions.

Keywords: Artificial intelligence, deep learning, machine learning, reinforcement learning

1.1 Introduction

Due to its effectiveness in solving complex sequential decision-making issues, Reinforcement Learning (RL) has become increasingly common over the past few years. Many of these accomplishments are due to the integration of deep learning techniques with RL. But, thanks to its ability to learn various levels of abstractions from data, deep RL has been effective in complex tasks with lower prior knowledge. For example, from visual perceptual inputs made up of thousands of pixels, a deep RL agent can successfully learn [14]. Deep RL also has potential for real-world areas such as medical, self-driving cars, finance and smart grids, to name a few. Nonetheless in implementing deep RL algorithms, many problems arise. The area of machine learning that deals with sequential decision-making is reinforcement learning (RL) [16, 20].

As an agent who has to make decisions in an atmosphere to maximize a given definition of accumulated rewards, the RL problem can be formalized. It will become apparent that this formalization extends to a wide range of tasks and captures many important characteristics of artificial intelligence, such as a sense of cause and effect, as well as a sense of doubt and non-determinism [5].

A main feature of RL is that good behavior is taught by an agent. This suggests that it incrementally modifies or acquires new habits and abilities. Another significant feature of RL is that it uses experience of trial and error (as opposed to for example, dynamic programming that a priori assumes maximum environmental knowledge). Therefore the RL agent does not need full environmental awareness or control; it just needs to be able to communicate with the environment and gather information. The knowledge is gained a priori in an offline environment, then it is used as a batch for learning (the offline setting is therefore also called batch RL) [3].

Figure 1.1 Reinforcement learning process.

In comparison to the online world, this is where information becomes available in a linear order and is used to change the agent’s actions gradually. The core learning algorithms are exactly the same in both situations, but the key difference is that the agent will affect how it gathers experience in an online environment. This is an important difficulty, primarily because while studying, the agent has to deal with the problem of exploration/exploitation. But learning in the online world can also be a benefit, as the agent can collect data directly about the most important part of the environment. For that purpose, RL approaches may provide the most algorithmically efficient solution in practice, even when the context is fully understood, compared to other dynamic programming methods that would have been inefficient due to this lack of precision [8].

Deep reinforcement learning contains aspects of neural networks and learning with reinforcement (Figure 1.1). Deep reinforcement learning is achieved using two different methods: deep Q-learning and policy specular highlights. Deep Q-learning techniques attempt to anticipate the rewards will accompany certain steps taken in a particular state, while policy gradient strategies seek to optimize the operational space, predicting the behavior themselves. Policy-based approaches of deep reinforcement learning are either stochastic in architecture. Certainly, probabilistic measures map states to policies, while probabilistic policies build probabilistic models for behavior [6].

The aim of this chapter is to provide the reader with accessible tailoring of basic deep reinforcement learning and to support research experts. The primary contribution made by this work is

Originated with a complete review study of comprehensive deep reinforcement learning concept and framework.

Provided detailed applications and challenges in deep reinforcement learning.

This chapter is clearly distinguished by the points mentioned above from other recent surveys. This gives the data as comprehensive as previous works. The chapter is organized as follows: Section 1.2 summarizes the complete description of reinforcement learning. The different applications and problems are explored in Section 1.3, accompanied by a conclusion in Section 1.4.

1.2 Comprehensive Study

1.2.1 Introduction

In most Artificial Intelligence (AI) subjects, we build mathematical structures to tackle problems. For RL, the Markov Decision Process (MDP) is the solution. It sounds complicated, but it provides a basic structure to model a complex problem. The world is observed and behavior performed by an individual (e.g. a human). Rewards are released, but they may be rare and delayed. The long-delayed incentives very often make it incredibly difficult to untangle the data and track what series of acts led to the rewards [11].

Markov decision process (MDP) Figure 1.2 is composed of:

State in MDP can be represented as raw images or we use sensors for robotic controls to calculate the joint angles, velocity, and pose of the end effector.

A movement in a chess game or pushing a robotic arm or a joystick may be an event.

The reward is very scarce for a GO match: 1 if we win or −1 if we lose. We get incentives more often. We score whenever we hit the sharks in the Atari Seaquest game (

Figure 1.3

).

If it is less than one the discount factor discounts potential incentives. In the future, money raised also has a smaller current value, and we will need it to further converge the solution for a strictly technical reason.

We can indefinitely rollout behaviour or limit the experience to N steps in time. This is called the horizon.

Figure 1.2 Markov process.

Figure 1.3 Raw images of State.

System dynamics is the transformation function. After taking action, it predicts the next condition. When we address model-based RL later, it is called the model that plays a significant role. RL’s ideas come from many areas of study, including the theory of power. In a particular setting, distinct notations can be used. It is possible to write the state as s or x, and the behavior as an or u. An action is the same as a control operation. We may increase the benefits or and the costs that are actually negative for each other [10].

1.2.2 Framework

Compared to other fields such as Deep Learning, where well-established frameworks such as Tensor Flow, PyTorch, or MXnet simplify the lives of DL practitioners, the practical implementations of Reinforcement Learning are relatively young. The advent of RL frameworks, however, has already started and we can select from many projects right now that greatly encourage the use of specialized RL techniques. Frameworks such as Tensor Flow or PyTorch have appeared in recent years to help transform pattern recognition into a product, making deep learning easier for practitioners to try and use [17].

In the Reinforcement Learning arena, a similar pattern is starting to play out. We are starting to see the resurgence of many open source libraries and tools to deal with this, both by helping to create new pieces (not by writing from scratch) and above all, by combining different algorithmic components of prebuild. As a consequence, by generating high abstractions of the core components of an RL algorithm, these Reinforcement Learning frameworks support engineers [7].

A significant number of simulations include Deep Reinforcement Learning algorithms, introducing another multiplicative dimension to the time load of Deep Learning itself. This is mainly needed by the architectures we have not yet seen in this sequence, such as, among others, the distributed actor-critic methods or behaviors of multi-agents. But even choosing the best model also involves tuning hyper parameters and searching between different settings of hyper parameters; it can be expensive. All this includes the need for supercomputers based on distributed systems of heterogeneous servers (with multi-core CPUs and hardware accelerators such as GPUs or TPUs) to provide high computing power [18].

1.2.3 Choice of the Learning Algorithm and Function Approximator Selection

In deep learning, the function approximator characterizes how the characteristics are handled to higher levels of abstraction (a fortiori can therefore give certain characteristics more or less weight). In the first levels of a deep neural network, for example, if there is an attention system, the mapping made up of those first layers can be used as a framework for selecting features. On the other hand, an asymptotic bias can occur if the function approximator used for the weighted sum and/or the rule and/or template is too basic. But on the other hand, there would be a significant error due to the limited size of the data (over fitting) when the feature approximator has weak generalization.

An especially better decision of a model-based or model-free method identified as a leading function approximator choice may infer that the state’s y-coordinate is less essential than the x-coordinate, and generalize that to the rule. It is helpful to share a performant function approximator in either a model-free or a model-based approach depending on the mission. Therefore the option to focus more on one or the other method is also a key factor in improving generalization [13, 19].

One solution to eliminating non-informative characteristics is to compel the agent to acquire a set of symbolic rules tailored to the task and to think on a more extreme scale. This abstract level logic and increased generalization have the potential to activate cognitive high-level functions such as analogical reasoning and cognitive transition. For example, the feature area of environmental may integrate a relational learning system and thus extend the notion of contextual reinforcement learning.

1.2.3.1 Auxiliary Tasks

In the era of successful reinforcement learning, growing a deep reinforcement learning agent with allied tasks within a jointly learned representation would substantially increase sample academic success.

This is accomplished by causing genuine several pseudo-reward functions, such as immediate prediction of rewards (= 0), predicting pixel changes in the next measurement, or forecasting activation of some secret unit of the neural network of the agent.

The point is that learning similar tasks creates an inductive bias that causes a model to construct functions useful for the variety of tasks in the neural network. This formation of more essential characteristics, therefore, contributes to less over fitting. In deep RL, an abstract state can be constructed in such a way that it provides sufficient information to match the internal meaningful dynamics concurrently, as well as to estimate the estimated return of an optimal strategy. The CRAR agent shows how a lesser version of the task can be studied by explicitly observing both the design and prototype components via the description of the state, along with an estimated maximization penalty for entropy. In contrast, this approach would allow a model-free and model-based combination to be used directly, with preparation happening in a narrower conditional state space.

1.2.3.2 Modifying the Objective Function

In order to optimize the policy acquired by a deep RL algorithm, one can implement an objective function that diverts from the real victim. By doing so, a bias is typically added, although this can help with generalization in some situations. The main approaches to modify the objective function are

i) Reward shaping

For faster learning, incentive shaping is a heuristic to change the reward of the task to ease learning. Reward shaping incorporates prior practical experience by providing intermediate incentives for actions that lead to the desired outcome. This approach is also used in deep reinforcement training to strengthen the learning process in environments with sparse and delayed rewards.

ii) Tuning the discount factor

When the model available to the agent is predicted from data, the policy discovered using a short iterative horizon will probably be better than a policy discovered with the true horizon. On the one hand, since the objective function is revised, artificially decreasing the planning horizon contributes to a bias. If a long planning horizon is focused, there is a greater chance of over fitting (the discount factor is close to 1). This over fitting can be conceptually interpreted as related to the aggregation of errors in the transformations and rewards derived from data in relation to the real transformation and reward chances [4].

1.3 Deep Reinforcement Learning: Value-Based and Policy-Based Learning

1.3.1 Value-Based Method

Algorithms such as Deep-Q-Network (DQN) use Convolutional Neural Networks (CNNs) to help the agent select the best action [9]. While these formulas are very complicated, these are usually the fundamental steps (Figure 1.4):

Figure 1.4 Value based learning.

Take the status picture, transform it to grayscale, and excessive parts are cropped.

Run the picture through a series of contortions and pooling in order to extract the important features that will help the agent make the decision.

Calculate each possible action’s Q-Value.

To find the most accurate Q-Values, conduct back-propagation.

1.3.2 Policy-Based Method

In the modern world, the number of potential acts may be very high or unknown. For instance, a robot learning to move on open fields may have millions of potential actions within the space of a minute. In these conditions, estimating Q-values for each action is not practicable. Policy-based approaches learn the policy specific function, without computing a cost function for each action. An illustration of a policy-based algorithm is given by Policy Gradient (Figure 1.5).

Policy Gradient, simplified, works as follows:

Requires a condition and gets the probability of some action based on prior experience

Chooses the most possible action

Reiterates before the end of the game and evaluates the total incentives

Using back propagation to change connection weights based on the incentives.

Figure 1.5 Policy based learning.

1.4 Applications and Challenges of Applying Reinforcement Learning to Real-World

1.4.1 Applications

The ability to tackle a wide range of Deep RL techniques has been demonstrated to a variety of issues which were previously unsolved. A few of the most renowned accomplishments are in the game of backgammon, beating previous computer programmes, achieving superhuman-level performance from the pixels in Atari games, mastering the game of Go and beating professional poker players in the Nolimit Texas Hold’em Heads Up Game: Libratus and Deep stack.

Such achievements in popular games are essential because in a variety of large and nuanced tasks that require operating with high-dimensional inputs, they explore the effectiveness of deep RL. Deep RL has also shown a great deal of potential for real-world applications such as robotics, self-driving vehicles, finance, intelligent grids, dialogue systems, etc. Deep RL systems are still in production environments, currently. How Facebook uses Deep RL, for instance, can be found for pushing notifications and for faster video loading with smart prefetching.

RL is also relevant to fields where one might assume that supervised learning alone, such as sequence prediction, is adequate. It has also been cast as an RL problem to build the right neural architecture for supervised learning tasks. Notice that evolutionary techniques can also be addressed for certain types of tasks. Finally, it should be remembered that deep RL has prospects in the areas of computer science in classical and basic algorithmic issues, such as the travelling salesman problem. This is an NP-complete issue and the ability to solve it with deep RL illustrates the potential effect it could have on many other NP-complete issues, given that it is possible to manipulate the structure of these problems [2, 12].

1.4.2 Challenges

Off-Line Learning

Training is also not possible directly online, but learning happens offline, using records from a previous iteration of the management system. Broadly speaking, we would like it to be the case that the new system version works better than the old one and that implies that we will need to perform off-policy assessment (predicting performance before running it on the actual system). There are a couple of approaches, including large sampling, for doing this. The introduction of the first RL version (the initial policy) is one special case to consider; there is also a minimum output requirement to be met before this is supposed to occur. The warm-start efficiency is therefore another important ability to be able to assess.

Learning From Limited Samples

There are no different training and assessment environments for many actual systems. All training knowledge comes from the real system, and during training, the agent does not have a separate exploration policy as its exploratory acts do not come for free. Given this greater exploration expense, and the fact that very little of the state space is likely to be explored by logs for learning from, policy learning needs to be data-efficient. Control frequencies may be 1 h or even multi-month time steps (opportunities to take action) and even longer incentive horizons. One easy way to measure a model’s data efficiency is to look at the amount of data needed to meet a certain output threshold.

High-Dimensional State and Action Spaces

For several realistic real-world problems, there are wide and consistent state and action spaces, which can pose serious problems for traditional RL algorithms. One technique is to generate a vector of candidate action and then do a closest neighbor search to determine the nearest accessible real action.

Safety Constraints

Many control systems must function under security restrictions, even during phases of exploratory learning. Constrained MDPs (Markov Decision Processes) make it possible to define constraints on states and behavior. Budgeted MDPs enable the degree of constraint/performance trade-off to be explored rather than simply hard-wired by letting constraint levels be learned. Another solution is to add to the network a protection layer that prevents any breaches of safety.

Partial Observability

It is partly measurable for almost all real systems where we would like to incorporate reinforcement learning. For example, the efficiency of mechanical parts may deteriorate over time, ‘identical’ widgets may exhibit performance variations provided the same control inputs, or it may simply be unknown the condition of certain parts of the system (e.g. the mental state of users of a suggested system).

Two common strategies to dealing with partial observability, including input history, and modelling history using repeated networks in the model. In addition, Robust MDP formalisms provide clear mechanisms to ensure that sensor and action noise and delays are robust to agents. If a given deployment setting may have initially unknown but learnable noise sources, then techniques for device detection may be used to train a policy that can learn in which environment it operates.

Reward Functions

Device or product owners do not have a good image of what they want to refine in certain instances. The incentive function is always multidimensional and involves different sub-goals to be balanced. Another great insight here which reminds me of machine latency discussions) is that ‘normal performance’ (i.e. expectation) is always an inadequate measure, and for all task instances, the system needs to perform well. A common approach is to use a Conditional Value at Risk (CVaR) target to measure the full distribution of rewards across classes, which looks at a given percentile of the distribution of rewards rather than the predicted reward.

Explainability/Interpretability

Real systems are owned and controlled by humans who need to be informed about the actions of the controller and need insights into cases of failure. For this purpose, for real-world policies, policy clarity is critical. In order to obtain stakeholder buy-in, it is necessary to consider the longer-term purpose of the policy, particularly in cases where the policy can find another solution and unforeseen approach to managing a system.

Real-Time Inference

Policy inference has to occur within the system’s control frequency. This could be in the order of milliseconds or shorter. This prevents us from using costly computational methods that do not follow the constraints (for example, certain types of model-based planning). Of course, systems with longer control intervals cause the opposite problem: in order to speed up data generation, we cannot run the task faster than in real time.

Delayed Rewards

Most real systems have interruptions in the state’s sensation, the actuators, or the feedback on the reward. For instance, delays in the effects of a braking system, or delays between a recommendation system’s choices and consequent user behaviors. There are a number of possible methods to deal with this, including memory-based agents that leverage a memory recovery system to allocate credit to distant past events that are helpful in forecasting [1, 15].

1.5 Conclusion

Deep Reinforcement Learning is the fusion of reinforcement learning (RL) and deep learning. This field of research has been able to solve a wide range of dynamic decision-making operations that were traditionally out of control for a computer. In applications such as medical, automation, smart grids, banking, and plenty more, deep RL thus brings up many new applications. We give an overview of the deep reinforcement learning (RL) paradigm and learning algorithm choices. We begin with deep learning and reinforcement learning histories, as well as the implementation of the Markov method. Next, we summarize some popular applications in various fields and, eventually, we end up addressing some possible challenges in the future growth of DRL.

References

1. Arulkumaran, K., Deisenroth, M., Brundage, M., Bharath, A., A Brief Survey of Deep Reinforcement Learning. IEEE Signal Process. Mag., 34, 1–16, 2017, 10.1109/MSP .2017.2743240.

2. Botvinick, M., Wang, J., Dabney, W., Miller, K., Kurth-Nelson, Z., Deep Reinforcement Learning and its Neuroscientific Implications, Neuron, 107, 603–616. 2020.

3. Duryea, E., Ganger, M., Hu, W., Exploring Deep Reinforcement Learning with Multi Q-Learning. Intell. Control Autom., 07, 129–144, 2016, 10.4236/ica.2016.74012.

4. Fenjiro, Y. and Benbrahim, H., Deep Reinforcement Learning Overview of the state of the Art. J. Autom. Mob. Robot. Intell. Syst., 12, 20–39, 2018, 10.14313/JAMRIS_3-2018/15.

5. Francois, V., Henderson, P., Islam, R., Bellemare, M., Pineau, J., An Introduction to Deep Reinforcement Learning, Foundations and Trends in Machine Learning, Boston—Delft, 2018, 10.1561/2200000071.

6. Haj Ali, A., Ahmed, N., Willke, T., Gonzalez, J., Asanovic, K., Stoica, I., A View on Deep Reinforcement Learning in System Optimization, arXiv:1908.01275v3 Intel Labs, University of California, Berkeley, 2019.

7. Heidrich-Meisner, V., Lauer, M., Igel, C., Riedmiller, M., Reinforcement learning in a Nutshell. ESANN'2007 Proceedings - European Symposium on Artificial Neural Networks, Bruges (Belgium), 277–288, 2007.

8. Ivanov, S. and D’yakonov, A., Modern Deep Reinforcement Learning Algorithms, arXiv preprint arXiv:1906.10025, 1–56, 2019.

9. Le Pham, T., Layek, A., Vien, N., Chung, T.C., Deep reinforcement learning algorithms for steering an underactuated ship in: 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI 2017), 602–607, 2017, 10.1109/MFI.2017.8170388.

10. Li, M.-J., Li, A.-H., Huang, Y.-J., Chu, S.-I., Implementation of Deep Reinforcement Learning. ICISS 2019: Proceedings of the 2019 2nd International Conference on Information Science and Systems, pp. 232–236, 2019, 10.1145/3322645.3322693.

11. Liu, Q., Zhai, J.-W., Zhang, Z.-Z., Zhong, S., Zhou, Q., Zhang, P., Xu, J., A Survey on Deep Reinforcement Learning. Jisuanji Xuebao/Chin. J. Comput., 41, 1–27, 2018, 10.11897/SP.J.1016 .2018. 00001.

12. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A., Veness, J., Bellemare, M., Graves, A., Riedmiller, M., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D., Human-level control through deep reinforcement learning. Nature, 518, 529–33, 2015, 10.1038/nature14236.

13. Mosavi, A., Ghamisi, P., Faghan, Y., Duan, P., Band, S., Comprehensive Review of Deep Reinforcement Learning Methods and Applications in Economics, Mathematics, 8, 1–42, 2020, 10.20944/preprints 202003.0309.v1.

14. Mousavi, S., Schukat, M., Howley, E., Deep Reinforcement Learning: An Overview, in: Lecture Notes in Networks and Systems, pp. 426–440, 2018, 10.1007/978-3-319-56991-8_32.

15. Nguyen, C., Dinh Thai, H., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., Kim, D.I., Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. IEEE Commun. Surv. Tutorials, 21, 4, 1–1, 2019, 10.1109/COMST.2019.2916583.

16. Sangeetha, S.K.B. and Ananthajothi, K., Machine Learning Tools for Digital Pathology—The Next Big Wave in Medical Science. Solid State Technol., 63, 3732–3749, 2020.

17. Santhya, R., Latha, S., Balamurugan, S., Charanyaa, S., Further investigations on strategies developed for efficient discovery of matching dependencies. Int. J. Innov. Res. Comput. Commun. Eng. (An ISO 3297:2007 Certified Organization), 3, 18998–19004, 2014.

18. Tan, F. and Yan, P., Deep Reinforcement Learning: From Q-Learning to Deep Q-Learning, Springer, Cham, Guangzhou, China, pp. 475–483, 2017, 10.1007/978-3-319-70093-9_50.

19. Xiliang, C., Cao, L., Li, C.-X., Xu, Z.-X., Lai, J., Ensemble Network Architecture for Deep Reinforcement Learning. Math. Probl. Eng., 2018, 1–6, 2018, 10.1155/2018/2129393.

20. Li, Y., Deep Reinforcement Learning: An Overview, arXiv:1701.07274v6, 2017.

*

Corresponding author

:

[email protected]

2Impact of AI in 5G Wireless Technologies and Communication Systems

A. Sivasundari* and K. Ananthajothi†

Department of Computer Science and Engineering, Misrimal Navajee Munoth Jain Engineering College, Chennai, India

Abstract

4G networks (with Internet Protocol or IP, telecommunications and reaction-based connectivity) have managed the network architecture. They have evolved and are now accessible in a multitude of ways, including advanced learning and deep learning. 5G is flexible and responsive and will establish the need for integrated real time decision-making. As the rollout has begun across the globe, recent technical and architectural developments in 5G networks have proved their value. In various fields of classification, recognition and automation, AI has already proved its efficacy with greater precision. The integration of artificial intelligence with internet-connected computers and superfast 5G wireless networks opens up possibilities around the globe and even in outer space. In this section, we offer an in-depth overview of the Artificial Intelligence implementation of 5G wireless communication systems. The focus of this research is in this context, to examine the application of AI and 5G in warehouse building and to discuss the role and difficulties faced, and to highlight suggestions for future studies on integrating Advanced AI in 5G wireless communications.

Keywords: Artificial intelligence, 5G, deep learning, machine learning, mobile networks, wireless communication

2.1 Introduction

Although 5G provides low latency and very high speed support capabilities (e.g., eMBB), a wide number of devices (e.g., mMTC), a heterogeneous mix of traffic types from a diverse and challenging suite of applications (e.g., URLLC), AI is complemented by observing from specific environments to provide independent reach of operation, turning 5G into a data-driven adaptive real-time network [13]. AI is used for 5G system modeling, automation of core network (e.g. provisioning, scheduling, prediction of faults, protection, fraud detection), distributed computing, reduction of operating costs, and improvement of both service quality and customer evolves on chatbots, recommendation systems, and strategies such as automated processes. In addition, AI is used across all layers, from the disaggregated radio access layer (5G RAN) to the distributed cloud layer (5G Edge/Core) to the integrated access backhaul to fine tune performance [5].

AI is used for the 5G distributed cloud layer to optimize device resource usage, autoscaling, identification of anomalies, predictive analytics, prescriptive policies, and so on. In addition, the 5G distributed cloud layer offers acceleration technologies to enable federated and distributed learning for AI workloads [19].

Figure 2.1 Growth of 5G Connections worldwide.

The growth of mobile 5G worldwide and its market overview are shown in Figure 2.1 and Figure 2.2. To support a multitude of emerging