Deep Learning and its Applications using Python E-Book

• Herausgeber: John Wiley & Sons
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 276

Veröffentlichungsjahr: 2023

Beschreibung

This book thoroughly explains deep learning models and how to use Python programming to implement them in applications such as NLP, face detection, face recognition, face analysis, and virtual assistance (chatbot, machine translation, etc.). It provides hands-on guidance in using Python for implementing deep learning application models. It also identifies future research directions for deep learning.

Leseprobe

Table of Contents

Cover

Table of Contents

Series Page

Title Page

Copyright Page

Preface

1 Introduction to Deep Learning

1.1 History of Deep Learning

1.2 A Probabilistic Theory of Deep Learning

1.3 Back Propagation and Regularization

1.4 Batch Normalization and VC Dimension

1.5 Neural Nets—Deep and Shallow Networks

1.6 Supervised and Semi-Supervised Learning

1.7 Deep Learning and Reinforcement Learning

References

2 Basics of TensorFlow

2.1 Tensors

2.2 Computational Graph and Session

2.3 Constants, Placeholders, and Variables

2.4 Creating Tensor

2.5 Working on Matrices

2.6 Activation Functions

2.7 Loss Functions

2.8 Common Loss Function

2.9 Optimizers

2.10 Metrics

References

3 Understanding and Working with Keras

3.1 Major Steps to Deep Learning Models

3.2 Load Data

3.3 Pre-Process Data

3.4 Define the Model

3.5 Compile the Model

3.6 Fit and Evaluate the Mode

3.7 Prediction

3.8 Save and Reload the Model

3.9 Additional Steps to Improve Keras Models

3.10 Keras with TensorFlow

References

4 Multilayer Perceptron

4.1 Artificial Neural Network

4.2 Single-Layer Perceptron

4.3 Multilayer Perceptron

4.4 Logistic Regression Model

4.5 Regression to MLP in TensorFlow

4.6 TensorFlow Steps to Build Models

4.7 Linear Regression in TensorFlow

4.8 Logistic Regression Mode in TensorFlow

4.9 Multilayer Perceptron in TensorFlow

4.10 Regression to MLP in Keras

4.11 Log-Linear Model

4.12 Keras Neural Network for Linear Regression

4.13 Keras Neural Network for Logistic Regression

4.14 MLPs on the Iris Data

4.15 MLPs on MNIST Data (Digit Classification)

4.16 MLPs on Randomly Generated Data

References

5 Convolutional Neural Networks in Tensorflow

5.1 CNN Architectures

5.2 Properties of CNN Representations

5.3 Convolution Layers, Pooling Layers – Strides - Padding and Fully Connected Layer

5.4 Why TensorFlow for CNN Models?

5.5 TensorFlow Code for Building an Image Classifier for MNIST Data

5.6 Using a High-Level API for Building CNN Models

5.7 CNN in Keras

5.8 Building an Image Classifier for MNIST Data in Keras

5.9 Building an Image Classifier with CIFAR-10 Data

5.10 Define the Model Architecture

5.11 Pre-Trained Models

References

6 RNN and LSTM

6.1 Concept of RNN

6.2 Concept of LSTM

6.3 Modes of LSTM

6.4 Sequence Prediction

6.5 Time-Series Forecasting with the LSTM Model

6.6 Speech to Text

6.7 Examples Using Each API

6.8 Text-to-Speech Conversion

6.9 Cognitive Service Providers

6.10 The Future of Speech Analytics

References

7 Developing Chatbot’s Face Detection and Recognition

7.1 Why Chatbots?

7.2 Designs and Functions of Chatbot’s

7.3 Steps for Building a Chatbot’s

7.4 Best Practices of Chatbot Development

7.5 Face Detection

7.6 Face Recognition

7.7 Face Analysis

7.8 OpenCV—Detecting a Face, Recognition and Face Analysis

7.9 Deep Learning–Based Face Recognition

7.10 Transfer Learning

7.11 API’s

References

8 Advanced Deep Learning

8.1 Deep Convolutional Neural Networks (AlexNet)

8.2 Networks Using Blocks (VGG)

8.3 Network in Network (NiN)

8.4 Networks with Parallel Concatenations (GoogLeNet)

8.5 Residual Networks (ResNet)

8.6 Densely Connected Networks (DenseNet)

8.7 Gated Recurrent Units (GRU)

8.8 Long Short-Term Memory (LSTM)

8.9 Deep Recurrent Neural Networks (D-RNN)

8.10 Bidirectional Recurrent Neural Networks (Bi-RNN)

8.11 Machine Translation and the Dataset

8.12 Sequence to Sequence Learning

References

9 Enhanced Convolutional Neural Network

9.1 Introduction

9.2 Deep Learning-Based Architecture for Absence Seizure Detection

9.3 EEG Signal Pre-Processing Strategy and Channel Selection

9.4 Input Formulation and Augmentation of EEG Signal for Deep Learning Model

9.5 Deep Learning Based Feature Extraction and Classification

9.6 Performance Analysis

9.7 Summary

References

10 Conclusion

10.1 Introduction

10.2 Future Research Direction and Prospects

10.3 Research Challenges in Deep Learning

10.4 Practical Deep Learning Case Studies

10.5 Summary

References

Index

End User License Agreement

List of Tables

Chapter 1

Table 1.1 Key discoveries of deep learning.

Chapter 2

Table 2.1 Noticeable TensorFlow evolution.

Chapter 3

Table 3.1 Notable Keras evolution.

Chapter 4

Table 4.1 Training example.

Table 4.2 Testing example.

Chapter 8

Table 8.1 Representation of AlexNet layers.

Table 8.2 Representations of VGG layers.

Table 8.3 Representations of NiN layers.

Table 8.4 Representation of GoogLeNet layers.

Table 8.5 Representation of ResNet layers.

Table 8.6 Representation of dense layers.

Table 8.7 Representation of ResNet layers.

Table 8.8 Representation of ResNet layers.

Chapter 9

Table 9.1 Measurements of bio-signal.

Table 9.2 EEG wave forms and its features.

Table 9.3 Information of absence and normal subject.

Table 9.4 Power measure of a single absence seizure EEG segment.

Table 9.5 Power measure of absence seizure subject and normal subject.

Table 9.6 Performance of absence seizure detection.

Table 9.7 Details of normal, abnormal and absence subject for augmentation.

Table 9.8 Schematics of convolution layer.

Chapter 10

Table 10.1a Human activity categorization findings on the MRSDaily database: p...

Table 10.1b Classification of human activity results from the CD-60 database, ...

Table 10.1c Comparison in brief classification of human activities in the MRSD...

Table 10.2 Comparison of results — Seizure prediction.

Table 10.3 Comparison between efficiency and effectiveness computation time — ...

Table 10.4 Comparison of performance (RMS error) – 3D from 2D image.

Table 10.5a Computation time — Scene classification.

Table 10.5b Comparison of performance - Scene classification.

Table 10.6 Summary of results—Fingerprint recognition.

Table 10.7a Lists the outcomes.

Table 10.7b Comparison of results - Fingerprint recognition.

Table 10.8 Performance comparison - Face recognition.

Table 10.9 Comparing training performance—Butterfly classification.

Table 10.10 Performance comparison: Leaf classification (CNN doesn’t pre-proce...

Table 10.11 Comparison of results — Traffic sign recognition.

Table 10.12 Comparison of results - Programming language classification.

Table 10.13a Types of Chicago crime data.

Table 10.13b Preprocessing steps.

Table 10.13c Comparison of performances — Natural language text classification...

Table 10.14 Performance comparisons — Speech recognition.

Table 10.15a Comparison of results — Music genre classification.

Table 10.15b Comparison of results — Music genre classification.

Table 10.16 Comparison of results — Credit card fraud detection.

Table 10.17 Comparison of results — Drill site prediction.

Table 10.18 Comparison of performances — Market microstructure.

Table 10.19 Comparison of performance — Fault detection.

List of Illustrations

Chapter 1

Figure 1.1 MNIST data set.

Figure 1.2 Structure of neural network.

Figure 1.3 Bell curve.

Figure 1.4 Metrics to calculate accuracy.

Figure 1.5 Representation of confusion matrix.

Figure 1.6 Output for the confusion matrix.

Figure 1.7 Code to demonstrate the working of precision, recall and F1 score....

Figure 1.8 The graphical representation of TPR and FPR.

Figure 1.9 Working of back propagation.

Figure 1.10 Transformation of handwritten digit data.

Figure 1.11 Presentation of intelligent learning algorithms in computer scienc...

Chapter 2

Figure 2.1 The program structure of two-phase TensorFlow with description.

Figure 2.2 Representation of image as tensor.

Figure 2.3 An activation function.

Figure 2.4 Tangent activation function.

Figure 2.5 Sigmoid activation function.

Chapter 3

Figure 3.1 Key features of Keras.

Figure 3.2 Four steps in deep learning model.

Figure 3.3 Steps involved in deep learning to process data using Keras.

Figure 3.4 Defining the model with input and output.

Figure 3.5 List of loss functions.

Figure 3.6 List of optimizers.

Figure 3.7 List of evaluation metrics.

Figure 3.8 Summary of model.

Chapter 4

Figure 4.1 Structure of human neuron.

Figure 4.2 Artificial Neural Network with input, hidden and output layers.

Figure 4.3 ANN with transfer function and activation function (Single layer AN...

Figure 4.4 Logistic regression model with single input (single input vector)....

Figure 4.5 Logistic regression model with multiple input (linear model).

Figure 4.6 Multilayer perceptron (fully connected multilayer ANN).

Figure 4.7 Structure of logistic regression.

Figure 4.8 Code to demonstrate output of linear regression model (weight, bias...

Figure 4.9 Actual and predicted values of linear model.

Figure 4.10 Cross entropy loss of logistic regression model.

Figure 4.11 Training and testing loss of multilayer perception.

Chapter 5

Figure 5.1 Layers of convolutional neural network.

Figure 5.2 Subsampling of input with filter to get output.

Figure 5.3 Max-pooling of CNN.

Figure 5.4 Classification using CNN.

Chapter 6

Figure 6.1 Working of simple RNN classifier.

Figure 6.2 Working of LSTM classifier.

Figure 6.3 Working of GRU classifier.

Figure 6.4 Different modes of LSTM.

Chapter 8

Figure 8.1 Structure of AlexNet.

Figure 8.2 Output of AlexNet model.

Figure 8.3 Structure of VGG model.

Figure 8.4 Output of VGG model.

Figure 8.5 Structure of NiN model.

Figure 8.6 Output of VGG model.

Figure 8.7 Structure of GoogLeNet.

Figure 8.8 Output of GoogLeNet.

Figure 8.9 Structure of ResNet.

Figure 8.10 Output of ResNet.

Figure 8.11 Structure of DenseNet.

Figure 8.12 Output of DenseNet.

Figure 8.13 Structure of GRU.

Figure 8.14 Output of GRU.

Figure 8.15 Structure of LSTM.

Figure 8.16 Output of LSTM.

Figure 8.17 Output of D-RNN.

Figure 8.18 Output of bidirectional RNN.

Chapter 9

Figure 9.1 General methodology for bio-signal processing.

Figure 9.2 Structure of human neuron.

Figure 9.3 Human brain structure with label.

Figure 9.4 Raw EEG signal (x axis represents time and y axis represents freque...

Figure 9.5 Five sub-bands in time domain (x axis represents sample points (tim...

Figure 9.6 Five sub-bands of EEG in frequency domain (x axis represents sample...

Figure 9.7 Placement of electrode of human scalp.

Figure 9.8 Absence seizure EEG signal (x axis represent time and y axis repres...

Figure 9.9 EEG signal of normal, abnormal and absence in time domain.

Figure 9.10 Deep learning based absence seizure detection architecture.

Figure 9.11 Deep learning based absence seizure detection work flow.

Figure 9.12 Working of first module for pre-processing and channel selection....

Figure 9.13 Absence seizure EEG input of FP1-F7 channel.

Figure 9.14 Decomposition of EEG signal into five sub-bands.

Figure 9.15 Absence seizure EEG segment of FP1-F7 channel.

Figure 9.16 Five sub-bands of EEG segment in time domain. Here the y-axis of e...

Figure 9.17 Five sub-bands of EEG segment in frequency domain. Here the y-axis...

Figure 9.18

A

bsence seizure subject’s four channels power measures of sub-band...

Figure 9.19 Classification output of normal and absence subject.

Figure 9.20 This shows the augmentation of a single channel instance.

Figure 9.21 Single instance input before augmentation.

Figure 9.22 This shows the augmented initial eight segments of single instance...

Figure 9.23 Maximum value of the signal of subjects in various stages. (Ab—abs...

Figure 9.24 Standard deviation of the signal of subjects in various stages. (A...

Figure 9.25 Box plot of mean function for three different classes (1—absence, ...

Figure 9.26 Box plot of standard deviation function for three different classe...

Figure 9.27 Box plot of skewness function for three different classes (1—absen...

Figure 9.28 Two convolution layers with input and output layers.

Figure 9.29 This shows the process of feature extraction along with parameters...

Figure 9.30 Shows convolution layer output of absence seizure pattern in time ...

Figure 9.31 Structure of GRU.

Figure 9.32 Working of GRU-SVM.

Figure 9.33 Performance of the classifiers.

Chapter 10

Figure 10.1 Skeleton body joints given by Kincet.

Figure 10.2 (a) Fingerprint ridge patterns. (b) The fingerprint details the th...

Guide

Cover Page

Series Page

Title Page

Copyright Page

Preface

Table of Contents

Begin Reading

Index

Wiley End User License Agreement

Pages

ii

iii

iv

ix

x

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

Scrivener Publishing100 Cummings Center, Suite 541JBeverly, MA 01915-6106

Publishers at ScrivenerMartin Scrivener ([email protected])Phillip Carmical ([email protected])

Deep Learning and Its Applications Using Python

Niha Kamal Basha

Department of Information Security, School of Computer Science and Engineering, Vellore Institute of Science and Technology, India

Surbhi Bhatia Khan

Department of Data Science, School of Science, Engineering and Environment, University of Salford, Manchester, United Kingdom

Abhishek Kumar

Department of Computer Science & Engineering, Chandigarh University, Punjab, India

and

Arwa Mashat

Faculty of Computing and Information Technology, King Abdulaziz University, Rabigh, Saudi Arabia

This edition first published 2023 by John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA and Scrivener Publishing LLC, 100 Cummings Center, Suite 541J, Beverly, MA 01915, USA© 2023 Scrivener Publishing LLCFor more information about Scrivener publications please visit www.scrivenerpublishing.com.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.

Wiley Global Headquarters111 River Street, Hoboken, NJ 07030, USA

For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.

Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials, or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read.

Library of Congress Cataloging-in-Publication Data

ISBN 978-1-394-16646-6

Cover image: Pixabay.ComCover design by Russell Richardson

Preface

Research into deep learning has come a long way across multiple domains, such as healthcare, marketing, banking, manufacturing, education, and so on. Notable applications within these domains are trending, like visual recognition, fraud detection, virtual assistance, NLP, etc. Deep learning models are used to implement these applications. Those models include Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Long Short-Term Memory (LSTM), and others. To deploy these application-based deep learning models, Python programing is ideal, with its open source software libraries like keras, tensorflow, and soon.

This book thoroughly explains deep learning models and how to use Python programming to implement them in applications such as NLP, face detection, face recognition, face analysis, and virtual assistance (chatbot, machine translation, etc.). This book provides hands-on guidance to using Python for implementing deep learning application models. It also identifies future research directions for deep learning.

Chapter 1 deals with the history of deep learning. The origin tracks back to 1943, when Walter Pitts and Warren McCulloch computed a model (computer model) that supported the human brain (the neural networks). To mimic the thought processes of humans, Pitts and McCulloch used a collection of algorithms and arithmetic concepts, called threshold logic. Since then, deep learning has continuously evolved.

TensorFlow and Keras play a major role in the implementation of deep learning models. Therefore, Chapter 2 explains TensorFlow fundamentals, based on a deep learning framework. TensorFlow plays a major role in pattern recognition, specifically in regards to language, images, sound, and time-series data. Classification, prediction, clustering, and feature extraction occurs, too, with the help of deep learning.

Chapter 3 explains how a Python library called Keras has been used for deep learning. The chapter outlines the objective and focus of Keras to create neural networks layers and mathematical models. Chapter 4 continues the presentation of intelligent learning algorithms, explaining how multilayer perceptron is one of the supervised learning algorithms and is an example of artificial neural network with a feedback loop. Among other deep learning models, Chapter 5 delves into CNN algorithms, which act as an artificial neuron (neural network). This neural network is widely used to deal with image input for image processing, recognition, and classification.

A Chapter 6 deals with RNN and LSTM, which are deep learning algorithms that can be used for sequencing numerical inputs to enable tasks, such as handwritten recognition without segmentation or speech recognition. In Chapter 7, the importance of artificial intelligence and its vital role in communicating with humans and machines, as shown in the form of text or voice. Programs that can converse naturally with humans are called chatbots, and this chapter explores have chatbots can be automated to perform user-performed tasks.

Chapter 8 discusses different advanced models of deep learning algorithms that are used frequently. Upgrades or modifications are possible because of the flexible nature of neural network, which can lead to the design of an end-to-end model. Therefore, this advancement allows researchers to build simple to complex structures that align need with imagination. Some advanced deep learning algorithms include AlexNet, VGG, NiN, GoogLeNet, ResNet, DenseNet, GRU, LSTM, D-RNN, and Bi-RNN. These advanced deep learning algorithms are discussed in detail, along with code for implementation using Python’s Keras library on the TensorFlow platform.

Chapter 9 includes a detailed discussion on new trends in deep learning, current and future challenges, and its prospects. The chapter is supported by a practical case study and working proofs. Some notable works highlighted in this chapter come from different domains, such as semantic intelligence, quantum AI, cyber security, the LAMSTAR system, epilepsy seizure onset prediction, etc.

We hope this book will expand your knowledge on this expanding subject. Our thanks go to the prestigious Wiley and Scrivener Publishing for their continuous kind support and guidance.

Editors

Niha Kamal Basha

Surbhi Bhatia Khan

Abhishek Kumar

Arwa Mashat