The Deep Learning with Keras Workshop E-Book

Matthew Moocarme

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 450

Veröffentlichungsjahr: 2020

Beschreibung

Cut through the noise and get real results with a step-by-step approach to understanding deep learning with Keras programming

Key Features

• Ideal for those getting started with Keras for the first time
• A step-by-step Keras tutorial with exercises and activities that help build key skills
• Structured to let you progress at your own pace, on your own terms
• Use your physical print copy to redeem free access to the online interactive edition

Book Description

You already know that you want to learn Keras, and a smarter way to learn is to learn by doing. The Deep Learning with Keras Workshop focuses on building up your practical skills so that you can develop artificial intelligence applications or build machine learning models with Keras. You'll learn from real examples that lead to real results.

Throughout The Deep Learning with Keras Workshop, you'll take an engaging step-by-step approach to understand Keras. You won't have to sit through any unnecessary theory. If you're short on time you can jump into a single exercise each day or spend an entire weekend tinkering with your own neural networks. It's your choice. Learning on your terms, you'll build up and reinforce key skills in a way that feels rewarding.

Every physical print copy of The Deep Learning with Keras Workshop unlocks access to the interactive edition. With videos detailing all exercises and activities, you'll always have a guided solution. You can also benchmark yourself against assessments, track progress, and receive content updates. You'll even earn a secure credential that you can share and verify online upon completion. It's a premium learning experience that's included with your printed copy. To redeem, follow the instructions located at the start of your book.

Fast-paced and direct, The Deep Learning with Keras Workshop is the ideal companion for those who are just getting started with Keras. You'll build and iterate on your code like a software developer, learning along the way. This process means that you'll find that your new skills stick, embedded as best practice. A solid foundation for the years ahead.

What you will learn

• Gain insight into the fundamental concepts of neural networks
• Learn to think like a data scientist and understand the difference between machine learning and deep learning
• Discover various techniques to evaluate, tweak, and improve your models
• Explore different techniques to manipulate your data
• Explore alternative techniques to verify the accuracy of your model

Who this book is for

Our goal at Packt is to help you be successful, in whatever it is that you choose to do. The Deep Learning with Keras Workshop is an ideal tutorial for the programmer who is getting started with Keras and deep learning. Pick up a Workshop today and let Packt help you develop skills that stick with you for life.

Matthew Moocarme is a director and senior data scientist in Viacom’s advertising science team. As a data scientist at Viacom, he designs data-driven solutions to help Viacom gain insights, streamline workflows, and solve complex problems using data science and machine learning. Matthew lives in New York City and outside of work enjoys combining deep learning with music theory. He is a classically-trained physicist, holding a Ph.D. in physics from The Graduate Center of CUNY and is an active artificial intelligence developer, researcher, practitioner, and educator. Mahla Abdolahnejad is a Ph.D. candidate in systems and computer engineering with Carleton University, Canada. She also holds a bachelor's degree and a master's degree in biomedical engineering, which first exposed her to the field of artificial intelligence and artificial neural networks, in particular. Her Ph.D. research is focused on deep unsupervised learning for computer vision applications. She is particularly interested in exploring the differences between a human's way of learning from the visual world and a machine's way of learning from the visual world, and how to push machine learning algorithms toward learning and thinking like humans. Ritesh Bhagwat has a master's degree in applied mathematics with a specialization in computer science. He has over 14 years of experience in data-driven technologies and has led and been a part of complex projects ranging from data warehousing and business intelligence to machine learning and artificial intelligence. He has worked with top-tier global consulting firms as well as large multinational financial institutions. Currently, he works as a data scientist. Besides work, he enjoys playing and watching cricket and loves to travel. He is also deeply interested in Bayesian statistics.

Leseprobe

The Deep Learning with Keras Workshop

Second Edition

An Interactive Approach to Understanding Deep Learning with Keras

Matthew Moocarme

Mahla Abdolahnejad

Ritesh Bhagwat

The Deep Learning with Keras Workshop

Second Edition

Copyright © 2020 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Authors: Matthew Moocarme, Mahla Abdolahnejad, and Ritesh Bhagwat

Reviewers: Vikraman Karunanidhi, Asheesh Mehta, Bernard Ong, and Anuj Shah

Managing Editor: Bhavesh Bangera

Acquisitions Editors: Kunal Sawant, Archie Vankar, and Karan Wadekar

Production Editor: Samita Warang

Editorial Board: Shubhopriya Banerjee, Bharat Botle, Ewan Buckingham, Megan Carlisle, Mahesh Dhyani, Manasa Kumar, Alex Mazonowicz, Bridget Neale, Dominic Pereira, Shiny Poojary, Abhishek Rane, Brendan Rodrigues, Mugdha Sawarkar, Erol Staveley, Ankita Thakur, Nitesh Thakur, and Jonathan Wray

First published: April 2019

Second edition: February 2020

Production reference: 1270220

ISBN 978-1-83921-757-9

Published by Packt Publishing Ltd.

Livery Place, 35 Livery Street

Birmingham B3 2PB, UK

Table of Contents

Preface   i

1. Introduction to Machine Learning with Keras   1

Introduction   2

Data Representation   4

Tables of Data   4

Loading Data   5

Exercise 1.01: Loading a Dataset from the UCI Machine Learning Repository   7

Data Preprocessing   10

Exercise 1.02: Cleaning the Data   12

Appropriate Representation of the Data   22

Exercise 1.03: Appropriate Representation of the Data   23

Life Cycle of Model Creation   26

Machine Learning Libraries   26

scikit-learn   27

Keras   29

Advantages of Keras 30

Disadvantages of Keras 30

More than Building Models   31

Model Training   32

Classifiers and Regression Models   32

Classification Tasks   34

Regression Tasks   35

Training Datasets and Test Datasets   35

Model Evaluation Metrics   36

Exercise 1.04: Creating a Simple Model   38

Model Tuning   42

Baseline Models   42

Exercise 1.05: Determining a Baseline Model   42

Regularization   44

Cross-Validation   45

Activity 1.01: Adding Regularization to the Model   46

Summary   48

2. Machine Learning versus Deep Learning   51

Introduction   52

Advantages of ANNs over Traditional Machine Learning Algorithms   54

Advantages of Traditional Machine Learning Algorithms over ANNs   56

Hierarchical Data Representation   56

Linear Transformations   58

Scalars, Vectors, Matrices, and Tensors   58

Tensor Addition   59

Exercise 2.01: Performing Various Operations with Vectors, Matrices, and Tensors   60

Reshaping   63

Matrix Transposition   64

Exercise 2.02: Matrix Reshaping and Transposition   65

Matrix Multiplication   68

Exercise 2.03: Matrix Multiplication   70

Exercise 2.04: Tensor Multiplication   71

Introduction to Keras   73

Layer Types   74

Activation Functions   76

Model Fitting   77

Activity 2.01: Creating a Logistic Regression Model Using Keras   79

Summary   80

3. Deep Learning with Keras   83

Introduction   84

Building Your First Neural Network   84

Logistic Regression to a Deep Neural Network   85

Activation Functions   88

Forward Propagation for Making Predictions   90

Loss Function   92

Backpropagation for Computing Derivatives of Loss Function   93

Gradient Descent for Learning Parameters   94

Exercise 3.01: Neural Network Implementation with Keras   97

Activity 3.01: Building a Single-Layer Neural Network for Performing Binary Classification   103

Model Evaluation   105

Evaluating a Trained Model with Keras   106

Splitting Data into Training and Test Sets   107

Underfitting and Overfitting   109

Early Stopping   114

Activity 3.02: Advanced Fibrosis Diagnosis with Neural Networks   115

Summary   118

4. Evaluating Your Model with Cross-Validation Using Keras Wrappers   121

Introduction   122

Cross-Validation   122

Drawbacks of Splitting a Dataset Only Once   123

K-Fold Cross-Validation   125

Leave-One-Out Cross-Validation   126

Comparing the K-Fold and LOO Methods   128

Cross-Validation for Deep Learning Models   129

Keras Wrapper with scikit-learn   130

Exercise 4.01: Building the Keras Wrapper with scikit-learn for a Regression Problem   131

Cross-Validation with scikit-learn   133

Cross-Validation Iterators in scikit-learn   134

Exercise 4.02: Evaluating Deep Neural Networks with Cross-Validation   136

Activity 4.01: Model Evaluation Using Cross-Validation for an Advanced Fibrosis Diagnosis Classifier   138

Model Selection with Cross-Validation   140

Cross-Validation for Model Evaluation versus Model Selection   140

Exercise 4.03: Writing User-Defined Functions to Implement Deep Learning Models with Cross-Validation   143

Activity 4.02: Model Selection Using Cross-Validation for the Advanced Fibrosis Diagnosis Classifier   148

Activity 4.03: Model Selection Using Cross-validation on a Traffic Volume Dataset   150

Summary   152

5. Improving Model Accuracy   155

Introduction   156

Regularization   157

The Need for Regularization   157

Reducing Overfitting with Regularization   160

L1 and L2 Regularization   162

L1 and L2 Regularization Formulation   162

L1 and L2 Regularization Implementation in Keras   163

Activity 5.01: Weight Regularization on an Avila Pattern Classifier   164

Dropout Regularization   167

Principles of Dropout Regularization   167

Reducing Overfitting with Dropout   168

Exercise 5.01: Dropout Implementation in Keras   169

Activity 5.02: Dropout Regularization on the Traffic Volume Dataset   173

Other Regularization Methods   175

Early Stopping   175

Exercise 5.02: Implementing Early Stopping in Keras   177

Data Augmentation   183

Adding Noise   183

Hyperparameter Tuning with scikit-learn   185

Grid Search with scikit-learn   185

Randomized Search with scikit-learn   188

Activity 5.03: Hyperparameter Tuning on the Avila Pattern Classifier   189

Summary   191

6. Model Evaluation   193

Introduction   194

Accuracy   194

Exercise 6.01: Calculating Null Accuracy on a Pacific Hurricanes Dataset   196

Advantages and Limitations of Accuracy   197

Imbalanced Datasets   198

Working with Imbalanced Datasets   199

Confusion Matrix   200

Metrics Computed from a Confusion Matrix   201

Exercise 6.02: Computing Accuracy and Null Accuracy with APS Failure for Scania Trucks Data   205

Activity 6.01: Computing the Accuracy and Null Accuracy of a Neural Network When We Change the Train/Test Split   211

Exercise 6.03: Deriving and Computing Metrics Based on a Confusion Matrix   212

Activity 6.02: Calculating the ROC Curve and AUC Score   221

Summary   222

7. Computer Vision with Convolutional Neural Networks   225

Introduction   226

Computer Vision   226

Convolutional Neural Networks   227

The Architecture of a CNN   228

Input Image   228

Convolution Layer   229

The Pooling Layer   231

Flattening   232

Image Augmentation   235

Advantages of Image Augmentation   236

Exercise 7.01: Building a CNN and Identifying Images of Cars and Flowers   237

Activity 7.01: Amending Our Model with Multiple Layers and the Use of softmax   240

Exercise 7.02: Amending Our Model by Reverting to the Sigmoid Activation Function   242

Exercise 7.03: Changing the Optimizer from Adam to SGD   245

Exercise 7.04: Classifying a New Image   248

Activity 7.02: Classifying a New Image   249

Summary   251

8. Transfer Learning and Pre-Trained Models   253

Introduction   254

Pre-Trained Sets and Transfer Learning   254

Feature Extraction   255

Fine-Tuning a Pre-Trained Network    257

The ImageNet Dataset   258

Some Pre-Trained Networks in Keras   258

Exercise 8.01: Identifying an Image Using the VGG16 Network   259

Activity 8.01: Using the VGG16 Network to Train a Deep Learning Network to Identify Images   262

Exercise 8.02: Classifying Images That Are Not Present in the ImageNet Database   263

Exercise 8.03: Fine-Tuning the VGG16 Model   268

Exercise 8.04: Image Classification with ResNet   273

Activity 8.02: Image Classification with ResNet    276

Summary   277

9. Sequential Modeling with Recurrent Neural Networks   279

Introduction   280

Sequential Memory and Sequential Modeling    280

Recurrent Neural Networks (RNNs)   281

The Vanishing Gradient Problem   287

A Brief Explanation of the Exploding Gradient Problem   288

Long Short-Term Memory (LSTM)   289

Exercise 9.01: Predicting the Trend of Alphabet's Stock Price Using an LSTM with 50 Units (Neurons)   292

Activity 9.01: Predicting the Trend of Amazon's Stock Price Using an LSTM with 50 Units (Neurons)   298

Exercise 9.02: Predicting the Trend of Alphabet's Stock Price Using an LSTM with 100 units   300

Activity 9.02: Predicting Amazon's Stock Price with Added Regularization   304

Activity 9.03: Predicting the Trend of Amazon's Stock Price Using an LSTM with an Increasing Number of LSTM Neurons (100 Units)   306

Summary   308

Appendix   311

Preface

About

This section briefly introduces this book and the software requirements you need in order to complete all of the included activities and exercises.

About the Book

You already know that you want to learn Keras, and a smarter way to learn is to learn by doing. The Deep Learning with Keras Workshop focuses on building up your practical skills so that you can develop artificial intelligence applications or build machine learning models with Keras. You'll learn from real examples that lead to real results.

Throughout The Deep Learning with Keras Workshop, you'll take an engaging step-by-step approach to understand Keras. You won't have to sit through any unnecessary theory. If you're short on time, you can jump into a single exercise each day or spend an entire weekend tinkering with your own neural neyworks. It's your choice. Learning on your terms, you'll build up and reinforce key skills in a way that feels rewarding.

Every physical print copy of The Deep Learning with Keras Workshop unlocks access to the interactive edition. With videos detailing all exercises and activities, you'll always have a guided solution. You can also benchmark yourself against assessments, track progress, and receive content updates. You'll even earn a secure credential that you can share and verify online upon completion. It's a premium learning experience that's included with your print copy. To redeem, follow the instructions located at the start of your data science book.

Fast-paced and direct, The Deep Learning with Keras Workshop is the ideal companion for those who are just getting started with Keras. You'll build and iterate on your code like a software developer, learning along the way. This process means that you'll find that your new skills stick, embedded as best practice—a solid foundation for the years ahead.

Audience

Our goal at Packt is to help you be successful, in whatever it is that you choose to do. The Deep Learning with Keras Workshop is an ideal tutorial for the programmer who is getting started with Keras and deep learning. Pick up a Workshop today and let Packt help you develop skills that will stick with you for life.

About the Chapters

Chapter 1, Introduction to Machine Learning with Keras, will introduce you to the fundamental concepts of machine learning by using the scikit-learn package. You will learn how to present data for model building, then train a logistic regression model using a real-world dataset.

Chapter 2, Machine Learning versus Deep Learning, will present the difference between traditional machine learning algorithms and deep learning algorithms. You will learn the linear transformations necessary for building neural networks and build your first neural network with the Keras library.

Chapter 3, Deep Learning with Keras, will extend your knowledge of neural network building. You will learn how to build multi-layer neural networks and recognize when your model is underfitting or overfitting to the training data.

Chapter 4,Evaluating Your Model with Cross-Validation Using Keras Wrappers, will teach you how to use Keras wrappers with scikit-learn to incorporate Keras models into a scikit-learn workflow. You will apply cross-validation to evaluate your models and use this technique to choose the optimal hyperparameters.

Chapter 5, Improving Model Accuracy, will introduce various regularization techniques to prevent your models from overfitting the training data. You will learn different methods to search for the optimal hyperparameters that result in the highest model accuracy.

Chapter 6, Model Evaluation, will demonstrate a variety of methods to evaluate your models. Beyond accuracy, you will learn about more model evaluation metrics including sensitivity, specificity, precision, false positive rate, ROC curves, and AUC scores to understand how well your models perform.

Chapter 7, Computer Vision with Convolutional Neural Networks, will introduce you to building image classifiers with convolutional neural networks. You will learn about all the components that comprise the architecture of convolutional neural networks and then build image processing applications to classify images.

Chapter 8, Transfer Learning and Pre-Trained Models, will introduce you to the concept of transferring the learning from one model to solve for other applications. You will achieve this by using different pre-trained models and modifying them slightly to fit different applications.

Chapter 9, Sequential Modeling with Recurrent Neural Networks, will teach you how to build models with sequential data. You will learn the architecture of recurrent neural networks and how to train them to predict the succeeding values from sequential data. You will test your knowledge by predicting the future values of various stock prices.

Conventions

Code words in text, database table names, folder names, filenames, file extensions, path names, dummy URLs, user input, and Twitter handles are shown as follows:

"sklearn has a class called train_test_split, which provides the functionality for splitting data."

Words that you see on the screen, for example, in menus or dialog boxes, also appear in the same format.

A block of code is set as follows:

# import libraries

import pandas as pd

from sklearn.model_selection import train_test_split

New terms and important words are shown like this:

"A dictionary contains multiple elements, like a list, but each element is organized as a key-value pair."

Before You Begin

Each great journey begins with a humble step. Our upcoming adventure with Applied Deep Learning with Keras, is no exception. Before we can do awesome things with Keras library, we need to be prepared with a productive environment. In this short section, we shall see how to do that.

Installing Python

Installing Python on Windows:

To install Python on Windows, do the following:

Find your desired version of Python on the official installation page at https://packt.live/37AxDz4.Ensure you select Python 3.7 from the download page.Ensure that you install the correct architecture for your computer system; that is; either 32-bit or 64-bit. You can find out this information in the System Properties window of your OS.After you download the installer, simply double-click on the file and follow the user-friendly prompts onscreen.

Installing Python on Linux:

To install Python on Linux, you have a couple of good options:

Open Comand Prompt and verify that p\Python 3 is not already installed by running python3 –version.To install Python 3, run this:

sudo apt-get update

sudo apt-get install python3.7

If you encounter problems, there are numerous sources online that can help you troubleshoot the issue.Install Anaconda Linux by downloading the installer from https://packt.live/2OYAmMw and following the instructions.

Installing Python on macOS:

Similar to Linux, you have a couple of methods for installing Python on a Mac. To install Python on macOS, do the following:

Open the Terminal for Mac by pressing CMD + Spacebar, type terminal in the open search box, and hit Enter.Install Xcode through the command line by running xcode-select –install.The easiest way to install Python 3 is using Homebrew, which is installed through the command line by running ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)".Add Homebrew to your $PATH environment variable. Open your profile in the command line by running sudo nano ~/.profile and inserting export PATH="/usr/local/opt/python/libexec/bin:$PATH" at the bottom.The final step is to install Python. In the command line, run brew install python.Again, you can also install Python via the Anaconda installer available from https://packt.live/2OZwwm2.

Installing the Code Bundle

Download the code files from GitHub at https://packt.live/2OL5E9t. Refer to these code files for the complete code bundle.

The high-quality color images used in book can be found at https://packt.live/2u9Tno4.

If you have any issues or questions about installation, please email us at [email protected].

1. Introduction to Machine Learning with Keras