40,81 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Demonstrate fundamentals of Deep Learning and neural network methodologies using Keras 2.x
Key Features
- Experimental projects showcasing the implementation of high-performance deep learning models with Keras.
- Use-cases across reinforcement learning, natural language processing, GANs and computer vision.
- Build strong fundamentals of Keras in the area of deep learning and artificial intelligence.
Book Description
Keras 2.x Projects explains how to leverage the power of Keras to build and train state-of-the-art deep learning models through a series of practical projects that look at a range of real-world application areas.
To begin with, you will quickly set up a deep learning environment by installing the Keras library. Through each of the projects, you will explore and learn the advanced concepts of deep learning and will learn how to compute and run your deep learning models using the advanced offerings of Keras. You will train fully-connected multilayer networks, convolutional neural networks, recurrent neural networks, autoencoders and generative adversarial networks using real-world training datasets. The projects you will undertake are all based on real-world scenarios of all complexity levels, covering topics such as language recognition, stock volatility, energy consumption prediction, faster object classification for self-driving vehicles, and more.
By the end of this book, you will be well versed with deep learning and its implementation with Keras. You will have all the knowledge you need to train your own deep learning models to solve different kinds of problems.
What you will learn
- Apply regression methods to your data and understand how the regression algorithm works
- Understand the basic concepts of classification methods and how to implement them in the Keras environment
- Import and organize data for neural network classification analysis
- Learn about the role of rectified linear units in the Keras network architecture
- Implement a recurrent neural network to classify the sentiment of sentences from movie reviews
- Set the embedding layer and the tensor sizes of a network
Who this book is for
If you are a data scientist, machine learning engineer, deep learning practitioner or an AI engineer who wants to build speedy intelligent applications with minimal lines of codes, then this book is the best fit for you. Sound knowledge of machine learning and basic familiarity with Keras library would be useful.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 498
Veröffentlichungsjahr: 2018
Ähnliche
Keras 2.x Projects
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Pravin DhandreAcquisition Editor: Devika BattikeContent Development Editor: Nathanya DiasTechnical Editor: Joseph SunilCopy Editor: Safis EditingProject Coordinator: Kirti PisatProofreader: Safis EditingIndexer: Mariammal ChettiyarGraphics: Jisha ChirayilProduction Coordinator: Shraddha Falebhai
First published: December 2018
Production reference: 1311218
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78953-664-5
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Giuseppe Ciaburro holds a PhD in environmental technical physics and two master's degrees. His research was focused on machine learning applications in the study of the urban sound environments. He works at Built Environment Control Laboratory—Università degli Studi della Campania Luigi Vanvitelli (Italy). He has over 15 years of professional experience in programming (Python, R, and MATLAB), first in the field of combustion and then in acoustics and noise control. He has several publications to his credit.
About the reviewer
Sudharsan Ravichandiran is a data scientist, researcher, artificial intelligence enthusiast, and YouTuber (search for Sudharsan reinforcement learning). He completed his bachelors in information technology at Anna University. His area of research focuses on practical implementations of deep learning and reinforcement learning, which includes natural language processing and computer vision. He is an open source contributor and loves answering questions on Stack Overflow. He also authored a best seller, Hands-On Reinforcement Learning with Python, published by Packt Publishing.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Keras 2.x Projects
About Packt
Why subscribe?
Packt.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Getting Started with Keras
Introduction to Keras
Keras backend options
TensorFlow
Theano
CNTK
Installation
Optional dependencies
Installing the backend engine
Keras installation and configuration
Model fitting in Keras
The Keras sequential model architecture
Keras functional API model architecture
Summary
Modeling Real Estate Using Regression Analysis
Defining a regression problem
Basic regression concepts
Different types of regression
Creating a linear regression model
Multiple linear regression concepts
Neural networks for regression using Keras
Exploratory analysis
Data splitting
Neural network Keras model
Multiple linear regression model
Summary
Heart Disease Classification with Neural Networks
Basics of classification problems
Different types of classification
Classification algorithms
Naive Bayes algorithm
Gaussian mixture models
Discriminant analysis
K-nearest neighbors
Support vector machine
Bayesian decision theory
Bayes' theorem
Pattern recognition using a Keras neural network
Exploratory analysis
Handling missing data in Python
Data scaling
Data visualization
Keras binary classifier
Summary
Concrete Quality Prediction Using Deep Neural Networks
Basic concepts of ANNs
Architecture of ANNs
Learning paradigms
Supervised learning
Unsupervised learning
Semi-supervised learning
Understanding the structure of neural networks
Weights and biases
Types of activation functions
Unit step activation function
Sigmoid
Hyperbolic tangent
Rectified linear unit
Multilayer neural networks
Implementing multilayer neural networks in Keras
Exploratory analysis
Data visualization
Data scaling
Building a Keras deep neural network model
Improving the model performance by removing outliers
Summary
Fashion Article Recognition Using Convolutional Neural Networks
Understanding computer vision concepts
Convolutional neural networks
Convolution layer
Pooling layers
Rectified linear units
Fully connected layer
Structure of a CNN
Common CNN architecture
LeNet-5
AlexNet
ResNet
VGG Net
GoogleNet
Implementing a CNN for object recognition
Exploratory analysis
Data scaling
Using Keras in the CNN model
Exploring the model's results
Summary
Movie Reviews Sentiment Analysis Using Recurrent Neural Networks
Sentiment analysis basic concepts
Sentiment analysis techniques
The next challenges for sentiment analysis
Lexicon and semantics analysis
Recurrent neural networks
Fully recurrent neural networks
Recursive neural networks
Hopfield recurrent neural networks
Elman neural networks
Long short-term memory network
Classifying sentiment in movie reviews using an RNN
IMDB Movie reviews dataset
Exploratory analysis
Keras recurrent neural network model
Exploring model results
Summary
Stock Volatility Forecasting Using Long Short-Term Memory
The basics of forecasting
Forecast horizon
Forecasting methods
Quantitative methods
Qualitative methods
Time series analysis
The classical approach to time series
Estimation of the trend component
Estimating the seasonality component
Time series models
Autoregressive models
Moving average models
Autoregressive moving average model
Autoregressive integrated moving average models
Long short-term memory in Keras
Implementing an LSTM to forecast stock volatility
Exploratory analysis
Data scaling
Data splitting
Keras LSTM model
Summary
Reconstruction of Handwritten Digit Images Using Autoencoders
Basic concepts of image recognition
Image digitization
Image recognition
Optical character recognition
Approaches to the problem
Generative neural networks
The restricted Boltzmann machine
Autoencoders
Variational autoencoders
The generative adversarial network
The adversarial autoencoder
The Keras autoencoders model
Implementing autoencoder Keras layers to reconstruct handwritten digit images
The MNIST dataset
Min–max normalization
Keras model architecture
Exploring model results
Summary
Robot Control System Using Deep Reinforcement Learning
Robot control overview
Three laws of robotics
Short robotics timeline
First-generation robots
Second-generation robots
Third-generation robots
Fourth-generation robots
Automatic control
The environment for controlling robot mobility
OpenAI Gym
Reinforcement learning basics
Agent-environment interface
Reinforcement learning algorithms
Dynamic Programming
Monte Carlo methods
Temporal difference learning
Keras DQNs
Q-learning
Deep Q-learning
Keras-RL library
DQN to control a robot's mobility
OpenAI Gym installation and methods
The CartPole system
Q-learning solution
Deep Q-learning solution
Summary
Reuters Newswire Topics Classifier in Keras
Natural language processing
NLP phases
Morphology analysis
Syntax analysis
Semantic analysis
Pragmatic analysis
Automatic processing problems
NLP applications
Information retrieval
Information extraction
Question-answering
Automatic summarization
Automatic translation
Sentiment analysis
NLP methods
Sentence splitting
Tokenization
Part-of-speech tagging
Shallow parsing
Named entity recognition
Syntactic parsing
Semantic role labeling
Natural language processing tools
The Natural Language Toolkit
The Stanford NLP Group software
Apache OpenNLP
GATE
The Natural Language Toolkit
Getting started with the NLTK
Corpora
Brown corpus
Word and sentence tokenize
Part-of-speech tagger
Stemming and lemmatization
Stemming
Lemmatization
Implementing a DNN to label sentences
Exploratory analysis
Data preparation
Keras deep neural network model
Summary
What is Next?
Deep learning methods
Deep feedforward network
Convolutional neural networks
Recurrent neural networks
Long short-term memory
Restricted Boltzmann machine
Deep belief network
Generative adversarial networks
Automated machine learning
Auto-Keras
Google Cloud ML Engine
Azure Machine Learning Studio
Amazon Web Services
Differentiable neural computer
Genetic programming and evolutionary strategies
Introducing the genetic algorithm
The fitness function
Selection
Mutation
Inverse reinforcement learning
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Keras 2.x Projects explains how to leverage the power of Keras to build and train state-of-the-art deep learning models through a series of practical projects that look at a range of real-world application areas.
To begin with, you will quickly set up a deep learning environment by installing the Keras library. Through each of the projects, you will explore and learn the advanced concepts of deep learning and will learn how to compute and run your deep learning models using the advanced offerings of Keras. You will train fully-connected multilayer networks, convolutional neural networks, recurrent neural networks, autoencoders and generative adversarial networks using real-world training datasets. The projects you will undertake are all based on real-world scenarios of all complexity levels, covering topics such as language recognition, stock volatility, energy consumption prediction, faster object classification for self-driving vehicles, and more.
By the end of this book, you will be well versed with deep learning and its implementation with Keras. You will have all the knowledge you need to train your own deep learning models to solve different kinds of problems.
Who this book is for
If you are a data scientist, machine learning engineer, deep learning practitioner or an AI engineer who wants to build speedy intelligent applications with minimal lines of codes, then this book is the best fit for you.
What this book covers
Chapter 1, Getting Started with Keras, covers an overview of the Keras environment. We will learn how to install and configure Keras, how to work with the Keras library, and we will discover the basic concepts of the Keras architecture. We will also see how Keras uses TensorFlow as its tensor manipulation library, as well as how we can switch the Keras backend from TensorFlow (which is the default option) to Theano and CNTK, which are other available frameworks. Finally, we will understand the different types of Keras models, looking at model classes used with sequential layers and model classes used with functional API layers.
Chapter 2, Modeling Real Estate Using Regression Analysis, looks at the different types of regression techniques. We will apply regression methods to your data and understand how the regression algorithm works. We will then understand the basic concepts that multiple linear regression methods use to fit equations to data using the Keras layers. We will also learn how to evaluate the model's performance and how to tune a model to improve the model's performance.
Chapter 3, Heart Disease Classification with Neural Networks, covers the basic concept of the classification problem. We will look at a practical case of heart disease classification using Keras. The basic concepts of classification methods and how to implement them in the Keras environment will be covered. How to import and organize data for neural network classification analysis, and how to implement a Keras classification model using a real-life example, will be explored.
Chapter 4, Concrete Quality Prediction Using Deep Neural Networks, covers the basic concepts of multilayer neural networks (MNNs) and how to implement them in the Keras environment. Many variations of MNNs will be introduced, such as CNNs, RNNs, deep belief networks (DBNs), and restricted Boltzmann machines (RBMs). We will look at an example of a multilayered artificial network. We will learn about how to implement a model that allows us to calculate the compressive strength of concrete according to the ingredients used in the mixture. Finally, we will see how to remove outliers to improve the performance of the model.
Chapter 5, Fashion Article Recognition Using Convolutional Neural Networks, covers the connection pattern between neurons in CNNs, which is inspired by the structure of the visual cortex in the brain. We will see how the individual neurons present in this part of the brain (visual cortex) respond to certain stimuli in a narrow region of observation called the receptive field. The receptive fields of different neurons are partially overlapped so that together they cover the entire field of view. The response of a single neuron to stimuli taking place in its receptive field can be mathematically approximated by a convolution operation. In this chapter, a CNN will be trained to solve a pattern recognition problem.
Chapter 6, Movie Review Sentiment Analysis Using Recurrent Neural Networks, covers the RNN, which is a neural model wherein a bidirectional flow of information is present. In other words, while the propagation of signals in feedforward networks takes place only in a continuous manner in a direction from inputs to outputs, RNNs are different. In them, this propagation can also occur from a neural layer following a previous one, or between neurons belonging to the same layer, and even between a neuron and itself. In this chapter, an RNN is used to solve a language recognition problem.
Chapter 7, Stock Volatility Forecasting Using Long-Short Term Memory, covers the basic concepts of forecasting techniques. Forecast horizon and several forecast methods will be analyzed. Time series data will also be dealt with. A time series constitutes a sequence of observations of a phenomenon. In a time series, we can identify several components—trend, seasonality, cycle, and residual. We will learn how to remove seasonality from a time series with a practical example. We will address the most-used models to represent time series—AR, MA, ARMA, and ARIMA. For each one, the basic concepts will be analyzed and then a mathematical formulation of the model will be provided. Finally, a Keras LSTM model for time series analysis will be proposed. Using a practical example, we will see how to deal with a time series regression problem with an RNN model of the LSTM type.
Chapter 8, Reconstruction of Handwritten Digital Images Using Autoencoders, looks at autoencoders. The autoencoder is a neural network whose purpose is to code its input into small dimensions, where the result obtained helps to reconstruct the input itself. Autoencoders are made up of the union of two subnets: an encoder and a decoder. The encoder and the decoder will be differentiable with respect to the distance function, so the parameters of the encoding/decoding functions can be optimized to minimize the loss of reconstruction, using the stochastic gradient. In this chapter, an autoencoder is used to reconstruct handwritten digit images.
Chapter 9, Robot Control System Using Deep Reinforcement Learning, introduces the basic concepts of reinforcement learning and how to use these techniques to control a mechanical system. To start with, an overview of robot control will be addressed. Then, the OpenAI Gym library will be introduced, which helps us to implement algorithms based on reinforcement learning. It includes a growing collection of benchmark issues that are shared on a common interface, and a website where people can share their results andcompare algorithm performance. We will explore the different environments available and how to install the library. Finally, the CartPole system will be used to implement Q-learning and Deep Q-learning algorithms. The CartPole system is a classic problem of reinforcement learning. The system consists of a pole (which acts like an inverted pendulum) attached to a cart via a joint. The system is controlled by applying a force of +1 or -1 to the cart. The force applied to the cart can be controlled, and the objective is to swing the pole upward and stabilize it.
Chapter 10, Reuters Newswire Topics Classifier in Keras, covers natural language processing (NLP), which is the process of automatically processing information that is written or spoken in a natural language using an electronic calculator. This process is made particularly difficult and complex due to the intrinsic ambiguity of human language. In this chapter, Keras layers are used to build a model to classify Reuter's newswire topics. Data is available from a dataset that contains 11,228 newswires from Reuters with 46 labeled topics. This dataset is from Keras, and each wire is encoded as a sequence of word indexes.
Chapter 11, What is Next?, will summarize what will have been covered in this book and what the next steps are that you can take. You will learn how to apply the skills you have gained to other projects, including real-life challenges in building and deploying Keras deep learning models and other common technologies that data scientists often use. By the end of this chapter, you will have a better understanding of the real-life challenges in building and deploying machine learning models, and the additional resources and technologies that you can use to sharpen your machine learning skills.
To get the most out of this book
Sound knowledge of machine learning and a basic familiarity with the Keras library would be useful.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Keras-2.x-Projects. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781789536645_ColorImages.pdf.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
Getting Started with Keras
Keras is an open source neural network library written in Python. This book will help you to experiment with deep neural networks as simply as possible. Its principal author and maintainer is François Chollet, a Google engineer. In 2017, Google's TensorFlow team decided to support Keras in TensorFlow's main library. Keras contains several implementations of commonly used neural network blocks, such as levels, objectives, activation functions, optimizers, and a set of tools to facilitate work with image and text data. In this chapter, an overview of the Keras environment will be addressed.
The following topics are covered:
Introduction to Keras
Keras backend options
Installation
Model fitting in Keras
At the end of the chapter, the reader will learn how to work with the keras library, and how to install and configure Keras. We will also discover the basic concepts of the Keras architecture. We will learn how Keras uses TensorFlow as its tensor manipulation library. We will also understand the different type of Keras models, sequential and functional APIs, and learn how to implement Keras layers.
Introduction to Keras
Keras is a Python library that provides a simple and clean way to create a range of deep learning models. Keras code was released under the MIT license. Keras has been structured based on austerity and simplicity, and it provides a programming model without ornaments that maximizes readability. It allows expressing neural networks in a very modular way, considering a model like a sequence or a single graph. This is a good approximation, because the components of a deep learning model are discrete elements that can be arbitrarily combined. The new components are easily aggregated and modifiable within the framework designed for engineers, to quickly test and explore new ideas. Last but not least, using the Python programming language provides constructs that allow clear programming on both a small and large scale. In the following screenshot, we can see the Keras official home page (https://keras.io/):
Ease of use is the strongpoint of Keras. During the design phase, the user has been the focus of attention of the developers, producing a product that reduces the user's work via simple and consistent APIs. In this way, the number of actions necessary to solve common use cases is reduced. In addition, the results are returned in a clear manner, making the identification of possible errors very simple.
In Keras, a model is represented by a sequence of autonomous and completely configurable modules that can be connected with the lowest possible number of restrictions. Everything in Keras is a module—neural layers, cost functions, optimizers, initialization schemes, activation functions, and regularization schemes. These independent modules can be combined to create new, more complex models.
All modules available in Keras are simple to add, and so are new classes and functions in a programming language. In addition, the modules are already available, and are accompanied by numerous examples that explain their practical use. But Keras is not limited to the availability of the built-in modules. The user will be able to easily create new modules, making Keras an easily extensible environment.
The keras library bases its technology on the levels that are used to manage input and output. An application in Keras can be implemented through the following four simple steps:
Prepare input and output data.
Create the first level to manage the input data.
Set up intermediate levels to perform the analysis.
Create the output level to manage the targets.
Keras works as a specific high-level API for neural networks. It can act as a user interface and can extend the functionality of other deep learning framework backends on which it runs. Thanks to this feature, Keras has become a wrapper for migration between frameworks. Not only can algorithms and models of neural networks for deep learning be exchanged, but also networks and preliminary weights.
Wrapper libraries consist of a thin layer of code that translates a library's existing interface into a compatible interface.
On the other hand, since Keras is autonomous, it can be used without having to interact with the backend framework on which it is running. Keras has its own chart data structures for defining computational charts; it is not based on the data structures of the underlying backend framework. This way, you will not have to learn how to program the backend framework.
Keras is easy to learn and use. Using Keras is like working with Lego blocks, you just have to put in sequence a series of compatible modules. It was created so that people can quickly perform the experimental phase of the model using a highly modular and extensible framework. Keras focuses on defining levels for the neural network. You do not have to deal with tensors, but it's easy to write with less code.
Keras backend options
Keras is a model-level library that provides high-level blocks for the development of deep learning models. Keras developers have focused their efforts on creating high-level models by neglecting low-level operations such as tensor products, convolutions, and so on. These operations have been entrusted to specialized and well-optimized tensor manipulation libraries that already exist, thus acting as a backend engine for Keras. Several backend engines can be connected perfectly to Keras. Actually, Keras has three backend implementations available—TensorFlow, Theano, and Microsoft Cognitive Toolkit (CNTK).
TensorFlow
TensorFlow is an open source software library for numerical calculation based on graph modeling (data flow graphs). A graph is defined as an abstract pipeline of mathematical operations operating on tensors, and are also known as multidimensional arrays. Each graph consists of nodes and arcs, wherein the nodes are operations on the data, and the arcs represent the tensors that pass through the various operations.
You can find the updated version of the library and all the documentation supplied at the following link: (http://www.tensorflow.org).
TensorFlow is the most commonly used library in the field of machine learning and neural networks. It has numerous APIs, including the lowest level, that is, TensorFlow Core, allows complete control over programming. These APIs are those typically used in the field of machine learning, since they make it possible to check in detail all the elements of the model being implemented. The highest level APIs are available and built from TensorFlow Core. In some cases, they can make some operations such as repetitive and predefined tasks faster and simpler, but generally preclude the possibility of going into detail, and in the implementation of a neural network it is often necessary to have a more precise control over operations. However, they can still be useful for the development of standard machine learning models.
TensorFlow provides interfaces for different languages, including Python and C or C ++, with full support, and Go or Java in Beta. It also supports parallel computing on GPUs or CPUs, and distributed computation allows execution even on mobile devices.
A TensorFlow program is typically structured in the following two distinct phases:
Construction
: In this phase, the various operations of the graph that will be performed on the input tensors are defined.
Execution
: In this phase, the operations defined in the previous phase are evaluated so as to retrieve the numerical output. The execution of operations is managed through the
session
object of TensorFlow.
The fundamental unit in TensorFlow is the tensor. A tensor consists of a set of primitive type values modeled as a multidimensional array. In TensorFlow, almost all the functions contained in the API take tensors as inputs and always output tensors. Each tensor contained in the graph has a unique name that can be specified by the user, or otherwise automatically assigned.
For more information on tensors and any other resources related to TensorFlow, you can refer to the official site of the framework, as shown in the following screenshot:
The default behavior in TensorFlow is to allocate all the components (tensors and operations) in the GPU memory (if it has been installed with computational support on a GPU). However, you can manually specify where to allocate each tensor and operation. It is recommended to minimize switches between the CPU and GPU as these slow down execution.
TensorFlow also allows you to define scopes for variables through which a namespace mechanism is managed; they facilitate the definition of complex models. Scopes are also very important for sharing variables between multiple graphs.
TensorFlow provides the following two options for reading input data:
Manual
: Observations are manually read and organized in batches, and then passed to the model. It is a simple mechanism to use, but it can become very slow because the data must be continually copied from the Python environment to the TensorFlow environment.
Integrated
: All operations to read data and organize them in batches of observations are implemented within the graph. It is a less intuitive mechanism to use, but presents a large increase in performance, since all data always remains in the TensorFlow environment.
All basic operations for neural networks are implemented in TensorFlow as graph nodes. The framework automatically manages everything needed to implement the forward and backward pass, including the automatic calculation of derivatives.
Among the main operations available for the construction of neural networks models, we find the following:
Convolutions
Sum of the bias
Fully connected levels
Activation functions
Pooling
Prediction functions
Finally, there is a suite of tools for graphic display that is fully integrated into TensorFlow called TensorBoard. It allows visualizing the computational graph of the model and many other statistics useful for the analysis of the training process.
Theano
Theano is an open source library of numerical computation for the Python programming language developed by a group of machine learning experts at the University of Montreal. In Theano, calculations are expressed using a syntax that is similar to NumPy's, and is compiled to perform efficiently on both CPU and GPU architectures.
You can find the updated version of the library and all the documentation supplied at the following link: http://deeplearning.net/software/theano.
It is named after a Greek mathematician, Theano of Crotone. Theano is a compiler for mathematical expressions written in Python. It allows defining, optimizing, and evaluating mathematical expressions, in particular those with multidimensional arrays. Using Theano, it is possible to achieve comparable speeds with C or C ++ applications for problems with large amounts of data. Theano combines aspects of a computer algebra system with aspects of an optimizing compiler. These characteristics are useful when complex mathematical expressions are evaluated repeatedly, and the evaluation speed is crucial. In the following screenshot, we can see the official home page of Theano:
It combines the convenience of the NumPy syntax with the speed of an optimized machine language. The user composes mathematical expressions in a high-level description that mimics the syntax and semantics of NumPy, even though it is statically typed. These expressions allow Theano to provide symbolic differentiation. Before performing the calculation, Theano optimizes the choice of expressions, translates them into C ++ (or CUDA for GPU), and automatically compiles them into dynamically loaded Python modules. The machine learning algorithms implemented with Theano are particularly fast.
CNTK
The CNTK is an open source toolkit for commercial-grade applications that is distributed in deep learning. CNTK implements neural networks as a series of computational steps via a directed graph. CNTK is a command-line program that can do simple and deep neural network analyses. CNTK was originally developed as an internal Microsoft tool.
You can find the updated version of the library and all the documentation supplied at the following link: https://github.com/Microsoft/CNTK.
Using CNTK, the most common machine learning models are easily achievable and can be combined. CNTK is available as a library in Python, C#, or C++ environments, or it can be used as a standalone machine learning tool through its own model description language. In addition, CNTK model evaluation functionality can be used from Java programs. CNTK supports 64-bit Linux or 64-bit Windows operating systems. To install CNTK, you can either choose precompiled binary packages, or compile the toolkit from the source provided in GitHub. In the following screenshot, we can see the official home page of CNTK:
In the Tutorials section, there is a collection of code samples, recipes, and tutorials on the various ways you can use the toolkit with scenarios for images, text, and speech data.
Installation
Previously, we have already said that Keras is written in Python, so in order for it to work, it is necessary to have a previously installed version of Python (Keras is compatible with Python 2.7-3.6). Platforms that support Python development environments can support Keras as well. Furthermore, before installing Keras, it is necessary to provide for the installation of the backend engine, and some optional dependencies useful for the implementation of machine learning models.
Optional dependencies
Some useful optional dependencies are listed in the following list:
NumPy
: This is an open source library of the Python programming language, which adds support for multidimensional and large vectors, and even matrices with high-level mathematical functions to work with.
SciPy
: This is an open source library of mathematical algorithms and tools. It contains modules for optimization, linear algebra, integration, special functions,
fast fourier transform
(
FFT
), signal and image processing,
ordinary differential equation
(
ODT
) solvers, and other common tools in science and engineering.
Scikit-learn
: This is an open source machine learning library for the Python programming language. It contains classification, regression, clustering algorithms, support vector machines, logistic regression, bayesian classifier, k-means, and DBSCAN, and is designed to work with the NumPy and SciPy libraries.
cuDNN
: This is a GPU accelerated library of primitives for deep neural networks. cuDNN provides highly tuned implementations for standard routines, such as forward and backward convolution, pooling, normalization, and activation layers.
HDF5
: This is a data model, library, and file format for storing and managing data. It supports an unlimited variety of datatypes, and is designed to be flexible and efficient.
H5py
: This is a Python interface to the HDF5 binary data format.
Graphviz
: This is an open source program used to draw graphs described in the DOT language. It provides libraries for applications using the tools provided. Graphviz is free software licensed under the
Common Public License
(
CPL
).
Pydot
: This is a Python interface for Graphviz and the DOT language.
We can now proceed with the installation of every single library or install all the dependencies with a single line of code. Alternatively, you can install the Anaconda Python module, which will automatically install these libraries and a lot of other libraries that are needed for scientific computing.
Installing the backend engine
After installing the dependencies, it is necessary to select the backend engine and proceed with its installation. Keras developers recommend the TensorFlow backend. This is the one set by default.
Links to installation instructions for all the available backends are listed in the following list:
TensorFlow installation instructions:
https://www.tensorflow.org/install/
Theano installation instructions:
http://deeplearning.net/software/theano/install.html#install
CNTK installation instructions:
https://docs.microsoft.com/en-us/cognitive-toolkit/setup-cntk-on-your-machine
In all cases, we can use the pip command to install the correct packages.
Keras installation and configuration
At this point, the environment is ready for Keras installation. If all the dependencies and the backend engine have been correctly installed, we can proceed. Actually, the work that remains to be done is very simple and immediate.
There are two ways to install Keras as follows:
The first step involves the use of the Python Package Index (PyPI).
This is a repository of software for the Python programming language. PyPI helps you find and install software developed and shared by the Python community. This is the recommended option.
After installing the PyPI package, simply type the following command:
$ sudo pip install keras
If you are using a virtualenv, you may want to avoid using sudo:
$ pip install keras
Alternatively, we can install Keras from the GitHub source. To do this, we first need to clone Keras using git:
$ git clone https://github.com/keras-team/keras.git
Then, add cd to the keras folder and run the install command:
cd keras
sudo python setup.py install
We have the Keras environment available to implement our deep learning models. Let us look at some more information about the configuration. As we have already mentioned, Keras uses TensorFlow as the default backend. If we want to use the other two options, Theano and CNTK, we must modify the Keras configuration file. This is a file with the .json extension, and is named keras.json..
The file's position depends on the operating system that is in use in our PC:
$ HOME/.keras/keras.json
We can use the following command for Unix-like OSes:
%USERPROFILE%/.Keras/keras.json
And, for Windows OS, the contents of the configuration file are of the following type:
{ "image_data_format": "channels_last", "epsilon": 1e-07, "floatx": "float32", "backend": "tensorflow"}
The parameters have the following meaning:
image_data_format
: String, either
channels_last
or
channels_first
. It specifies which data format convention Keras will follow.
epsilon
: Float, a numeric fuzzing constant used to avoid dividing by zero in some operations.
floatx
: String,
float16
,
float32
, or
float64
. Default float precision.
backend
: String,
tensorflow
,
theano
, or
cntk
.
Switching from one backend to another is very fast. Simply change the field backend to theano, tensorflow, or cntk, and Keras will use the new configuration the next time you run any Keras code.
Model fitting in Keras
We have just installed and configured our Keras environment, and we can now focus on the implementation of our model based on deep neural networks. When developing a deep learning application, we follow a general pipeline characterized by the following steps:
Collecting the data
: Everything starts from the data, no doubt about it, but one might wonder where so much data comes from. In practice, it is collected through lengthy procedures that may, for example, derive from measurement campaigns or face-to-face interviews. In all cases, the data is collected in a database so that it can then be analyzed to derive knowledge.
If we do not have specific requirements, to save time and effort we can use publicly available data. In this regard, a large collection of data is available in the UCI Machine Learning Repository at the following link: https://archive.ics.uci.edu/ml/index.php.
Preparing the data
: We have collected the data; now we have to prepare it for the next step. Once we have this data, we must make sure it is in a format usable by the algorithm we want to use. To do this, you may need to do some formatting. Recall that some algorithms need data in an integer format, whereas others require data in the form of strings. Finally, others need to be in a special format. We will get to this later, but the specific formatting is usually simple compared to data collection.
The following diagram shows the deep learning process workflow:
Exploring the data
: At this point, we can look at data to verify that it is actually working and we do not have a bunch of empty values. In this step, through the use of plots, we can recognize patterns or whether there are some data points that are vastly different from the rest of the set. Plotting data in one, two, or three dimensions can also help.
Training the algorithm
: Now, let's get serious. In this step, the deep learning begins to work with the definition of the model and the next training round. The model starts to extract knowledge from large amounts of data that we had available. For unsupervised learning, there's no training step because you don't have a target value.
Testing the algorithm
: In this step, we use the information learned in the previous step to see if the model actually works. The evaluation of an algorithm verifies how well the model approximates the real system. In the case of supervised learning, we have some known values that we can use to evaluate the algorithm. In unsupervised learning, we may need to use some other metrics to evaluate success. In both cases, if we are not satisfied, we can return to the previous steps, change some things, and retry the test.
Evaluating the algorithm
: We have reached the point where we can apply what has been done so far. We can assess the approximation ability of the model by applying it to real data. The model, preventively trained and tested, is then valued in this phase.
Improving algorithm performance
: Finally, we can focus on the finishing steps. We've verified that the model works, we have evaluated the performance, and now we are ready to analyze the whole process to identify any possible room for improvement.
In Keras, there are two ways to define a model—sequential, and functional API. The sequential model lets you create layer-by-layer models for most problems. Limits are dictated by the inability to create models that share levels or that have multiple inputs or outputs. Alternatively, the functional API allows you to create models with greater flexibility. We can easily define models in which the levels are connected in different ways and not just from the previous level to the next. In fact, we can link a layer to any other level, thus creating complex networks.
Summary
In this chapter, an overview of the Keras environment has been explored. We have learned how to install and configure Keras and how to work with the keras library, and have discovered the basic concepts of the Keras architecture. We have also seen how Keras uses TensorFlow as its tensor manipulation library, how we can switch the Keras backend from TensorFlow, which is the default option, to Theano and CNTK, and other available frameworks. Finally, we have understood the different types of Keras model, and we discussed model classes used with sequential layers and those used with functional API layers.
In the next chapter, you will learn the different types of regression techniques and how to apply regression methods to your data, and will understand how the regression algorithm works. We will understand the basic concepts that multiple linear regression methods use to fit equations to data using Keras layers. We will also learn how to evaluate the model's performance, and learn how to tune a model to improve its performance.
Modeling Real Estate Using Regression Analysis
The real estate market is a type of market where the sales and purchases between sellers and buyers refer to the exchange of real estate of any kind, such as housing, land, commercial premises, and so on. Real estate prices depend on a series of factors that make the asset more palatable for potential buyers. Regression analysis is the statistical process of studying the relationship between a set of independent variables (explanatory variables) and the dependent variable (response variable). Through this technique, it is possible to understand how the value of the response variable changes when the explanatory variable is varied. In this chapter, the real estate market will be modeled through a regression analysis.
In this chapter, we will cover the following topics:
Defining a regression problem
Creating a linear regression model
Multiple linear regression concepts
Neural networks for regression using Keras
By the end of this chapter, we will have learned about the different types of regression techniques. We will apply regression methods to your data and understand how the regression algorithm works. We will then understand the basic concepts that multiple linear regression methods use to fit equations to data using the Keras layers. We will also learn how to evaluate the model's performance and how to tune a model to improve the model's performance.
Defining a regression problem
Regression analysis is the starting point in data science. This is because regression models represent the most well-understood models in numerical simulation. Once we experience the workings of regression models, we will be able to understand all other machine learning algorithms. Regression models are easily interpretable as they are based on solid mathematical bases (such as matrix algebra, for example). In the following sections, we will see that linear regression allows us to derive a mathematical formula that's representative of the corresponding model. Perhaps this is why such techniques are extremely easy to understand.
Regression analysis is a statistical process that's implemented to study the relationship between a set of independent variables (explanatory variables) and the dependent variable (response variable). Through this technique, it will be possible to understand how the value of the response variable changes when the explanatory variable is varied.
Consider some data that is collected about a group of students, on the number of study hours per day, attendance at school, and the scores that they have obtained on the final exam. Through regression techniques, we can quantify the average increase in the final exam score when we add one more hour of study. Lower attendance in school (decreasing the student's experience) lowers the scores in the final exam.
A regression analysis can have two objectives:
Explanatory analysis: To understand and weigh the effects of the independent variable on the dependent variable, according to a particular theoretical model
Predictive analysis: To locate a linear combination of the independent variable to predict the value assumed by the dependent variable optimally
However, regression, given its cross-disciplinary characteristics, has numerous and varied areas of applications, right from psychology to agrarianism, and from economics to medicine and business management, just to name a few.
The purpose of regression as a statistical tool is of two types, namely to synthesize and generalize, as we can see in the following diagram:
Synthesize: The first purpose (synthesize) means predisposing collected data into a form (tables, graphs, or numerical summaries), which allows you to understand better, the phenomena on which the detection was performed. The synthesis is met by the need to simplify, which in turn results from the limited ability of the human mind to handle articulated, complex, or multidimensional information. In this way, we can use techniques that allow for a global study of a large number of quantitative and qualitative information to highlight features, ties, differences, or associations between detected variables.
Generalize: The second purpose (generalize) is to extend the result of an analysis performed on the data of a limited group of statistical units (sample) to the entire population group (population). The contribution of regression is not limited to the data analysis phase. It's true that the added value is expressed in the formulation of research hypotheses, argumentation of theses, adoption of appropriate solutions and methodologies, choices of methods of detection, formulation of the sample, and the procedure of extending the results to the reference universes.
Keeping these phases under control means producing reliable and economically useful results, and mastering descriptive statistics and data analysis as well as inferential ones. In this regard, we recall that the descriptive statistics are concerned with describing the experimental data with a few significant numbers or graphs. Therefore, they photograph a given situation and summarize its salient characteristics. Inferential statistics use statistical data that is also appropriately summarized by the descriptive statistics to make probabilistic forecasts on future or otherwise uncertain situations.
People, families, businesses, public administrations, mayors, ministers, and researchers constantly make decisions. For most of them, the outcome is uncertain, in the sense that it is not known exactly what will result, although the expectation is that they will achieve the (positive) effects they are hoping for. Decisions would be better and the effects would be expected closer to those desired if they were made on the basis of relevant data in a decision-making context.
Let's look at some applications of regression in the real world in the following section:
A student who graduates this year must choose the faculty and university degree in which he or she will enroll. Perhaps he or she has already gained a vocation for his or her future profession, or studies, and may have confirmed his or her predisposition for a particular discipline. Maybe a well-established family tradition advises him or her to follow their parent's profession. In these cases, the uncertainty of choice will be greatly reduced. However, if the student does not have genuine vocations or is not geared particularly to specific choices, he or she may want to know something about the professional outcomes of the graduates. In this regard, some statistical studies on graduate data from the previous years may help him or her make a decision.
A distribution company such as a supermarket chain wants to open a new sales outlet in a big city and must choose the best location. It will use and analyze numerous statistical data on the density of the population in different neighborhoods, the presence of young families, the presence of children under the age of six (if it is interested in selling to this category of consumers), and the presence of schools, offices, other supermarkets, and retail outlets.
Another company wants to invest its profits. It must make a portfolio choice and it has to decide whether to invest in government bonds, national shares, foreign securities, funds, or real estate. To make this choice, it will first conduct an analysis of the returns and risks of different investment alternatives based on statistical data.
