32,39 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
Recent developments in deep learning, including Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and Deep Reinforcement Learning (DRL) are creating impressive AI results in our news headlines - such as AlphaGo Zero beating world chess champions, and generative AI that can create art paintings that sell for over $400k because they are so human-like.
Advanced Deep Learning with Keras is a comprehensive guide to the advanced deep learning techniques available today, so you can create your own cutting-edge AI. Using Keras as an open-source deep learning library, you'll find hands-on projects throughout that show you how to create more effective AI with the latest techniques.
The journey begins with an overview of MLPs, CNNs, and RNNs, which are the building blocks for the more advanced techniques in the book. You’ll learn how to implement deep learning models with Keras and TensorFlow 1.x, and move forwards to advanced techniques, as you explore deep neural network architectures, including ResNet and DenseNet, and how to create autoencoders. You then learn all about GANs, and how they can open new levels of AI performance. Next, you’ll get up to speed with how VAEs are implemented, and you’ll see how GANs and VAEs have the generative power to synthesize data that can be extremely convincing to humans - a major stride forward for modern AI. To complete this set of advanced techniques, you'll learn how to implement DRL such as Deep Q-Learning and Policy Gradient Methods, which are critical to many modern results in AI.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 362
Veröffentlichungsjahr: 2018
Ähnliche
Table of Contents
Advanced Deep Learning with Keras
Advanced Deep Learning with Keras
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Acquisition Editor: Frank Pohlmann, Andrew Waldron, Suresh Jain
Content Development Editor: Alex Sorrentino
Technical Editor: Gaurav Gavas
Project Editor: Kishor Rit
Proofreader: Safis Editing
Indexers: Aishwarya Gangawane
Graphics: Tom Scaria
Production Coordinator: Sandip Tadge
First published: October 2018
Production reference: 1311018
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78862-941-6
www.packtpub.com
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Rowel Atienza is an Associate Professor at the Electrical and Electronics Engineering Institute of the University of the Philippines, Diliman. He holds the Dado and Maria Banatao Institute Professorial Chair in Artificial Intelligence. Rowel has been fascinated with intelligent robots since he graduated from the University of the Philippines. He received his MEng from the National University of Singapore for his work on an AI-enhanced four-legged robot. He finished his Ph.D. at The Australian National University for his contribution on the field of active gaze tracking for human-robot interaction. Rowel's current research work focuses on AI and computer vision. He dreams on building useful machines that can perceive, understand, and reason. To help make his dreams become real, Rowel has been supported by grants from the Department of Science and Technology (DOST), Samsung Research Philippines, and Commission on Higher Education-Philippine California Advanced Research Institutes (CHED-PCARI).
I would like to thank my family, Che, Diwa, and Jacob. They never cease to support my work.
I would like to thank my mother who instilled into me the value of education.
I would like to express my gratitude to the people of Packt and this book's technical reviewer, Frank, Kishor, Alex, and Valerio. They are inspiring and easy to work with.
I would like to thank the institutions who always support my teaching and research agenda, University of the Philippines, DOST, Samsung Research PH, and CHED-PCARI.
I would like to acknowledge my students. They have been patient as I develop my courses in AI.
About the reviewer
Valerio Maggio is currently a Post-Doc Data Scientist at Fondazione Bruno Kessler (FBK) in Trento, Italy, responsible for Machine and Deep Learning in the MPBA lab (Predictive Models for Biomedicine and Environment). Valerio has a Ph.D. in Computational Science from the University of Naples "Federico II" His research interests are focused on Machine Learning and Deep Learning applied to Software Maintenance and Computational Biology. Valerio is very much involved in the scientific Python community, and he is an active speaker at many Python conference.
He is also the lead organiser of PyCon Italy/PyData Florence, and EuroSciPy. He uses Python as the mainstream language for his deep/machine learning code, making an intensive use of Python to analyse, visualise, and learn from data. In the context of Deep Learning, Valerio is the author of a quite popular Keras/TensorFlow tutorial, publicly available on his GitHub Profile github.com/leriomaggio/deep-learning-keras-tensorflow and presented in many conferences (EuroSciPy, PyData London, PySS) and University courses. Valerio is also passionate about (black) tea, and an "old-school" Magic The Gathering (MTG) player, who enjoys playing and teaching MTG to newbies.
Packt is Searching for Authors Like You
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Preface
In recent years, deep learning has made unprecedented success stories in difficult problems in vision, speech, natural language processing and understanding, and all other areas with abundance of data. The interest in this field by companies, universities, governments, and research organizations has accelerated the advances in the field. This book covers select important advances in deep learning. The advanced theories are explained by giving a background of the principles, digging into the intuition behind the concepts, implementing the equations and algorithms using Keras, and examining the results.
Artificial Intelligence (AI), as it stands today, is still far from being a well-understood field. Deep learning, as a sub field of AI, is in the same position. While it is far from being a mature field, many real-world applications such as vision-based detection and recognition, product recommendation, speech recognition and synthesis, energy conservation, drug discovery, finance, and marketing are already using deep learning algorithms. Many more applications will be discovered and built. The aim of this book is to explain advanced concepts, give sample implementations, and let the readers, as experts in their field, identify the target applications.
A field that is not completely mature is a double-edged sword. On one edge, it offers a lot of opportunities for discovery and exploitation. There are many unsolved problems in deep learning. This translates into opportunities to be the first to market – product development, publication, or recognition. The other edge is that it would be difficult to trust a not completely well-understood field in a mission-critical environment. We can safely say that if asked, very few machine learning engineers will ride an auto-pilot plane controlled by a deep learning system. There is a lot of work to be done to gain this level of trust. The advanced concepts that are discussed in this book have a high chance of playing a major role as the foundation in gaining this level of trust.
Every book in deep learning will not be able to completely cover the whole field. This book is not an exception. Given the time and space, we could have touched interesting areas such as detection, segmentation and recognition, visual understanding, probabilistic reasoning, natural language processing and understanding, speech synthesis, and automated machine learning. However, this book believes in choosing and explaining select areas so that readers can take up other fields that are not covered.
As the reader is about to read the rest of this book, they need to keep in mind that they chose an area that is exciting and can have a huge impact on the society. We are fortunate to have a job that we look forward to working on as we wake up in the morning.
Who this book is for
The book is intended for machine learning engineers and students who would like to gain a better understanding of advanced topics in deep learning. Each discussion is supplemented with code implementation in Keras. This book is for readers who would like to understand how to translate theory into a working code implementation in Keras. Apart from understanding theories, code implementation is usually one of the difficult tasks in applying machine learning to real-world problems.
What this book covers
Chapter 1, Introducing Advanced Deep Learning with Keras, covers the key concepts of deep learning such as optimization, regularization, loss functions, fundamental layers, and networks and their implementation in Keras. This chapter also serves as a review of both deep learning and Keras using sequential API.
Chapter 2, Deep Neural Networks, discusses the functional API of Keras. Two widely-used deep network architectures, ResNet and DenseNet, are examined and implemented in Keras, using functional API.
Chapter 3, Autoencoders, covers a common network structure called autoencoder that is used to discover the latent representation of the input data. Two example applications of autoencoders, denoising and colorization, are discussed and implemented in Keras.
Chapter 4, Generative Adversarial Networks (GANs), discusses one of the recent significant advances in deep learning. GAN is used to generate new synthetic data that appear real. This chapter explains the principles of GAN. Two examples of GAN, DCGAN and CGAN, are examined and implemented in Keras.
Chapter 5, Improved GANs, covers algorithms that improve the basic GAN. The algorithms address the difficulty in training GANs and improve the perceptual quality of synthetic data. WGAN, LSGAN, and ACGAN are discussed and implemented in Keras.
Chapter 6, Disentangled Representation GANs, discusses how to control the attributes of the synthetic data generated by GANs. The attributes can be controlled if the latent representations are disentangled. Two techniques in disentangling representations, InfoGAN and StackedGAN, are covered and implemented in Keras.
Chapter 7, Cross-Domain GANs, covers a practical application of GANs, translating images from one domain to another or commonly known as cross-domain transfer. CycleGAN, a widely used cross-domain GAN, is discussed and implemented in Keras. This chapter also demonstrates CycleGAN performing colorization and style transfer.
Chapter 8, Variational Autoencoders (VAEs), discusses another recent significant advance in deep learning. Similar to GAN, VAE is a generative model that is used to produce synthetic data. Unlike GAN, VAE focuses on decodable continuous latent space that is suitable for variational inference. VAE and its variations, CVAE and -VAE, are covered and implemented in Keras.
Chapter 9, Deep Reinforcement Learning, explains the principles of reinforcement learning and Q-Learning. Two techniques in implementing Q-Learning for discrete action spaces are presented, Q Table update and Deep Q Network (DQN). Implementation of Q-Learning using Python and DQN in Keras are demonstrated in OpenAI gym environments.
Chapter 10, Policy Gradient Methods, explains how to use neural networks to learn the policy for decision making in reinforcement learning. Four methods are covered and implemented in Keras and OpenAI gym environment, REINFORCE, REINFORCE with Baseline, Actor-Critic, and Advantage Actor-Critic. The example presented in this chapter demonstrates policy gradient methods on a continuous action space.
Get in touch
Feedback from our readers is always welcome.
General feedback: Email <[email protected]>, and mention the book's title in the subject of your message. If you have questions about any aspect of this book, please email us at <[email protected]>.
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book we would be grateful if you would report this to us. Please visit, http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at <[email protected]> with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Implementing the core deep learning models - MLPs, CNNs, and RNNs
We've already mentioned that we'll be using three advanced deep learning models, they are:
These are the three networks that we will be using throughout this book. Despite the three networks being separate, you'll find that they are often combined together in order to take advantage of the strength of each model.
In the following sections of this chapter, we'll discuss these building blocks one by one in more detail. In the following sections, MLPs are covered together with other important topics such as loss function, optimizer, and regularizer. Following on afterward, we'll cover both CNNs and RNNs.
The difference between MLPs, CNNs, and RNNs
Multilayer perceptrons or MLPs are a fully-connected network. You'll often find them referred to as either deep feedforward networks or feedforward neural networks in some literature. Understanding these networks in terms of known target applications will help us get insights about the underlying reasons for the design of the advanced deep learning models. MLPs are common in simple logistic and linear regression problems. However, MLPs are not optimal for processing sequential and multi-dimensional data patterns. By design, MLPs struggle to remember patterns in sequential data and requires a substantial number of parameters to process multi-dimensional data.
For sequential data input, RNNs are popular because the internal design allows the network to discover dependency in the history of data that is useful for prediction. For multi-dimensional data like images and videos, a CNN excels in extracting feature maps for classification, segmentation, generation, and other purposes. In some cases, a CNN in the form of a 1D convolution is also used for networks with sequential input data. However, in most deep learning models, MLPs, RNNs, and CNNs are combined to make the most out of each network.
MLPs, RNNs, and CNNs do not complete the whole picture of deep networks. There is a need to identify an objective or loss function, an optimizer, and a regularizer. The goal is to reduce the loss function value during training since it is a good guide that a model is learning. To minimize this value, the model employs an optimizer. This is an algorithm that determines how weights and biases should be adjusted at each training step. A trained model must work not only on the training data but also on a test or even on unforeseen input data. The role of the regularizer is to ensure that the trained model generalizes to new data.
