28,14 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Learn how to train popular deep learning architectures such as autoencoders, convolutional and recurrent neural networks while discovering how you can use deep learning models in your software applications with Microsoft Cognitive Toolkit
Key Features
- Understand the fundamentals of Microsoft Cognitive Toolkit and set up the development environment
- Train different types of neural networks using Cognitive Toolkit and deploy it to production
- Evaluate the performance of your models and improve your deep learning skills
Book Description
Cognitive Toolkit is a very popular and recently open sourced deep learning toolkit by Microsoft. Cognitive Toolkit is used to train fast and effective deep learning models. This book will be a quick introduction to using Cognitive Toolkit and will teach you how to train and validate different types of neural networks, such as convolutional and recurrent neural networks.
This book will help you understand the basics of deep learning. You will learn how to use Microsoft Cognitive Toolkit to build deep learning models and discover what makes this framework unique so that you know when to use it. This book will be a quick, no-nonsense introduction to the library and will teach you how to train different types of neural networks, such as convolutional neural networks, recurrent neural networks, autoencoders, and more, using Cognitive Toolkit. Then we will look at two scenarios in which deep learning can be used to enhance human capabilities. The book will also demonstrate how to evaluate your models' performance to ensure it trains and runs smoothly and gives you the most accurate results. Finally, you will get a short overview of how Cognitive Toolkit fits in to a DevOps environment
What you will learn
- Set up your deep learning environment for the Cognitive Toolkit on Windows and Linux
- Pre-process and feed your data into neural networks
- Use neural networks to make effcient predictions and recommendations
- Train and deploy effcient neural networks such as CNN and RNN
- Detect problems in your neural network using TensorBoard
- Integrate Cognitive Toolkit with Azure ML Services for effective deep learning
Who this book is for
Data Scientists, Machine learning developers, AI developers who wish to train and deploy effective deep learning models using Microsoft CNTK will find this book to be useful. Readers need to have experience in Python or similar object-oriented language like C# or Java.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 249
Veröffentlichungsjahr: 2019
Ähnliche
Deep Learning with Microsoft Cognitive Toolkit Quick Start Guide
Copyright © 2019 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor:Amey VarangaonkarAcquisition Editor: Siddharth MandalContent Development Editor:Mohammed Yusuf ImaratwaleTechnical Editor:Jane D'souzaCopy Editor: Safis EditingProject Coordinator:Kinjal BariProofreader: Safis EditingIndexer:Tejal Daruwale SoniGraphics:Alishon MendonsaProduction Coordinator:Jisha Chirayil
First published: March 2019
Production reference: 1260319
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78980-299-3
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Willem Meints is a software architect and engineer with a wide variety of interests. His background in software engineering hasn't stopped him from exploring new areas, such as machine learning, as part of his daily work. This sparked a deep passion for everything related to artificial intelligence and deep learning.
Willem studied electronics after his high school career, but quickly discovered he had more fun building applications. This led to his decision to leave the world of electronics and launch a career in software engineering. After he finished his bachelor's degree in software engineering, he started working for Info Support, where he's been working ever since.
About the reviewer
Bahrudin Hrnjica holds a PhD in technical science from the University of Bihać. Currently, he is assistant professor at the university, teaching students in the fields of numerical analysis, mathematical modeling, and machine learning. Besides teaching at the university, he has many years' experience in the software industry, working on custom solutions based on cloud technologies, machine learning, .NET, and Visual Studio. As an expert in development technologies, Microsoft recognized him as Microsoft Most Valuable Professional (Microsoft MVP) for the first time in 2011. He is an author of several books, online articles, and open source projects, as well as having spoken at many local and regional conferences, code camps, and workshops.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Deep Learning with Microsoft Cognitive Toolkit Quick Start Guide
Dedication
About Packt
Why subscribe?
Packt.com
Contributors
About the author
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Code in Action
Conventions used
Get in touch
Reviews
Getting Started with CNTK
The relationship between AI, machine learning, and deep learning
Limitations of machine learning
How does deep learning work?
The neural network architecture
Artificial neurons
Predicting output with a neural network
Optimizing a neural network
What is CNTK?
Features of CNTK
A high-speed low-level API
Basic building blocks for quickly creating neural networks
Measuring model performance
Loading and processing large datasets
Using models from C# and Java
Installing CNTK
Installing on Windows
Installing Anaconda
Upgrading pip
Installing CNTK
Installing on Linux
Installing Anaconda
Upgrading pip to the latest version
Installing the CNTK package
Using your GPU with CNTK
Enabling GPU usage on Windows
Enabling GPU usage on Linux
Summary
Building Neural Networks with CNTK
Technical requirements
Basic neural network concepts in CNTK
Building neural networks using layer functions
Customizing layer settings
Using learners and trainers to optimize the parameters in a neural network
Loss functions
Model metrics
Building your first neural network
Building the network structure
Choosing an activation function
Choosing an activation function for the output layer
Choosing an activation function for the hidden layers
Picking a loss function
Recording metrics
Training the neural network
Choosing a learner and setting up training
Feeding data into the trainer to optimize the neural network
Checking the performance of the neural network
Making predictions with a neural network
Improving the model
Summary
Getting Data into Your Neural Network
Technical requirements
Training a neural network efficiently with minibatches
Working with small in-memory datasets
Working with numpy arrays
Working with pandas DataFrames
Working with large datasets
Creating a MinibatchSource instance
Creating CTF files
Feeding data into a training session
Taking control over the minibatch loop
Summary
Validating Model Performance
Technical requirements
Choosing a good strategy to validate model performance
Using a hold-out dataset for validation
Using k-fold cross-validation
What about underfitting and overfitting?
Validating performance of a classification model
Using a confusion matrix to validate your classification model
Using the F-measure as an alternative to the confusion matrix
Measuring classification performance in CNTK
Validating performance of a regression model
Measuring the accuracy of your predictions
Measuring regression model performance in CNTK
Measuring performance for out-of-memory datasets
Measuring performance when working with minibatch sources
Measuring performance when working with a manual minibatch loop
Monitoring your model
Using callbacks during training and validation
Using ProgressPrinter
Using TensorBoard
Summary
Working with Images
Technical requirements
Convolutional neural network architecture
Network architecture used for image classification
Working with convolution layers
Working with pooling layers
Other uses for convolutional networks
Building convolutional networks
Building the network structure
Training the network with images
Picking the right combination of layers
Improving model performance with data augmentation
Summary
Working with Time Series Data
Technical requirements
What are recurrent neural networks?
Recurrent neural networks variations
Predicting a single output based on a sequence
Predicting a sequence based on a single sample
Predicting sequences based on sequences
Stacking multiple recurrent layers
How do recurrent neural networks work?
Making predictions with a recurrent neural network
Training a recurrent neural network
Using other recurrent layer types
Working with gated recurrent units
Working with long short-term memory units
When to use other recurrent layer types
Building recurrent neural networks with CNTK
Building the neural network structure
Stacking multiple recurrent layers
Training the neural network with time series data
Predicting output 
Summary
Deploying Models to Production
Technical requirements
Using machine learning in a DevOps environment
Keeping track of your data
Training models in a continuous integration pipeline
Deploying models to production
Gathering feedback on your models
Storing your models
Storing model checkpoints to continue training at a later point
Storing portable models for use in other applications
Storing a model in ONNX format
Using ONNX models in C#
Using Azure Machine Learning service to manage models
Deploying Azure Machine Learning service
Exploring the machine learning workspace
Running your first experiment
Deploying your model to production
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Artificial intelligence (AI) is here to enhance humans by automating some of the tasks we do every day, so we can spend more time fully realizing our potential. We've been using software programs as tools to automate many of the simpler tasks. Now it is time to take on the challenge of automating more complicated tasks.
There's a lot happening in the area of AI and more people than ever are looking to expand their existing toolkit with new techniques to make their software smarter. Machine learning, and especially deep learning, are highly important tools with which we can enhance what we are already doing with our computers.
This book aims to help you get to grips with one of the most popular deep learning tools, CNTK. We will look at what this relatively young open source deep learning framework offers. At the end of this book, you'll have a solid understanding of the framework and some of the scenarios in which it can be used.
Who this book is for
This book is great for developers with some experience in Java, C#, or Python. We're assuming you're pretty new to machine learning. However, this book is also great for people who have worked with other deep learning frameworks before and want to learn another great deep learning tool.
What this book covers
Chapter 1, Getting Started with CNTK, introduces you to the CNTK framework and the world of deep learning. It explains how to install the tools on your computer and how to use a GPU with CNTK.
Chapter 2, Building Neural Networks with CNTK, explains how to build your first neural network with CNTK. We dive into the basic building blocks and see how to train a neural network with CNTK.
Chapter 3, Getting Data into Your Neural Network, shows you different methods of loading data for training neural networks. You'll learn how to work with both small datasets, and datasets that don't fit in your computer's memory.
Chapter 4, Validating Model Performance, teaches you how to work with metrics to validate the performance of your neural network. You'll learn how to validate regression models and classification models and what to look for when trying to debug your neural network.
Chapter 5, Working with Images, explains how to use convolutional neural networks to classify images. We'll show you the building blocks needed to work with spatially-ordered data. We'll also show you some of the most well-known neural network architectures for working with images.
Chapter 6, Working with Time Series Data, teaches you how to use recurrent neural networks to build models that can reason over time. We'll explain the various building blocks that you need to build and validate a recurrent neural network yourself, based on a IoT sample.
Chapter 7, Deploying Models to Production, shows you what it takes to deploy deep learning models to production. We'll take a look at a DevOps environment with a continuous integration/continuous deployment (CI/CD) pipeline to teach you what it takes to train and deploy models in an agile engineering environment. We'll show you how you can use a tool such as Azure Machine Learning service to take your machine learning efforts to the next level.
To get the most out of this book
We recommend you have experience with Python 3 so that you know what the syntax looks like. You will need to run either Linux or Windows on a machine with a decent amount of memory and CPU power, as the samples in this book can take a long time to run on an older machine. If you are lucky enough to have a gaming graphics card in your machine from NVIDIA, we definitely recommend looking at the instructions on how to install the GPU version of CNTK, as this can speed up the samples by quite a large factor. Some sections in the book assume that you know a little bit about Java or C#. Although not required, it is useful to have a basic understanding of the syntax of one or more of these languages.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Deep-Learning-with-Microsoft-Cognitive-Toolkit-Quick-Start-Guide. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Code in Action
Visit the following link to check out videos of the code being run:http://bit.ly/2UcIfSe
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
Getting Started with CNTK
Deep learning is a machine learning technique that is getting a lot of attention from the public and researchers. In this chapter, we will explore what deep learning is and how large companies are using it to solve complex problems. We'll look at what makes this technique so exciting and what concepts drive deep learning.
We will then talk about Microsoft Cognitive Toolkit (CNTK), what it is, and how it fits into the bigger picture of deep learning. We'll also discuss what makes CNTK unique compared to other frameworks.
In this chapter, we'll also show you how to get CNTK installed on your computer. We will explore installation on both Windows and Linux. If you have a compatible graphics card, you'll also want to check out the instructions on how to configure your graphics card for use with CNTK, as it will significantly speed up the calculations that are needed to train deep learning models.
In this chapter we will cover the following topics:
The relationship between AI, machine learning, and deep learning
How does deep learning work?
What is CNTK?
Installing CNTK
The relationship between AI, machine learning, and deep learning
In order to understand what deep learning is, we have to explore what Artificial Intelligence (AI) is and how it relates to machine learning and deep learning. Conceptually, deep learning is a form of machine learning, whilst machine learning is a form of AI:
In computer science, Artificial intelligence, is a form of intelligence demonstrated by machines. AI is a term that was invented in the 1950s by scientists doing research in computer science. AI encompasses a large set of algorithms that shows behavior that is more intelligent than the standard software we build for our computers.
Some algorithms demonstrate intelligent behavior but aren't capable of improving themselves. One group of algorithms, called machine learning algorithms, can learn from sample data that you show them and generate models that you then use on similar data to make predictions.
Within the group of machine learning algorithms there's the sub-category of deep learning algorithms. This group of algorithms uses models that are inspired by the structure and function of a biological brain found in humans or animals.
Both machine learning and deep learning learn from sample data that you provide. When we build regular programs, we write business rules by using different language constructs, such as if-statements, loops, and functions. The rules are fixed. In machine learning, we feed samples and an expected answer into an algorithm that then learns the rules that connect the samples to the expected answers:
There are two major components in machine learning: machine learning models and machine learning algorithms.
When you use machine learning to build a program, you first choose a machine learning model. A machine learning model is a mathematical equation containing trainable parameters that transforms input into a predicted answer. This model shapes the rules that the computer will learn. For example: predicting the miles per gallon for a car requires that you model reality in a certain way. Classifying whether a credit card transaction is fraudulent requires a different model.
The representation of the input could be the properties of a car turned into a vector. The output of the model could be the miles per gallon for a car. In the case of credit card fraud, the input could be the properties of the user account and the transaction that was done. The output representation could be a score between 0 and 1 where a value close to 1 means that the transaction should be rejected.
The mathematical transformation in the machine learning model is controlled by a set of parameters that need to be trained for the transformation to produce the correct output representation.
This is where the second part, the machine learning algorithm comes into play. To find the best values for the parameters in the machine learning model we need to perform a multi-step process:
Initially, the computer will choose a random value for each of the unknown parameters in your model
It will then use sample data to make an initial prediction
This prediction is fed into a
loss
function together with the expected output to get feedback regarding how well the model is performing
This feedback is then used by the machine learning algorithm to find better values for the parameters in the model
These steps are repeated many times to find the best possible values for the parameters in the model. If all goes well, you end up with a model that is capable of making accurate predictions for many complicated situations.
The fact that we can learn rules from examples is a useful concept. There are many situations where we can't use simple rules to solve a particular problem. For example: credit card fraud cases come in many shapes and sizes. Sometimes a hacker slowly breaks the system injecting smaller hacks over time and then stealing the money. Other times hackers simply try to steal a lot of money in one attempt. A rule-based program would become too hard to maintain because it would need to contain a lot of code to handle all different fraud cases. Machine learning is an elegant way to solve this problem. It understands how to handle different kinds of credit card fraud without a lot of code. And it is also capable of making a judgment on cases that it hasn't seen before, within reasonable boundaries.
Limitations of machine learning
Machine learning models are very powerful. You can use them in many cases where rule-based programs fall short. Machine learning is a good first alternative whenever you find a problem that can't be solved with a regular rule-based program. Machine learning models do, however, come with their limitations.
The mathematical transformation in machine learning models is very basic. For example: when you want to classify whether a credit transaction should be marked as fraud, you can use a linear model. A logistic regression model is a great model for this kind of use case; it creates a decision boundary function that separates fraud cases from non-fraud cases. Most of the fraud cases will be above the line and correctly marked as such. But no machine learning model is perfect and some of the cases will not be correctly marked as fraud by the model as you can see in the following image.
If your data happens to be perfectly linearly-separable all cases would be correctly classified by the model. But when have to deal with more complex types of data, the basic machine learning models fall short. And there are more reasons why machine learning is limited in what it can do:
Many algorithms assume that there's no interaction between features in the input
Machine learning are, in many cases, based on linear algorithms, that don't handle non-linearity very well
Often, you are dealing with a lot of features, classic machine learning algorithms have a harder time to deal with high dimensionality in the input data
How does deep learning work?
The limitations discovered in machine learning caused scientists to look for other ways to build more complex models that allowed them to handle non-linear relationships and cases where there's a lot of interaction between the input of a model. This led to the invention of the artificial neural network.
An artificial neural network is a graph composed of several layers of artificial neurons. It's inspired by how the structure and function of the biological brain found in humans and animals.
To understand the power of deep learning and how to use CNTK to build neural networks, we need to look at how a neural network works and how it is trained to detect patterns in samples you feed it.
The neural network architecture
A neural network is made out of different layers. Each layer contains multiple neurons.
A typical neural network is made of several layers of artificial neurons. The first layer in a neural network is called the input layer. This is where we feed input into the neural network. The last layer of a neural network is called the output layer. This is where the transformed data is coming out of the neural network. The output of a neural network represents the prediction made by the network.
In between the input and output layer of the neural network, you can find one or more hidden layers. The layers in between the input and output are hidden because we don't typically observe the data going through these layers.
Neural networks are mathematical constructs. The data passed through a neural network is encoded as floating-point numbers. This means that everything you want to process with a neural network has to be encoded as vectors of floating-point numbers.
Artificial neurons
The core of a neural network is the artificial neuron. The artificial neuron is the smallest unit in a neural network that we can train to recognize patterns in data. Each artificial neuron inside the neural network has one or more input. Each of the vector input gets a weight:
The artificial neuron inside a neural network works in much the same way, but doesn't use chemical signals. Each artificial neuron inside the neural network has one or more inputs. Each of the vector inputs gets a weight.
The numbers provided for each input of the neuron gets multiplied by this weight. The output of this multiplication is then added up to produce a total activation value for the neuron.
This activation signal is then passed through an activation function. The activation function performs a non-linear transformation on this signal. For example: it uses a rectified linear function to process the input signal:
The rectified linear function will convert negative activation signals to zero but performs an identity (pass-through) transformation on the signal when it is a positive number.
One other popular activation function is the sigmoid function. It behaves slightly different than the rectified linear function in that it transforms negative values to 0 and positive values to 1. There is, however, a slope in the activation between -0.5 and +0.5, where the signal is transformed in a linear fashion.
Activation functions in artificial neurons play an important role in the neural network. It's because of these non-linear transformation functions that the neural network is capable of working with non-linear relationships in the data.
Predicting output with a neural network
By combining layers of neurons together we create a stacked function that has non-linear transformations and trainable weights so it can learn to recognize complex relationships. To visualize this, let's transform the neural network from previous sections into a mathematical formula. First, let's take a look at the formula for a single layer:
The X variable is a vector that represents the input for the layer in the neural network. The w parameter represents a vector of weights for each of the elements in the input vector, X. In many neural network implementations, an additional term, b, is added, this is called the bias and basically increases or decreases the overall level of input required to activate the neuron. Finally, there's a function, f, which is the activation function for the layer.
Now that you've seen the formula for a single layer, let's put together additional layers to create the formula for the neural network:
Notice how the formula has changed. We now have the formula for the first layer wrapped in another layer function. This wrapping or stacking of functions continues when we add more layers to the neural network. Each layer introduces more parameters that need to be optimized to train the neural network. It also allows the neural network to learn more complex relationships from the data we feed into it.
To make a prediction with a neural network, we need to fill all of the parameters in the neural network. Let's assume we know those because we trained it before. What's left is the input value for the neural network.
