27,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
Hidden Markov Model (HMM) is a statistical model based on the Markov chain concept. Hands-On Markov Models with Python helps you get to grips with HMMs and different inference algorithms by working on real-world problems. The hands-on examples explored in the book help you simplify the process flow in machine learning by using Markov model concepts, thereby making it accessible to everyone.
Once you’ve covered the basic concepts of Markov chains, you’ll get insights into Markov processes, models, and types with the help of practical examples. After grasping these fundamentals, you’ll move on to learning about the different algorithms used in inferences and applying them in state and parameter inference. In addition to this, you’ll explore the Bayesian approach of inference and learn how to apply it in HMMs.
In further chapters, you’ll discover how to use HMMs in time series analysis and natural language processing (NLP) using Python. You’ll also learn to apply HMM to image processing using 2D-HMM to segment images. Finally, you’ll understand how to apply HMM for reinforcement learning (RL) with the help of Q-Learning, and use this technique for single-stock and multi-stock algorithmic trading.
By the end of this book, you will have grasped how to build your own Markov and hidden Markov models on complex datasets in order to apply them to projects.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 191
Veröffentlichungsjahr: 2018
Ähnliche
Hands-On Markov Models with Python
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author(s), nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Sunith ShettyAcquisition Editor:Varsha ShettyContent Development Editor:Karan ThakkarTechnical Editor: Sagar SawantCopy Editor: Safis EditingProject Coordinator: Nidhi JoshiProofreader: Safis EditingIndexer: Aishwarya GangawaneGraphics:Jisha ChirayilProduction Coordinator: Shraddha Falebhai
First published: September 2018
Production reference: 1250918
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78862-544-9
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Ankur Ankan is a BTech graduate from IIT (BHU), Varanasi. He is currently working in the field of data science. He is an open source enthusiast and his major work includes starting pgmpy with four other members. In his free time, he likes to participate in Kaggle competitions.
Abinash Panda has been a data scientist for more than 4 years. He has worked at multiple early-stage start-ups and helped them build their data analytics pipelines. He loves to munge, plot, and analyze data. He has been a speaker at Python conferences. These days, he is busy co-founding a start-up. He has contributed to books on probabilistic graphical models by Packt Publishing.
About the reviewer
Abdullah al Mamun is a professional software engineer and researcher. He has completed his graduation from rajshahi university of engineering & technology (RUET) and BSc in Computer Science and Engineering (CSE). Currently, he is working as a Senior Executive Officer of the Software section in Primeasia University. As a professional software engineer, he has experience in object-oriented design, software architectures, design patterns, test-driven development, and project management. Also, he is interested in research in the fields of artificial intelligence, neural network, pattern recognition, and machine learning. His research has been published in different international journals and conferences, including IEEE.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Hands-On Markov Models with Python
Packt Upsell
Why subscribe?
packt.com
Contributors
About the authors
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Introduction to the Markov Process
Random variables
Random processes
Markov processes
Installing Python and packages
Installation on Windows
Installation on Linux
Markov chains or discrete-time Markov processes
Parameterization of Markov chains
Properties of Markov chains
Reducibility
Periodicity
Transience and recurrence
Mean recurrence time
Expected number of visits
Absorbing states
Ergodicity
Steady-state analysis and limiting distributions
Continuous-time Markov chains
Exponential distributions
Poisson process
Continuous-time Markov chain example
Continuous-time Markov chain
Summary
Hidden Markov Models
Markov models
State space models
The HMM
Parameterization of HMM
Generating an observation sequence
Installing Python packages
Evaluation of an HMM
Extensions of HMM
Factorial HMMs
Tree-structured HMM
Summary
State Inference - Predicting the States
State inference in HMM
Dynamic programming
Forward algorithm
Computing the conditional distribution of the hidden state given the observations
Backward algorithm
Forward-backward algorithm (smoothing)
The Viterbi algorithm
Summary
Parameter Learning Using Maximum Likelihood
Maximum likelihood learning
MLE in a coin toss
MLE for normal distributions
MLE for HMMs
Supervised learning
Code
Unsupervised learning
Viterbi learning algorithm
The Baum-Welch algorithm (expectation maximization)
Code
Summary
Parameter Inference Using the Bayesian Approach
Bayesian learning
Selecting the priors
Intractability
Bayesian learning in HMM
Approximating required integrals
Sampling methods
Laplace approximations
Stolke and Omohundro's method
Variational methods
Code
Summary
Time Series Predicting
Stock price prediction using HMM
Collecting stock price data
Features for stock price prediction
Predicting price using HMM
Summary
Natural Language Processing
Part-of-speech tagging
Code
Getting data
Exploring the data
Finding the most frequent tag
Evaluating model accuracy
An HMM-based tagger
Speech recognition
Python packages for speech recognition
Basics of SpeechRecognition
Speech recognition from audio files
Speech recognition using the microphone
Summary
2D HMM for Image Processing
Recap of 1D HMM
2D HMMs
Algorithm
Assumptions for the 2D HMM model
Parameter estimation using EM
Summary
Markov Decision Process
Reinforcement learning
Reward hypothesis
State of the environment and the agent
Components of an agent
The Markov reward process
Bellman equation
MDP
Code example
Summary
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Using Hidden Markov Models (HMMs) is a technique for modeling Markov processes with unobserved states. They are a special case of Dynamic Bayesian Networks (DBNs) but have been found to perform well in a wide range of problems. One of the areas where HMMs are used a lot is speech recognition because HMMs are able to provide a very natural way to model speech data. This book starts by introducing the theoretical aspects of HMMs from the basics of probability theory, and then talks about the different applications of HMMs.
Who this book is for
A basic understanding of probability theory, linear algebra, and calculus will make reading this book a lot easier. For the code examples, basic familiarity with Python programming is expected.
What this book covers
Chapter 1, Introduction to Markov Process, starts with a discussion of basic probability theory, and then introduces Markov chains. The chapter also talks about the different types of Markov chain classifying based on continuous or discrete states and time intervals.
Chapter 2, Hidden Markov Models, builds on the concept of Markov processes and DBNs to introduce the concepts of the HMM.
Chapter 3, State Inference – Predicting the States, introduces algorithms that can be used to predict the states of a defined HMM. The chapter introduces the Forward algorithm, the backward algorithm, the forward-backward algorithm, and the Viterbi algorithm.
Chapter 4, Parameter Inference Using Maximum Likelihood, discusses the basics of maximum likelihood learning. The chapter then moves on to applying maximum likelihood learning in the case of HMMs and introduces the Viterbi learning algorithm and Baum-Welch algorithm.
Chapter 5, Parameter Inference Using Bayesian Approach, starts by introducing the basic concepts of Bayesian learning. The chapter then applies these concepts in the case of HMMs and talks about the different approximation methods used for learning using the Bayesian method.
Chapter 6, Time Series Predicting, discusses the application of HMMs in the case of time series data. The chapter takes the example of the variation of stock prices and tries to model it using an HMM.
Chapter 7, Natural Language Processing, discusses the application of HMMs in the field of speech recognition. The chapter discusses two main areas of application: part-of-speech tagging and speech recognition.
Chapter 8, 2D HMM for Image Processing, introduces the concept of 2D HMMs and discusses their application in the field of image processing.
Chapter 9, Markov Decision Process, introduces the basic concepts of reinforcement learning and then talks about Markov decision process and introduces the Bellman equation to solve them.
To get the most out of this book
You need to have Python 3.4 installed on your machine in order to work through the chapters smoothly.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub athttps://github.com/PacktPublishing/Hands-On-Markov-Models-with-Python. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available athttps://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781788625449_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "Now, let's define the MFCTagger class."
A block of code is set as follows:
from
hmmlearn.hmm
import
GaussianHMM
import
numpy
as
np
import
matplotlib.pyplot
as
plt
Any command-line input or output is written as follows:
pip install matplotlib datetime
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "The possible states of the outcomes are also known as the domain of the random variable."
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
Introduction to the Markov Process
In this chapter, we will develop the basic concepts that we need to understand Hidden Markov Models (HMM). We will cover the following topics:
Random processes
Markov processes
Markov chains or discrete-time Markov processes
Continuous-time Markov chains
Random processes
In the previous section, we discussed random variables that are able to mathematically represent the outcomes of a single random phenomenon. But what if we want to represent these random events over some period of time or the length of an experiment? For example, let's say we want to represent the stock prices for a whole day at intervals of every one hour, or we want to represent the height of a ball at intervals of every one second after being dropped from some height in a vacuum. For such situations, we would need a set of random variables, each of which will represent the outcome at the given instance of time. These sets of random variables that represent random variables over a period of time are also known as random processes.It is worth noting that the domains of all these random variables are the same. Therefore, we can also think of the process as just changing the states.
Now let's discuss the previous two examples in more detail. Starting with the example of dropping the ball from a height in a vacuum, if we know the exact value of gravity and the height from which the ball is being dropped, we will be able to determine the exact location of the ball at every interval of one second using Newton's laws of motion.
Such random processes, in which we can deterministically find the state of each random variable given the initial conditions (in this case, dropping the ball, zero initial velocity) and the parameters of the system (in this case, the value of gravity), are known as deterministic random processes (commonly called deterministic processes).
Now let's go to the second example; representing the stock price over time. In this case, even if we know the current price and the exact probability distribution of the price at the next one hour mark, we won't be able to deterministically compute the value. These random processes, in which we can't determine the state of a process, even if we are given the initial conditions and all the parameters of the system, are known as stochastic random processes (commonly called processes). A very good way of understanding or getting a feel for a stochastic process is to think of it as being the opposite of a deterministic process.
Installing Python and packages
Before moving ahead, we need to set up Python and all the packages required to run the code examples. For all the code examples in this book, we will be using Python 3.4. All the example code in the book is also available on GitHub at https://github.com/PacktPublishing/HandsOnMarkovModelswithPython. We highly recommend using Miniconda to set up your environment for running the examples. Miniconda can be downloaded from https://conda.io/miniconda.html.
Installation on Windows
Miniconda can be installed on a Windows system by just double-clicking on the downloaded .exe file and following the installation instructions. After installation, we will need to create a conda environment and install all the required packages in the environment. To create a new Python 3.4 environment with the name hmm, run the following command:
conda create -n hmm python=3.4
After creating the environment, we will need to activate it and install the required packages in it. This can be done using the following commands:
activate hmm
conda install numpy scipy
