42,22 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Updated for OpenCV 4 and Python 3, this book covers the latest on depth cameras, 3D tracking, augmented reality, and deep neural networks, helping you solve real-world computer vision problems with practical code
Key Features
- Build powerful computer vision applications in concise code with OpenCV 4 and Python 3
- Learn the fundamental concepts of image processing, object classification, and 2D and 3D tracking
- Train, use, and understand machine learning models such as Support Vector Machines (SVMs) and neural networks
Book Description
Computer vision is a rapidly evolving science, encompassing diverse applications and techniques. This book will not only help those who are getting started with computer vision but also experts in the domain. You'll be able to put theory into practice by building apps with OpenCV 4 and Python 3.
You'll start by understanding OpenCV 4 and how to set it up with Python 3 on various platforms. Next, you'll learn how to perform basic operations such as reading, writing, manipulating, and displaying still images, videos, and camera feeds. From taking you through image processing, video analysis, and depth estimation and segmentation, to helping you gain practice by building a GUI app, this book ensures you'll have opportunities for hands-on activities. Next, you'll tackle two popular challenges: face detection and face recognition. You'll also learn about object classification and machine learning concepts, which will enable you to create and use object detectors and classifiers, and even track objects in movies or video camera feed. Later, you'll develop your skills in 3D tracking and augmented reality. Finally, you'll cover ANNs and DNNs, learning how to develop apps for recognizing handwritten digits and classifying a person's gender and age.
By the end of this book, you'll have the skills you need to execute real-world computer vision projects.
What you will learn
- Install and familiarize yourself with OpenCV 4's Python 3 bindings
- Understand image processing and video analysis basics
- Use a depth camera to distinguish foreground and background regions
- Detect and identify objects, and track their motion in videos
- Train and use your own models to match images and classify objects
- Detect and recognize faces, and classify their gender and age
- Build an augmented reality application to track an image in 3D
- Work with machine learning models, including SVMs, artificial neural networks (ANNs), and deep neural networks (DNNs)
Who this book is for
If you are interested in learning computer vision, machine learning, and OpenCV in the context of practical real-world applications, then this book is for you. This OpenCV book will also be useful for anyone getting started with computer vision as well as experts who want to stay up-to-date with OpenCV 4 and Python 3. Although no prior knowledge of image processing, computer vision or machine learning is required, familiarity with basic Python programming is a must.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 445
Veröffentlichungsjahr: 2020
Ähnliche
Learning OpenCV 4 Computer Vision with Python 3 Third Edition
Copyright © 2020 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Richa TripathiAcquisition Editor:Alok DhuriContent Development Editor:Digvijay BagulSenior Editor: Rohit SinghTechnical Editor: Ketan KambleCopy Editor: Safis EditingProject Coordinator:Francy PuthiryProofreader: Safis EditingIndexer:Pratik ShirodkarProduction Coordinator:Shraddha Falebhai
First published: April 2013 Second edition: September 2015 Third edition: February 2020
Production reference: 1190220
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78953-161-9
www.packt.com
Packt.com
Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Fully searchable for easy access to vital information
Copy and paste, print, and bookmark content
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the authors
Joseph Howse lives in a Canadian fishing village with four cats; the cats like fish, but they prefer chicken.
Joseph provides computer vision expertise through his company, Nummist Media. His books include OpenCV 4 for Secret Agents, Learning OpenCV 4 Computer Vision with Python 3, OpenCV 3 Blueprints, Android Application Programming with OpenCV 3, iOS Application Development with OpenCV 3, and Python Game Programming by Example, published by Packt.
Joe Minichino is an R&D labs engineer at Teamwork. He is a passionate programmer who is immensely curious about programming languages and technologies and constantly experimenting with them. Born and raised in Varese, Lombardy, Italy, and coming from a humanistic background in philosophy (at Milan's Università Statale), Joe has lived in Cork, Ireland, since 2004. There, he became a computer science graduate at the Cork Institute of Technology.
About the reviewer
Sri Manikanta Palakollu is an undergraduate student pursuing his bachelor's degree in computer science and engineering at SICET under JNTUH. He is a founder of the Open Stack Developer Community in his college. He started his journey as a competitive programmer. He loves to solve problems related to the data science field. His interests include data science, app development, web development, cyber security, and technical writing. He has published many articles on data science, machine learning, programming, and cyber security with publications such as Hacker Noon, freeCodeCamp, Towards Data Science, and DDI.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Learning OpenCV 4 Computer Vision with Python 3 Third Edition
Dedication
About Packt
Why subscribe?
Contributors
About the authors
About the reviewer
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Code in Action
Download the color images
Conventions used
Get in touch
Reviews
Setting Up OpenCV
Technical requirements
What's new in OpenCV 4
Choosing and using the right setup tools
Installation on Windows
Using a ready-made OpenCV package
Building OpenCV from source
Installation on macOS
Using Homebrew with ready-made packages
Using Homebrew with your own custom packages
Installation on Debian, Ubuntu, Linux Mint, and similar systems
Using a ready-made OpenCV package
Building OpenCV from source
Installation on other Unix-like systems
Running samples
Finding documentation, help, and updates
Summary
Handling Files, Cameras, and GUIs
Technical requirements
Basic I/O scripts
Reading/writing an image file
Converting between an image and raw bytes
Accessing image data with numpy.array
Reading/writing a video file
Capturing camera frames
Displaying an image in a window
Displaying camera frames in a window
Project Cameo (face tracking and image manipulation)
Cameo – an object-oriented design
Abstracting a video stream with managers.CaptureManager
Abstracting a window and keyboard with managers.WindowManager
Applying everything with cameo.Cameo
Summary
Processing Images with OpenCV
Technical requirements
Converting images between different color models
Light is not paint 
Exploring the Fourier transform
HPFs and LPFs
Creating modules
Edge detection
Custom kernels – getting convoluted
Modifying the application
Edge detection with Canny
Contour detection
Bounding box, minimum area rectangle, and minimum enclosing circle
Convex contours and the Douglas-Peucker algorithm
Detecting lines, circles, and other shapes
Detecting lines
Detecting circles
Detecting other shapes
Summary
Depth Estimation and Segmentation
Technical requirements
Creating modules
Capturing frames from a depth camera
Converting 10-bit images to 8-bit
Creating a mask from a disparity map
Modifying the application
Depth estimation with a normal camera
Foreground detection with the GrabCut algorithm
Image segmentation with the Watershed algorithm
Summary
Detecting and Recognizing Faces
Technical requirements
Conceptualizing Haar cascades
Getting Haar cascade data
Using OpenCV to perform face detection
Performing face detection on a still image
Performing face detection on a video
Performing face recognition
Generating the data for face recognition
Recognizing faces
Loading the training data for face recognition
Performing face recognition with Eigenfaces
Performing face recognition with Fisherfaces
Performing face recognition with LBPH
Discarding results based on the confidence score
Swapping faces in the infrared
Modifying the application's loop
Masking a copy operation
Summary
Retrieving Images and Searching Using Image Descriptors
Technical requirements
Understanding types of feature detection and matching
Defining features
Detecting Harris corners
Detecting DoG features and extracting SIFT descriptors
Anatomy of a keypoint
Detecting Fast Hessian features and extracting SURF descriptors
Using ORB with FAST features and BRIEF descriptors
FAST
BRIEF
Brute-force matching
Matching a logo in two images
Filtering matches using K-Nearest Neighbors and the ratio test
Matching with FLANN
Performing homography with FLANN-based matches
A sample application – tattoo forensics
Saving image descriptors to file
Scanning for matches
Summary
Building Custom Object Detectors
Technical requirements
Understanding HOG descriptors
Visualizing HOG
Using HOG to describe regions of an image
Understanding NMS
Understanding SVMs
Detecting people with HOG descriptors
Creating and training an object detector
Understanding BoW
Applying BoW to computer vision
k-means clustering
Detecting cars
Combining an SVM with a sliding window
Detecting a car in a scene
Saving and loading a trained SVM
Summary
Tracking Objects
Technical requirements
Detecting moving objects with background subtraction
Implementing a basic background subtractor
Using a MOG background subtractor
Using a KNN background subtractor
Using GMG and other background subtractors
Tracking colorful objects using MeanShift and CamShift
Planning our MeanShift sample
Calculating and back-projecting color histograms
Understanding the parameters of cv2.calcHist
Understanding the parameters of cv2.calcBackProject
Implementing the MeanShift example
Using CamShift
Finding trends in motion using the Kalman filter
Understanding the predict and update phases
Tracking a mouse cursor
Tracking pedestrians
Planning the flow of the application
Comparing the object-oriented and functional paradigms
Implementing the Pedestrian class
Implementing the main function
Considering the next steps
Summary
Camera Models and Augmented Reality
Technical requirements
Understanding 3D image tracking and augmented reality
Understanding camera and lens parameters
Understanding cv2.solvePnPRansac
Implementing the demo application
Importing modules
Performing grayscale conversion
Performing 2D-to-3D spatial conversions
Implementing the application class
Initializing the tracker
Implementing the main loop
Tracking the image in 3D
Initializing and applying the Kalman filter
Drawing the tracking results and masking the tracked object
Running and testing the application
Improving the 3D tracking algorithm
Summary
Introduction to Neural Networks with OpenCV
Technical requirements
Understanding ANNs
Understanding neurons and perceptrons
Understanding the layers of a neural network
Choosing the size of the input layer
Choosing the size of the output layer
Choosing the size of the hidden layer
Training a basic ANN in OpenCV
Training an ANN classifier in multiple epochs
Recognizing handwritten digits with an ANN
Understanding the MNIST database of handwritten digits
Choosing training parameters for the MNIST database
Implementing a module to train the ANN
Implementing a minimal test module
Implementing the main module
Trying to improve the ANN's training
Finding other potential applications
Using DNNs from other frameworks in OpenCV
Detecting and classifying objects with third-party DNNs
Detecting and classifying faces with third-party DNNs
Summary
Appendix A: Bending Color Space with the Curves Filter
Formulating a curve
Caching and applying a curve
Designing object-oriented curve filters
Emulating photo films
Emulating Kodak Portra
Emulating Fuji Provia
Emulating Fuji Velvia
Emulating cross-processing
Summary
Other Book You May Enjoy
Leave a review - let other readers know what you think
Preface
Now in its third edition, this is the original book on OpenCV's Python bindings. Readers will learn a great range of techniques and algorithms, from the classics to the state-of-the-art, and from geometry to machine learning. All of this is in aid of solving practical computer vision problems in well-built applications. Using OpenCV 4 and Python 3, we adopt an approach that is accessible to computer vision novices, yet also informative for experts who want to expand and update their skills.
We start with an introduction to OpenCV 4 and explain how to set it up with Python 3 on various platforms. Next, you'll learn how to perform basic operations such as reading, writing, manipulating, and displaying still images, videos, and camera feeds. You'll learn about image processing and video analysis, along with depth estimation and segmentation, and you'll gain practice by building a simple GUI application. Next, you'll tackle two popular problems: face detection and face recognition.
As we advance, we'll explore concepts of object classification and machine learning, enabling you to create and use object detectors and classifiers, and even track objects in movies or video camera feeds. Then, we'll extend our work into 3D tracking and augmented reality. Finally, we'll learn about artificial neural networks (ANNs) and deep neural networks (DNNs) as we develop applications to recognize handwritten digits, and to classify a person's gender and age.
By the end of this book, you will have acquired the right knowledge and skills to embark on your own real-world computer vision projects.
Who this book is for
This book is intended for people interested in learning computer vision, machine learning, and OpenCV in the context of practical real-world applications. The book will appeal to computer vision novices as well as experts who want to get up to date with OpenCV 4 and Python 3. Readers should be familiar with basic Python programming, but no prior knowledge of image processing, computer vision, or machine learning is required.
What this book covers
Chapter 1, Setting Up OpenCV, explains how to set up OpenCV 4 with Python 3 on various platforms. It also provides troubleshooting steps for common problems.
Chapter 2, Handling Files, Cameras, and GUIs, introduces OpenCV's I/O functionalities. It also discusses an object-oriented design for a GUI project that we will develop further in other chapters.
Chapter 3, Processing Images with OpenCV, presents some techniques required to alter images, such as manipulating colors, sharpening an image, marking contours of objects, and detecting geometric shapes.
Chapter 4, Depth Estimation and Segmentation, shows you how to use data from a depth camera to identify foreground and background regions, such that we can limit an effect to only the foreground or background.
Chapter 5, Detecting and Recognizing Faces, introduces some of OpenCV's functionality for facedetection and recognition, along with the data files that define particular typesof detectable objects.
Chapter 6, Retrieving Images and Searching Using Image Descriptors, shows how to describe the features of an image with the help of OpenCV, and how to make use of features to match and search for images.
Chapter 7, Building Custom Object Detectors, applies a combination of computer vision and machine learning algorithms to locate and classify objects in an image. It shows how to implement this combination of algorithms with OpenCV.
Chapter 8, Tracking Objects, demonstrates ways to track and predict the motion of people and objects in a video or live camera feed.
Chapter 9, Camera Models and Augmented Reality, enables you to build an augmented reality application that uses information about cameras, objects, and motion to superimpose 3D graphics atop tracked objects in real time.
Chapter 10, Introduction to Neural Networks with OpenCV, introduces you to artificial neural networks (ANNs) and deep neural networks (DNNs) in OpenCV, and illustrates their usage in real-world applications.
Appendix A, Bending Color Space with a Curves Filter, describes the concept of color curves and our implementation of them using SciPy.
To get the most out of this book
The reader is expected to have at least basic proficiency in the Python programming language.
A Windows, macOS, or Linux development machine is recommended. You can refer to Chapter 1, Setting Up OpenCV, for instructions about setting up OpenCV 4, Python 3, and other dependencies.
This book takes a hands-on approach to learning and includes 77 sample scripts, along with sample data. Working through these examples as you read the book will help enforce the concepts.
The code for this book is released under the BSD 3-Clause open source license, which is the same as the license used by OpenCV itself. The reader is encouraged to use, modify, improve, and even publish their changes to these example programs.
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
.
Select the
Support
tab.
Click on
Code Downloads
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/Learning-OpenCV-4-Computer-Vision-with-Python-Third-Edition. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Code in Action
Code in Action videos for this book can be viewed at http://bit.ly/2STXnRN.
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packt-cdn.com/downloads/9781789531619_ColorImages.pdf.
Get in touch
Feedback from our readers is always welcome.
General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support, selecting your book, clicking on the Errata Submission Form link, and entering the details. Also, if you encounter a problem with the code from the book's GitHub repository, you can file an issue report at https://github.com/PacktPublishing/Learning-OpenCV-4-Computer-Vision-with-Python-Third-Edition/issues.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packt.com.
Setting Up OpenCV
You've picked up this book, so you may already have an idea of what OpenCV is. Maybe you heard of capabilities that seem to come straight out of science fiction, such as training an artificial intelligence model to recognize anything that it sees through a camera. If this is your interest, you will not be disappointed! OpenCV stands for Open Source Computer Vision. It is a free computer vision library that allows you to manipulate images and videos to accomplish a variety of tasks, from displaying frames from a webcam to teaching a robot to recognize real-life objects.
In this book, you will learn to leverage the immense potential of OpenCV with the Python programming language. Python is an elegant language with a relatively shallow learning curve and very powerful features. This chapter is a quick guide to setting up Python 3, OpenCV 4, and other dependencies. As part of OpenCV, we will set up the opencv_contrib modules, which offer additional functionality that is maintained by the OpenCV community rather than the core development team. After setup, we will also look at OpenCV's Python sample scripts and documentation.
The following related libraries are covered in this chapter:
NumPy
: This library is a dependency of OpenCV's Python bindings. It provides numeric computing functionality, including efficient arrays.
SciPy
: This library is a scientific computing library that is closely related to NumPy. It is not required by OpenCV, but it is useful if you wish to manipulate data in OpenCV images.
OpenNI 2
: This library is an optional dependency of OpenCV. It adds support for certain depth cameras, such as the Asus Xtion PRO.
For this book's purposes, OpenNI 2 can be considered optional. It is used throughout Chapter 4, Depth Estimation and Segmentation, but is not used in the other chapters or appendices.
We will cover the following topics in this chapter:
What's new in OpenCV 4
Choosing and using the right setup tools
Running samples
Finding documentation, help, and updates
Technical requirements
This chapter assumes that you are using one of the following operating systems:
Windows 7 SP1 or a later version
macOS 10.7 (Lion) or a later version
Debian Jessie or a later version, or a derivative such as the following:
Ubuntu 14.04 or a later version
Linux Mint 17 or a later version
For editing Python scripts and other text files, this book's authors simply recommend that you should have a good text editor. Examples include the following:
Notepad++ for Windows
BBEdit (free version) for macOS
GEdit for the GNOME desktop environment on Linux
Kate for the KDE Plasma desktop environment on Linux
Besides the operating system, there are no other prerequisites for this setup chapter.
What's new in OpenCV 4
If you are an OpenCV veteran, you might want to know more about OpenCV 4's changes before you decide to install it. Here are some of the highlights:
The C++ implementation of OpenCV has been updated to C++11. OpenCV's Python bindings wrap the C++ implementation, so as Python users, we may gain some performance advantages from this update, even though we are not using C++ directly.
The deprecated C implementation of OpenCV and the deprecated Python bindings for the C implementation have been removed.
Many new optimizations have been implemented. Existing OpenCV 3 projects can take advantage of many of these optimizations without further changes beyond updating the OpenCV version. For OpenCV C++ projects, an entirely new optimization pipeline named G-API is available; however, OpenCV's Python bindings currently do not support this optimization pipeline.
Many new machine learning models are available in OpenCV's DNN module.
The tools to train Haar cascades and LBP cascades (to detect custom objects) have been removed. There is a proposal to reimplement these tools, along with support for additional models, in a future update for OpenCV 4.
The KinectFusion algorithm (for three-dimensional reconstruction using a Microsoft Kinect 2 camera) is now supported.
The DIS algorithm for dense optical flow has been added.
A new module has been added for detecting and decoding QR codes.
Whether or not you have used a previous version of OpenCV, this book will serve you as a general guide to OpenCV 4, and some of the new features will receive special attention in subsequent chapters.
Choosing and using the right setup tools
We are free to choose various setup tools, depending on our operating system and how much configuration we want to do.
Regardless of the choice of operating system, Python offers some built-in tools that are useful for setting up a development environment. These tools include a package manager called pip and a virtual environment manager called venv. Some of this chapter's instructions will cover pip specifically, but if you would like to learn about venv, please refer to the official Python documentation at https://docs.python.org/3/library/venv.html.
Let's take an overview of the setup tools available for Windows, macOS, Ubuntu, and other Unix-like systems.
Installation on Windows
Windows does not come with Python preinstalled. However, an installation wizard is available for Python, and Python provides a package manager called pip, which lets us easily install ready-made builds of NumPy, SciPy, and OpenCV. Alternatively, we can build OpenCV from source in order to enable nonstandard features, such as support for depth cameras via OpenNI 2. OpenCV's build system uses CMake for configuring the system and Visual Studio for compilation.
Before anything else, let's install Python. Go to https://www.python.org/getit/ and download and run the most recent installer for Python 3.8. You probably want an installer for 64-bit Python, though OpenCV can work with 32-bit Python too.
Once Python has been installed, we can use pip to install NumPy and SciPy. Open the Command Prompt and run the following command:
> pip install numpy scipy
Now, we must decide whether we want a ready-made build of OpenCV (without support for depth cameras) or a custom build (with support for depth cameras). The next two subsections cover these alternatives.
Using a ready-made OpenCV package
OpenCV, including the opencv_contrib modules, can be installed as a pip package. This is as simple as running the following command:
> pip install opencv-contrib-python
If you want your OpenCV installation to include non-free content, such as patented algorithms, then you can run the following command instead:
> pip install opencv-contrib-python-nonfree
You might find that one of these pip packages offers all the OpenCV features you currently want. On the other hand, if you intend to use depth cameras, or if you want to learn about the general process of making a custom build of OpenCV, you should not install the OpenCV pip package; you should proceed to the next subsection instead.
Building OpenCV from source
If you want support for depth cameras, you should also install OpenNI 2, which is available as a set of precompiled binaries with an installation wizard. Then, we must build OpenCV from source using CMake and Visual Studio.
To obtain OpenNI 2, go to https://structure.io/openni and download the latest ZIP for Windows and for your system's architecture (x64 or x86). Unzip it to get an installer file, such as OpenNI-Windows-x64-2.2.msi. Run the installer.
Now, let's set up Visual Studio. To build OpenCV 4, we need Visual Studio 2015 or a later version. If you do not already have a suitable version, go to https://visualstudio.microsoft.com/downloads/ and download and run one of the installers for one of the following:
The free Visual Studio 2019 Community edition
Any of the paid Visual Studio 2019 editions, which have a 30-day trial period
During installation, ensure that any optional C++ components are selected. After the installation finishes, reboot.
For OpenCV 4, the build configuration process requires CMake 3 or a later version. Go to https://cmake.org/download/, download the installer for the latest version of CMake for your architecture (x64 or x86), and run it. During installation, select either Add CMake to the system PATH for all users or Add CMake to the system PATH for current user.
At this stage, we have set up the dependencies and build environment for our custom build of OpenCV. Now, we need to obtain the OpenCV source code and configure and build it. We can do this by following these steps:
Go to
https://opencv.org/releases/
and get the latest OpenCV download for Windows. It is a self-extracting ZIP. Run it and, when prompted, enter any destination folder, which we will refer to as
<opencv_unzip_destination>
. During extraction, a subfolder is created at
<opencv_unzip_destination>\opencv
.
Go to
https://github.com/opencv/opencv_contrib/releases
and download the latest ZIP of the
opencv_contrib
modules. Unzip this file to any destination folder, which we will refer to as
<opencv_contrib_unzip_destination>
.
Open the Command Prompt and run the following command to make another folder where our build will go:
> mkdir <build_folder>
Change the directory to the build folder:
> cd <build_folder>
Now, we are ready to configure our build with CMake's command-line interface. To understand all the options, we can read the code in
<opencv_unzip_destination>\opencv\CMakeLists.txt
. However, for this book's purposes, we only need to use the options that will give us a release build with Python bindings,
opencv_contrib
modules, non-free content, and depth camera support via OpenNI 2. Some options differ slightly, depending on the Visual Studio version and target architecture (x64 or x86). To create a 64-bit (x64) solution for Visual Studio 2019, run the following command (but replace
<opencv_contrib_unzip_destination>
and
<opencv_unzip_destination>
with the actual paths):
> cmake -DCMAKE_BUILD_TYPE=RELEASE -DOPENCV_SKIP_PYTHON_LOADER=ON -DPYTHON3_LIBRARY=C:/Python37/libs/python37.lib -DPYTHON3_INCLUDE_DIR=C:/Python37/include -DWITH_OPENNI2=ON -DOPENCV_EXTRA_MODULES_PATH="<opencv_contrib_unzip_destination>/modules" -DOPENCV_ENABLE_NONFREE=ON -G "Visual Studio 16 2019" -A x64 "<opencv_unzip_destination>/opencv/sources"
Alternatively, to create a 32-bit (x86) solution for Visual Studio 2019, run the following command (but replace <opencv_contrib_unzip_destination> and <opencv_unzip_destination> with the actual paths):
> cmake -DCMAKE_BUILD_TYPE=RELEASE -DOPENCV_SKIP_PYTHON_LOADER=ON -DPYTHON3_LIBRARY=C:/Python37/libs/python37.lib -DPYTHON3_INCLUDE_DIR=C:/Python37/include -DWITH_OPENNI2=ON -DOPENCV_EXTRA_MODULES_PATH="<opencv_contrib_unzip_destination>/modules"
-DOPENCV_ENABLE_NONFREE=ON -G "Visual Studio 16 2019" -A Win32
"<opencv_unzip_destination>/opencv/sources"
As the preceding command runs, it prints information about dependencies that are either found or missing. OpenCV has many optional dependencies, so do not panic (yet) about missing dependencies. However, if the build does not finish successfully, try installing missing dependencies. (Many are available as prebuilt binaries.) Then, repeat this step.
CMake will have generated a Visual Studio solution file at
<opencv_build_folder>/OpenCV.sln
. Open it in Visual Studio. Ensure that the
Release
configuration (not the
Debug
configuration) is selected in the drop-down list in the toolbar near the top of the Visual Studio window. (OpenCV's Python bindings will probably not build in Debug configuration, because most Python distributions do not contain debug libraries.) Go to the
BUILD
menu and select
Build Solution
. Watch the build messages in the
Output
pane at the bottom of the window, and wait for the build to finish.
By this stage, OpenCV has been built, but it hasn't been installed at a location where Python can find it. Before proceeding further, let's ensure that our Python environment does not already contain a conflicting build of OpenCV. Find and delete any OpenCV files in Python's DLLs folder and
site_packages
folder. For example, these files might match the following patterns:
C:\Python37\DLLs\opencv_*.dll
,
C:\Python37\Lib\site- packages\opencv
, and
C:\Python37\Lib\site-packages\cv2.pyd
.
Finally, let's install our custom build of OpenCV. CMake has generated an
INSTALL
project as part of the
OpenCV.sln
Visual Studio solution. Look in the
Solution Explorer
pane on the right-hand side of the Visual Studio window, find the
CMakeTargets | INSTALL
project, right-click on it, and select
Build
from the context menu. Again, w
atch the build messages in the
Output
pane at the bottom of the window and wait for the build to finish. Then, quit Visual Studio. Edit the system's
Path
variable and append either
;<build_folder>\install\x64\vc15\bin (for a 64-bit build)
or
;<build_folder>\install\x86\vc15\bin
(for a 32-bit build).
This folder is where the
INSTALL
project put the OpenCV
DLL
files, which are library files that Python will load dynamically at runtime. The OpenCV Python module is located at a path such as
C:\Python37\Lib\site-packages\cv2.pyd
. Python will find it there, so you do not need to add it to the
Path
. Log out and log back in (or reboot).
Click on the Start menu and launch Control Panel. Now, navigate to System and Security | System | Advanced system settings. Click on the EnvironmentVariables... button.
Now, under System variables, select Path and click on the Edit... button.
Make changes as directed.
To apply the changes, click on all the OK buttons (until we are back in the main window of the Control Panel).
Then, log out and log back in. (Alternatively, reboot.)
Now, we have completed the OpenCV build process on Windows, and we have a custom build that is suitable for all of this book's Python projects.
Installation on macOS
macOS comes with a preinstalled Python distribution that has been customized by Apple for the system's internal needs. To develop our own projects, we should make a separate Python installation to ensure that we do not conflict with the system's Python needs.
For macOS, there are several possible approaches for obtaining standard versions of Python 3, NumPy, SciPy, and OpenCV. All approaches ultimately require OpenCV to be compiled from source using the Xcode command-line tools. However, depending on the approach, this task is automated for us in various ways by third-party tools. We will look at this kind of approach using a package manager called Homebrew. A package manager can potentially do everything that CMake can, plus it helps us resolve dependencies and separate our development libraries from system libraries.
Before proceeding, let's make sure that the Xcode command line tools are set up properly. Open a Terminal and run the following command:
$ xcode-select --install
Agree to the license agreement and any other prompts. The installation should run to completion. Now, we have the compilers that Homebrew requires.
Using Homebrew with ready-made packages
Starting on a system where Xcode and its command-line tools are already set up, the following steps will give us an OpenCV installation via Homebrew:
Open a Terminal and run the following command to install Homebrew:
$ /usr/bin/ruby -e "$(curl -fsSL https://raw.github usercontent.com/Homebrew/install/master/install)"
Homebrew does not automatically put its executables in
PATH
. To do so, create or edit the
~/.profile
file and add the following line at the top of the code:
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
Save the file and run this command to refresh PATH:
$ source ~/.profile
Note that executables installed by Homebrew now take precedence over executables installed by the system.
For Homebrew's self-diagnostic report, run the following command:
$ brew doctor
Follow any troubleshooting advice it gives.
Now, update Homebrew:
$ brew update
Run the following command to install Python 3.7:
$ brew install python
Now, we want to install OpenCV with the
opencv_contrib
modules. At the same time, we want to install dependencies such as NumPy. To accomplish this, run the following command:
$ brew install opencv
Similarly, run the following command to install SciPy:
$ brew install scipy
Now, we have all we need to develop OpenCV projects with Python on macOS.
Using Homebrew with your own custom packages
Just in case you ever need to customize a package, Homebrew makes it easy to edit existing package definitions:
$ brew edit opencv
The package definitions are actually scripts in the Ruby programming language. Tips on editing them can be found on the Homebrew wiki page at https://github.com/Homebrew/brew/blob/master/docs/Formula-Cookbook.md. A script may specify Make or CMake configuration flags, among other things.
After making edits to the Ruby script, save it.
The customized package can be treated as normal. For example, it can be installed as follows:
$ brew install opencv
Installation on Debian, Ubuntu, Linux Mint, and similar systems
Debian, Ubuntu, Linux Mint, and related Linux distributions use the apt package manager. On these systems, it is easy to install packages for Python 3 and many Python modules, including NumPy and SciPy. An OpenCV package is also available via apt, but at the time of writing, this package has not been updated to OpenCV 4. Instead, we can obtain OpenCV 4 (without support for depth cameras) from Python's standard package manager, pip. Alternatively, we can build OpenCV 4 from source. When built from source, OpenCV can support depth cameras via OpenNI 2, which is available as a set of precompiled binaries with an installation script.
Regardless of our approach to obtaining OpenCV, let's begin by updating apt so that we can obtain the latest packages. Open a Terminal and run the following command:
$ sudo apt-get update
Having updated apt, let's run the following command to install NumPy and SciPy for Python 3:
$ sudo apt-get install python3-numpy python3-scipy
Now, we must decide whether we want a ready-made build of OpenCV (without support for depth cameras) or a custom build (with support for depth cameras). The next two subsections cover these alternatives.
Using a ready-made OpenCV package
OpenCV, including the opencv_contrib modules, can be installed as a pip package. This is as simple as running the following command:
$ pip3 install opencv-contrib-python
If you want your OpenCV installation to include non-free content, such as patented algorithms, then you can run the following command instead:
$ pip install opencv-contrib-python-nonfree
You might find that one of these pip packages offers all the OpenCV features you currently want. On the other hand, if you intend to use depth cameras, or if you want to learn about the general process of making a custom build of OpenCV, you should not install the OpenCV pip package; you should proceed to the next subsection instead.
Building OpenCV from source
To build OpenCV from source, we need a C++ build environment and the CMake build configuration system. Specifically, we need CMake 3. On Ubuntu 14.04, Linux Mint 17, and related systems, the cmake package is CMake 2, but a more up-to-date cmake3 package is also available. On these systems, run the following commands to ensure that the necessary versions of CMake and other build tools are installed:
$ sudo apt-get remove cmake
$ sudo apt-get install build-essential cmake3 pkg-config
On the other hand, on more recent operating systems, the cmake package is CMake 3, and we can simply run the following command:
$ sudo apt-get install build-essential cmake pkg-config
As part of the build process for OpenCV, CMake will need to access the internet to download additional dependencies. If your system uses a proxy server, ensure that your environment variables for the proxy server have been configured properly. Specifically, CMake relies on the http_proxy and https_proxy environment variables. To define these, you can edit your ~/.bash_profile script and add lines such as the following (but modify them so that they match your own proxy URLs and port numbers):
export http_proxy=http://myproxy.com:8080export https_proxy=http://myproxy.com:8081
To build OpenCV's Python bindings, we need an installation of the Python 3 development headers. To install these, run the following command:
$ sudo apt-get install python3-dev
To capture frames from typical USB webcams, OpenCV depends on Video for Linux (V4L). On most systems, V4L comes preinstalled, but just in case it is missing, run the following command:
$ sudo apt-get install libv4l-dev
As we mentioned previously, to support depth cameras, OpenCV depends on OpenNI 2. Go to https://structure.io/openni and download the latest ZIP of OpenNI 2 for Linux and for your system's architecture (x64, x86, or ARM). Unzip it to any destination, which we will refer to as <openni2_unzip_destination>. Run the following commands:
$ cd
<openni2_unzip_destination>$ sudo ./install.sh
The preceding installation script configures the system so that it supports depth cameras as USB devices. Moreover, the script creates environment variables that refer to library files inside <openni2_unzip_destination>. Therefore, if you move <openni2_unzip_destination> at a later date, you will need to run install.sh again.
Now that we have the build environment and dependencies installed, we can obtain and build the OpenCV source code. To do so, follow these steps:
Go to
https://opencv.org/releases/
and download the latest source package. Unzip it to any destination folder, which we will refer to as
<opencv_unzip_destination>
.
Go to
https://
github.
com/
opencv/opencv_
contrib/releases
and download the latest source package for the
opencv_contrib
modules. Unzip it to any destination folder, which we will refer to as
<opencv_contrib_unzip_destination>
.
Open a Terminal. Run the following commands to create a directory where we will put our OpenCV build:
$ mkdir <build_folder>
Change into the newly created directory:
$ cd <build_folder>
Now, we can use CMake to generate a build configuration for OpenCV. The output of this configuration process will be a set of Makefiles, which are scripts we can use to build and install OpenCV. A complete set of CMake configuration options for OpenCV is defined in the
<opencv_unzip_destination>/opencv/sources/CMakeLists.txt
file. For our purposes, we care about the options that relate to OpenNI 2 support, Python bindings,
opencv_contrib
modules, and non-free content. Configure OpenCV by running the following command:
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D BUILD_EXAMPLES=ON -D WITH_OPENNI2=ON -D BUILD_opencv_python2=OFF -D BUILD_opencv_python3=ON -D PYTHON3_EXECUTABLE=/usr/bin/python3.6 -D PYTHON3_INCLUDE_DIR=/usr/include/python3.6 -D PYTHON3_LIBRARY=/usr/lib/python3.6/config-3.6m-x86_64-linux- gnu/libpython3.6.so -D OPENCV_EXTRA_MODULES_PATH=<opencv_contrib_unzip_destination>
-D OPENCV_ENABLE_NONFREE=ON
<opencv_unzip_destination>
Finally, run the following commands to interpret our newly generated makefiles, and thereby build and install OpenCV:
$ make -j8
$ sudo make install
So far, we have completed the OpenCV build process on Debian, Ubuntu, or a similar system, and we have a custom build that is suitable for all of this book's Python projects.
Installation on other Unix-like systems
On other Unix-like systems, the package manager and available packages may differ. Consult your package manager's documentation and search for packages with opencv in their names. Remember that OpenCV and its Python bindings might be split into multiple packages.
Also, look for any installation notes that have been published by the system provider, the repository maintainer, or the community. Since OpenCV uses camera drivers and media codecs, getting all of its functionality to work can be tricky on systems with poor multimedia support. Under some circumstances, system packages might need to be reconfigured or reinstalled for compatibility.
If packages are available for OpenCV, check their version number. OpenCV 4 is recommended for this book's purposes. Also, check whether the packages offer Python bindings and depth camera support via OpenNI 2. Finally, check whether anyone in the developer community has reported success or failure in using the packages.
If, instead, you want to do a custom build of OpenCV from source, it might be helpful to refer to the previous section's steps for Debian, Ubuntu, and similar systems, and adapt these steps to the package manager and packages that are present on another system.
Running samples
Running a few sample scripts is a good way to test whether OpenCV has been set up correctly. Some samples are included in OpenCV's source code archive. If you have not already obtained the source code, go to https://opencv.org/releases/ and download one of the following archives:
For Windows, download the latest archive, labeled
Windows
.
It is a self-extracting ZIP. Run it and, when prompted, enter any destination folder, which we will refer to as
<opencv_unzip_destination>
.
Find the Python samples in
<opencv_unzip_destination>/opencv/samples/python
.
For other systems, download the latest archive, labeled
Sources
. It is a ZIP file. Unzip it to any destination
folder, which we will refer to as
<opencv_unzip_destination>
. Find the Python samples in
<opencv_unzip_destination>/samples/python
.
Some of the sample scripts require command-line arguments. However, the following scripts (among others) should work without any arguments:
hist.py
: This script displays a photo. Press
A
,
B
,
C
,
D
, or
E
to see the variations of the photo, along with a corresponding histogram of color or grayscale values.
opt_flow.py
: This script displays a webcam feed with a superimposed visualization of the optical flow, or in other words, the direction of motion. Slowly wave your hand at the webcam to see the effect. Press
1
or
2
for alternative visualizations.
To exit a script, press Esc (not the Windows close button).
If we encounter the ImportError: No module named cv2 message, then this means that we are running the script from a Python installation that does not know anything about OpenCV. There are two possible explanations for this:
Some steps in the OpenCV installation might have failed or been missed. Go back and review the steps.
If we have multiple Python installations on the machine, we might be using the wrong version of Python to launch the script. For example, on macOS, it might be the case that OpenCV has been installed for Homebrew Python, but we are running the script with the system's version of Python. Go back and review the installation steps about editing the system's
PATH
variable. Also, try launching the script manually from the command line using commands such as the following:
$ python hist.py
You can also try the following command:
$ python3.8 python/camera.py
As another possible means of selecting a different Python installation, try editing the sample script to remove the #! lines. These lines might explicitly associate the script with the wrong Python installation (for our particular setup).
Finding documentation, help, and updates
OpenCV's documentation can be found at http://docs.opencv.org/, where you can either read it online or download it for offline reading. If you write code on airplanes or other places without internet access, you will definitely want to keep offline copies of the documentation.
The documentation includes a combined API reference for OpenCV's C++ API and its Python API. When you look up a class or function, be sure to read the section under the heading Python.
If the documentation does not seem to answer your questions, try talking to the OpenCV community. Here are some sites where you will find helpful people:
The OpenCV forum:
https://answers.opencv.org/questions/
Adrian Rosebrock's website:
http://www.pyimagesearch.com/
Joseph Howse's website for his books and presentations:
http://nummist.com/opencv/
Lastly, if you are an advanced user who wants to try new features, bug fixes, and sample scripts from the latest (unstable) OpenCV source code, have a look at the project's repository at https://github.com/opencv/opencv/.
Summary
By now, we should have an OpenCV installation that will serve our needs for the diverse projects described in this book. Depending on which approach we took, we may also have a set of tools and scripts that can be used to reconfigure and rebuild OpenCV for our future needs.
Now, we also know where to find OpenCV's Python samples. These samples covered a different range of functionalities outside this book's scope, but they are useful as additional learning aids.
In the next chapter, we will familiarize ourselves with the most basic functions of the OpenCV API, namely, displaying images and videos, capturing videos through a webcam, and handling basic keyboard and mouse inputs.
Handling Files, Cameras, and GUIs
Installing OpenCV and running samples is fun, but at this stage, we want to try things out in our own way. This chapter introduces OpenCV's I/O functionality. We also discuss the concept of a project and the beginnings of an object-oriented design for this project, which we will flesh out in subsequent chapters.
By starting with a look at I/O capabilities and design patterns, we will build our project in the same way we would make a sandwich: from the outside in. Bread slices and spread, or endpoints and glue, come before fillings or algorithms. We choose this approach because computer vision is mostly extroverted—it contemplates the real world outside our computer—and we want to apply all of our subsequent algorithmic work to the real world through a common interface.
Specifically, in this chapter, our code samples and discussions will cover the following tasks:
Reading images from image files, video files, camera devices, or raw bytes of data in memory
Writing images to image files or video files
Manipulating image data in NumPy arrays
Displaying images in windows
Handling keyboard and mouse input
Implementing an application with an object-oriented design
