36,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Turn futuristic ideas about computer vision and machine learning into demonstrations that are both functional and entertaining
Key Features
- Build OpenCV 4 apps with Python 2 and 3 on desktops and Raspberry Pi, Java on Android, and C# in Unity
- Detect, classify, recognize, and measure real-world objects in real-time
- Work with images from diverse sources, including the web, research datasets, and various cameras
Book Description
OpenCV 4 is a collection of image processing functions and computer vision algorithms. It is open source, supports many programming languages and platforms, and is fast enough for many real-time applications. With this handy library, you'll be able to build a variety of impressive gadgets.
OpenCV 4 for Secret Agents features a broad selection of projects based on computer vision, machine learning, and several application frameworks. To enable you to build apps for diverse desktop systems and Raspberry Pi, the book supports multiple Python versions, from 2.7 to 3.7. For Android app development, the book also supports Java in Android Studio, and C# in the Unity game engine. Taking inspiration from the world of James Bond, this book will add a touch of adventure and computer vision to your daily routine. You'll be able to protect your home and car with intelligent camera systems that analyze obstacles, people, and even cats. In addition to this, you'll also learn how to train a search engine to praise or criticize the images that it finds, and build a mobile app that speaks to you and responds to your body language.
By the end of this book, you will be equipped with the knowledge you need to advance your skills as an app developer and a computer vision specialist.
What you will learn
- Detect motion and recognize gestures to control a smartphone game
- Detect car headlights and estimate their distance
- Detect and recognize human and cat faces to trigger an alarm
- Amplify motion in a real-time video to show heartbeats and breaths
- Make a physics simulation that detects shapes in a real-world drawing
- Build OpenCV 4 projects in Python 3 for desktops and Raspberry Pi
- Develop OpenCV 4 Android applications in Android Studio and Unity
Who this book is for
If you are an experienced software developer who is new to computer vision or machine learning, and wants to study these topics through creative projects, then this book is for you. The book will also help existing OpenCV users who want upgrade their projects to OpenCV 4 and new versions of other libraries, languages, tools, and operating systems. General familiarity with object-oriented programming, application development, and usage of operating systems (OS), developer tools, and the command line is required.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 383
Veröffentlichungsjahr: 2019
Ähnliche
OpenCV 4 for Secret Agents Second Edition
Copyright © 2019 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor:Richa TripathiAcquisition Editor: Chaitanya NairContent Development Editors: Digvijay BagulTechnical Editor: Riddesh DawneCopy Editor:Safis EditingProject Coordinator:Prajakta NaikProofreader:Safis EditingIndexer: Pratik ShirodkarGraphics: Tom ScariaProduction Coordinator: Nilesh Mohite
First published: January 2015 Second edition: April 2019
Production reference: 1300419
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78934-536-0
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Joseph Howse lives in a Canadian fishing village with four cats; the cats like fish, but they prefer chicken.
Joseph provides computer vision expertise through his company, Nummist Media. His books include OpenCV 4 for Secret Agents, OpenCV 3 Blueprints, Android Application Programming with OpenCV 3, iOS Application Development with OpenCV 3, Learning OpenCV 3 Computer Vision with Python, and Python Game Programming by Example, published by Packt.
About the reviewers
ChristianStehno studied computer science, receiving his diploma from Oldenburg University, Germany, in 2000. Since then, he's worked in different fields of computer science, first as a researcher on theoretical computer science at an academic institution, before switching later on to embedded system design at a research institute. In 2010, he started his own company, CoSynth, which develops embedded systems and intelligent cameras for industrial automation. In addition, he is a long-time member of the Irrlicht 3D engine developer team.
Arun Ponnusamy works as a computer vision research engineer at an AI start-up in India. He is a lifelong learner, passionate about image processing, computer vision, and machine learning. He is an engineering graduate from PSG College of Technology, Coimbatore. He started his career at MulticoreWare Inc., where he spent most of his time on image processing, OpenCV, software optimization, and GPU computing.
Arun loves to build his understanding of computer vision concepts clearly, allowing him to explain it in an intuitive way on his blog and in meetups. He has created an open source Python library for computer vision, named cvlib, which is aimed at simplicity and user friendliness. He is currently working on object detection, action recognition, and generative networks.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
OpenCV 4 for Secret Agents Second Edition
Dedication
About Packt
Why subscribe?
PacktPub.com
Contributors
About the author
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Section 1: The Briefing
Preparing for the Mission
Technical requirements
Setting up a development machine
Setting up Python and OpenCV on Windows
Building OpenCV on Windows with CMake and Visual Studio
Setting up Python and OpenCV on Mac
Mac with MacPorts
Mac with Homebrew
Setting up Python and OpenCV on Debian Jessie and its derivatives, including Raspbian, Ubuntu, and Linux Mint
Building OpenCV on Debian Jessie and its derivatives with CMake and GCC
Setting up Python and OpenCV on Fedora and its derivatives, including RHEL and CentOS
Setting up Python and OpenCV on openSUSE and its derivatives
Setting up Android Studio and OpenCV
Setting up Unity and OpenCV
Setting up a Raspberry Pi
Setting up the Raspberry Pi camera module
Finding OpenCV documentation, help, and updates
Alternatives to Raspberry Pi
Summary
Searching for Luxury Accommodations Worldwide
Technical requirements
Planning the Luxocator app
Creating, comparing, and storing histograms
Training the classifier with reference images
Acquiring images from the web
Acquiring images from Bing Image Search
Preparing images and resources for the app
Integrating everything into the GUI
Running Luxocator and troubleshooting SSL problems
Building Luxocator for distribution
Summary
Section 2: The Chase
Training a Smart Alarm to Recognize the Villain and His Cat
Technical requirements
Understanding machine learning in general
Planning the Interactive Recognizer app
Understanding Haar cascades and LBPH
Implementing the Interactive Recognizer app
Planning the cat-detection model
Implementing the training script for the cat-detection model
Planning the Angora Blue app
Implementing the Angora Blue app
Building Angora Blue for distribution
Further fun with finding felines
Summary
Controlling a Phone App with Your Suave Gestures
Technical requirements
Planning the Goldgesture app
Understanding optical flow
Setting up the project in Android Studio
Getting a cascade file and audio files
Specifying the app's requirements
Laying out a camera preview as the main view
Tracking back-and-forth gestures
Playing audio clips as questions and answers
Capturing images and tracking faces in an activity
Summary
Equipping Your Car with a Rearview Camera and Hazard Detection
Technical requirements
Planning The Living Headlights app
Detecting lights as blobs
Estimating distances (a cheap approach)
Implementing The Living Headlights app
Testing The Living Headlights app at home
Testing The Living Headlights app in a car
Summary
Creating a Physics Simulation Based on a Pen and Paper Sketch
Technical requirements
Planning the Rollingball app
Detecting circles and lines
Setting up OpenCV for Unity
Configuring and building the Unity project
Creating the Rollingball scene in Unity
Creating Unity assets and adding them to the scene
Writing shaders and creating materials
Creating physics materials
Creating prefabs
Writing our first Unity script
Writing the main Rollingball script
Creating the launcher scene in Unity
Tidying up and testing
Summary
Section 3: The Big Reveal
Seeing a Heartbeat with a Motion-Amplifying Camera
Technical requirements
Planning the Lazy Eyes app
Understanding what Eulerian video magnification can do
Extracting repeating signals from video using the fast Fourier transform
Choosing and setting up an FFT library
Compositing two images using image pyramids
Implementing the Lazy Eyes app
Configuring and testing the app for various motions
Summary
Seeing things in different light
Stopping Time and Seeing like a Bee
Technical requirements
Planning the Sunbaker app
Understanding the spectrum
Finding specialized cameras
XNiteUSB2S-MUV
Sony PlayStation Eye
Point Grey Grasshopper 3 GS3-U3-23S6M-C
Installing Spinnaker SDK and PySpin
Capturing images from industrial cameras using PySpin
Adapting the Lazy Eyes app to make Sunbaker
Summary
Making WxUtils.py Compatible with Raspberry Pi
Learning More about Feature Detection in OpenCV
Running with Snakes (or, First Steps with Python)
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Computer vision systems are deployed in the Arctic Ocean to spot icebergs at night. They are flown over the Amazon rainforest to create aerial maps of fires, blights, and illegal logging. They are set up in ports and airports worldwide to scan for suspects and contraband. They are sent to the depths of the Marianas Trench to guide autonomous submarines. They are used in operating rooms to help surgeons visualize the planned procedure and the patient's current condition. They are launched from battlefields as the steering systems of heat-seeking, anti-aircraft rockets.
We might seldom—or never—visit these places. However, stories often encourage us to imagine extreme environments and a person's dependence on tools in these unforgiving conditions. Perhaps fittingly, one of contemporary fiction's most popular characters is an almost ordinary man (handsome, but not too handsome; clever, but not too clever) who wears a suit, works for the British Government, always chooses the same drink, the same kind of woman, the same tone for delivering a pun, and is sent to do dangerous jobs with a peculiar collection of gadgets.
This book discusses seriously useful technologies and techniques, with a healthy dose of inspiration from spy fiction. The Bond franchise is rich in ideas about detection, disguise, smart devices, image capture, and sometimes, even computer vision specifically. With imagination, plus dedication to learning new skills, we can become the next generation of gadget makers to rival Bond's engineer, Q!
Who this book is for
This book is for tinkerers (and spies) who want to make computer vision a practical and fun part of their lifestyle. You should already be comfortable with 2D graphical concepts, object-oriented languages, GUIs, networking, and the command line. This book does not assume experience with any specific libraries or platforms. Detailed instructions cover everything from setting up the development environment to deploying finished apps.
A desire to learn multiple technologies and techniques, and then integrate them, is highly beneficial! This book will help you branch out to understand several types of systems and application domains where computer vision is relevant, and will help you to apply several approaches to detect, recognize, track, and augment faces, objects, and motions.
What this book covers
Chapter 1, Preparing for the Mission,helps us to install OpenCV, a Python development environment, and an Android development environment on Windows, macOS, or Linux systems. In this chapter, we also install a Unity development environment on Windows or macOS.
Chapter 2, Searching for Luxury Accommodations Worldwide,helps us to classify images of real estate based on color schemes. Are we outside a luxury dwelling or inside aStalinist apartment? In this chapter, we use the classifier in a search engine that labelsits image results.
Chapter 3, Training a Smart Alarm to Recognize the Villain and His Cat,helps us to detect and recognize human faces and cat faces as a means of controlling an alarm. HasErnst Stavro Blofeld returned, with his blue-eyed Angora cat?
Chapter 4, Controlling a Phone App with Your Suave Gestures,helps us to detect motion and recognize gestures as a means of controlling a guessing game on a smartphone. The phone knows why Bond is nodding, even if no one else does.
Chapter 5, Equipping Your Car with a Rearview Camera and Hazard Detection,helps us to detect car headlights, classify their color, estimate distances to them, and providefeedback to the driver. Is that car tailing us?
Chapter 6, Creating a Physics Simulation Based on a Pen and Paper Sketch, helps us to draw a ball-in-a-maze puzzle on paper, and see it come to life as a physics simulation on a smartphone. Physics and timing are everything!
Chapter 7, Seeing a Heartbeat with a Motion-Amplifying Camera,helps us to amplify motion in live video, in real time, so that a person's heartbeat and breathing become clearly visible. See the passion!
Chapter 8, Stopping Time and Seeing like a Bee, helps us improve the previous chapter's project by adopting specialized cameras for high-speed, infrared, or ultraviolet imaging. Surpass the limits of human vision!
Appendix A, Making WxUtils.py Compatible with Raspberry Pi, helps us solve a compatibility issue that affects the wxPython GUI library in some Raspberry Pi environments.
Appendix B, Learning More about Feature Detection in OpenCV, helps us discover more of OpenCV's feature-detection capabilities, beyond the ones we use in this book's projects.
Appendix C, Running with Snakes (or, First Steps with Python), helps us learn to run Python code and test an OpenCV installation in a Python environment.
To get the most out of this book
This book supports several operating systems as development environments, including Windows 7 SP 1 or later, macOS X 10.7 (Lion) or later, Debian Jessie, Raspbian, Ubuntu 14.04 or later, Linux Mint 17 or later, Fedora 28 or later, Red Hat Enterprise Linux (RHEL) 8 or a later version, CentOS 8 or later, openSUSE Leap 42.3, openSUSE Leap 15.0 or later, and openSUSE Tumbleweed.
The book contains six projects with the following requirements:
Four of these six projects run on Windows, macOS, or Linux, and require a webcam. Optionally, these projects can use Raspberry Pi or another single-board computer that runs Linux.
O
ne project runs on Android 5.0 (Lollipop) or a later version, and requires a front-facing camera (which most Android devices have).
One project runs on Android 4.1 (Jelly Bean) or a later version, and requires a rear-facing camera and gravity sensor (which most Android devices have). For development, it requires a Windows or macOS machine and approximately $95 worth of game development software.
Setup instructions for all required libraries and tools are covered in the book. Optional setup instructions for Raspberry Pi are also included.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packtpub.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/OpenCV-4-for-Secret-Agents-Second-Edition. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: http://www.packtpub.com/sites/default/files/downloads/9781789345360_ColorImages.pdf.
Conventions used
There are a number of text conventions used throughout this book.
CodeInText: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: "You can edit /etc/modules to check whether bcm2835-v4l2 is already listed there."
A block of code is set as follows:
set PYINSTALLER=pyinstallerREM Remove any previous build of the app.rmdir build /s /qrmdir dist /s /qREM Train the classifier.python HistogramClassifier.py
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
<
activity
android
:name=
"
.CameraActivity
"
android
:screenOrientation=
"landscape"
android
:theme=
"@android:style/Theme.NoTitleBar.Fullscreen"
> <
intent-filter
> <
action
android
:name=
"android.intent.action.MAIN"
/> <
category
android
:name=
"android.intent.category.LAUNCHER"
/> </
intent-filter
> </
activity
>
Any command-line input or output is written as follows:
$
echo "bcm2835-v4l2" | sudo tee -a /etc/modules
Bold: Indicates a new term, an important word, or words that you see onscreen. For example, words in menus or dialog boxes appear in the text like this. Here is an example: "Click the Android platform and then the Switch Platform button."
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Contacting the author: You can email Joseph Howse directly at [email protected]. He maintains this book's GitHub repository at https://github.com/PacktPublishing/OpenCV-4-for-Secret-Agents-Second-Edition, as well as his own support webpage for his books at http://nummist.com/opencv, so you may want to look for updates from him on these sites.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Section 1: The Briefing
Set up a multi-platform development environment. Integrate OpenCV with other libraries to make an application that classifies images from the web.
The following chapters will be covered in this section:
Chapter 1
,
Preparing for the Mission
Chapter 2
,
Searching for Luxury Accommodations Worldwide
Preparing for the Mission
James Bond is not a pedestrian. He cruises in a submarine car; he straps on a rocket belt; and, oh, how he skis, how he skis! He always has the latest stuff, and he is never afraid to put a dent in it, much to the dismay of Q, the engineer.
As software developers in the 2010s, we have witnessed an explosion of the adoption of new platforms. Under one family's roof, we might find a mix of Windows, Mac, iOS, and Android devices. Mom and Dad's workplaces provide different platforms. The kids have three game consoles, or five, if you count the mobile versions. The toddler has a LeapFrog learning tablet. Smart glasses are becoming more affordable.
We must not be afraid to try new platforms and consider new ways to combine them. After all, our users do.
This book embraces multiplatform development. It presents weird and wonderful applications that we can deploy in unexpected places. It uses several of the computer's senses, but especially computer vision, to breathe new life into the humdrum, heterogeneous clutter of devices that surrounds us.
Before Agent 007 runs amok with the gadgets, he is obligated to listen to Q's briefing. This chapter will performs Q's role. It is the setup chapter.
By the end of this chapter, you will have obtained all the tools to develop OpenCV applications in Python for Windows, Mac, or Linux, and in Java for Android. You will also be the proud new user of a Raspberry Pi single-board computer. (This additional hardware is optional.) You will even know a bit about Unity, a game engine into which we can integrate OpenCV. Specifically, this chapter will cover the following approaches to setting up a development environment:
Setting up Python and OpenCV on Windows. Optionally, this will include configuring and building OpenCV from a source with CMake and Visual Studio.
Setting up Python and OpenCV on Mac. This will include using either MacPorts or Homebrew as a package manager.
Setting up Python and OpenCV on Debian Jessie or one of its derivatives, such as Raspbian, Ubuntu, or Linux Mint. This will include using the
Advanced Package Tool
(
APT
) package manager. Optionally, it will also include configuring and building
OpenCV from a source with CMake and GCC.
Setting up Python and OpenCV on Fedora or one of its derivatives, such as
Red Hat Enterprise Linux
(
RHEL
) or CentOS. This will include using the yum package manager.
Setting up Python and OpenCV on openSUSE. This will include using the yum package manager.
Setting up Android Studio and OpenCV's Android libraries on Windows, Mac, or Linux.
Setting up Unity and OpenCV on Windows or Mac.
Setting up a Raspberry Pi.
If you find yourself a bit daunted by the extent of this setup chapter, be reassured that not all of the tools are required, and no single project uses all of them in combination. Although Q and I live for the big event of setting up multiple technologies at once, you can just skim this chapter and refer back to it later when the tools become useful, one by one, in our projects.
Technical requirements
This is the setup chapter. There are no particular software prerequisites at the outset; we will set up everything as we go along.
Basic instructions for running Python code are covered in Appendix C, Running with Snakes (or, First Steps with Python). After we set up a Python environment with OpenCV, you may want to refer to this appendix so that you know how to minimally test the environment.
Setting up a development machine
We can develop our OpenCV applications on a desktop, a notebook, or even the humble Raspberry Pi (covered later, in the Setting up a Raspberry Pi section). Most of our apps have a memory footprint of less than 128 MB, so they can still run (albeit slowly) on old or low-powered machines. To save time, develop on your fastest machine first and test on slower machines later.
This book assumes that you have one of the following operating systems on your development machine:
Windows 7 SP 1, or a later version.
Mac OS 10.7 (Lion), or a later version.
Debian Jessie, a later version, or a derivative such as the following:
Raspbian
2015-09-25, or a later version
Ubuntu 14.04, or a later version
Linux Mint 17, or a later version
Fedora 28, a later version, or a derivative such as the following:
RHEL 8, or a later version
CentOS 8, or a later version
openSUSE Leap 42.3, openSUSE Leap 15.0, or a later version; openSUSE Tumbleweed, or a derivative.
Other Unix-like systems might also work, but they will not be covered in this book.
You should have a USB webcam and any necessary drivers. Most webcams come with instructions for installing drivers on Windows and Mac. Linux distributions typically include the USB Video Class (UVC) Linux driver, which supports many webcams, listed at http://www.ideasonboard.org/uvc/#devices.
We are going to set up the following components:
On Mac, a third-party package manager to help us install libraries and their dependencies; we will use either MacPorts or Homebrew.
A Python development environment—at the time of writing this book, OpenCV supports Python 2.7, 3.4, 3.5, 3.6, and 3.7.
The Python code in this book supports all of these versions
. As part of the Python development environment, we will use Python's package manager, pip.
Popular Python libraries, such as NumPy (for numeric functions), SciPy (for numeric and scientific functions), Requests (for web requests), and wxPython (for cross-platform GUIs).
PyInstaller,
a cross-platform tool for bundling Python scripts, libraries, and data as redistributable apps, such that user machines do not require installations of Python, OpenCV, and other libraries. For this book's purposes, building redistributables of Python projects is an optional topic. We will cover the basics in
Chapter 2
,
Searching for Luxury Accommodations Worldwide,
but you might need to do your own testing and debugging, as
PyInstaller
(like other Python bundling tools) does not show entirely consistent behavior across operating systems, Python versions, and library versions. It is not well supported on Raspberry Pi or other ARM devices.
Optionally, we can use a C++ development environment to enable us to build OpenCV from a source. On Windows, we use Visual Studio 2015 or later. On Mac, we use Xcode. On Linux, we use GCC, which comes as standard.
A build of OpenCV and
opencv_contrib
(a set of extra OpenCV modules) with Python support, plus optimizations for certain desktop hardware. At the time of writing this book, OpenCV 4.0.x is the latest stable branch, and our instructions are tailored for this branch. However, generally, the code in this book also works with the previous stable branch, OpenCV 3.4.x, which is more widely available from package managers for users who prefer a prepackaged build.
Another build of OpenCV with Java support, plus optimizations for certain Android hardware. At the time of writing, OpenCV 4.0.1 is the most recent release.
An Android development environment, including Android Studio and Android SDK.
On 64-bit Windows or Mac, a three-dimensional game engine called
Unity
.
Let's break this setup down into three sets of platform dependent steps for a Python and OpenCV environment, plus a set of platform independent steps for an Android Studio and OpenCV environment, and another set of platform independent steps for a Unity and OpenCV environment.
Setting up Python and OpenCV on Windows
On Windows, we have the option of setting up a 32-bit development environment (to make apps that are compatible with both 32-bit and 64-bit Windows) or a 64-bit development environment (to make optimized apps that are only compatible with 64-bit Windows). OpenCV is available in 32-bit and 64-bit versions.
We also have a choice of either using binary installers or compiling OpenCV from source. For our Windows apps in this book, the binary installers provide everything we need. However, we also discuss the option of compiling from source because it enables us to configure additional features, which may be relevant to your future work or to our projects in other books.
Regardless of our approach to obtaining OpenCV, we need a general-purpose Python development environment. We will set up this environment using a binary installer. The installers for Python are available from http://www.python.org/getit/. Download and run the latest revision of Python 3.7, in either the 32-bit variant or the 64-bit variant.
To make Python scripts run using our new Python 3.7 installation by default, let's edit the system's Path variable and append ;C:\Python3.7 (assuming Python 3.7 is installed in the default location). Remove any previous Python paths, such as ;C:\Python2.7. Log out and log back in (or reboot).
Python comes with a package manager called pip, which simplifies the task of installing Python modules and their dependencies. Open Command Prompt and run the following command to install numpy, scipy, requests, wxPython, and pyinstaller:
>
pip install --user numpy scipy requests wxPython pyinstaller
Now, we have a choice. We can either install the binaries of OpenCV and opencv_contrib as a prebuilt Python module, or we can build this module from source. To install a prebuilt module, simply run the following command:
>
pip install --user opencv-contrib-python
Alternatively, to build OpenCV and opencv_contrib from source, follow the instructions in the section Building OpenCV on Windows with CMake and Visual Studio, as follow.
After either installing a prebuilt OpenCV and opencv_contrib module or building it from source, we will have everything we need to develop OpenCV applications for Windows. To develop for Android, we need to set up Android Studio as described in the section Setting up Android Studio, and OpenCV, later in this chapter.
Building OpenCV on Windows with CMake and Visual Studio
To compile OpenCV from source, we need a general purpose C++ development environment. As our C++ development environment, we will use Visual Studio 2015 or later. Use any installation media you may have purchased, or go to the downloads page at https://visualstudio.microsoft.com/downloads/. Download and run the installer for one of the following:
Visual Studio Community 2017, which is free
Any of the paid Visual Studio 2017 versions, which have 30-day free trials
If the installer lists optional C++ components, we should opt to install them all. After the installer runs to completion, reboot.
OpenCV uses a set of build tools called CMake, which we must install. Optionally, we may install several third-party libraries in order to enable extra features in OpenCV. As an example, let's install Intel Thread Building Blocks (TBB), which OpenCV can leverage in order to optimize some functions for multicore CPUs. After installing TBB, we will configure and build OpenCV. Lastly, we will ensure that our C++ and Python environments can find our build of OpenCV.
Here are the detailed steps:
Download and install the latest stable version of CMake from
https://cmake.org/download/
. CMake 3 or a newer version is required. Even if we are using 64-bit libraries and compilers, 32-bit CMake is compatible. When the installer asks about modifying
PATH
, select either
Add CMake to the system PATH for all users
or
Add CMake to the system PATH for current user
.
If your system uses a proxy server to access the internet, define two environment variables,
HTTP_PROXY
and
HTTPS_PROXY
, with values equal to the proxy server's URL, such as
http://myproxy.com:8080
. This ensures that CMake can use the proxy server to download some additional dependencies for OpenCV. (If in doubt, do not define these environment variables; you are probably not using a proxy server.)
Download the Op
enCV Win pack from
http://opencv.org/releases.html
. (Choose the latest version.) The downloaded file may have an
.exe
extension, but actually, it is a self-extracting
ZIP
. Double-click on the file and, when prompted, enter any destination folder, which we will refer to as
<opencv_unzip_destination>
. A subfolder,
<opencv_unzip_destination>/opencv
, will be created.
Download
opencv_contrib
as a ZIP from
https://github.com/opencv/opencv_contrib/releases
. (Choose the latest version.) Unzip it to any destination folder, which we will refer to as
<opencv_contrib_unzip_destination>
.
Download the latest stable version of TBB from
https://www.threadingbuildingblocks.org/download
. It includes both 32-bit and 64-bit binaries. Unzip it to any destination, which we will refer to as
<tbb_unzip_destination>
.
Open Command Prompt. Create a folder to store our build:
>
mkdir <build_folder>
Change directories to the newly created build folder:
>
cd <build_folder>
Having set up our dependencies, we can now configure OpenCV's build system. To understand all the configuration options, we can read the code in
<opencv_unzip_destination>/opencv/sources/CMakeLists.txt
. However, as an example, we will just use the options for a release build that includes Python bindings and multiprocessing through TBB:
To create a 32-bit project for Visual Studio 2017, run the following command (but replace the angle brackets and their contents with the actual paths):
>
CALL <tbb_unzip_destination>\bin\tbbvars.bat ia32 vs2017
>
cmake -DCMAKE_BUILD_TYPE=RELEASE DWITH_OPENGL=ON -DWITH_TBB=ON -DOPENCV_SKIP_PYTHON_LOADER=ON -DPYTHON3_LIBRARY=C:/Python37/libs/python37.lib -DPYTHON3_INCLUDE_DIR=C:/Python37/include -DOPENCV_EXTRA_MODULES_PATH="<opencv_contrib_unzip_destination>/modules" -G "Visual Studio 15 2017" "<opencv_unzip_destination>/opencv/sources"
Alternatively, to create a 64-bit project for Visual Studio 2017, run the following command (but replace the angle brackets and their contents with the actual paths):
>
CALL <tbb_unzip_destination>\bin\tbbvars.bat intel64 vs2017
>
cmake -DCMAKE_BUILD_TYPE=RELEASE DWITH_OPENGL=ON -DWITH_TBB=ON -DOPENCV_SKIP_PYTHON_LOADER=ON -DPYTHON3_LIBRARY=C:/Python37/libs/python37.lib -DPYTHON3_INCLUDE_DIR=C:/Python37/include -DOPENCV_EXTRA_MODULES_PATH="<opencv_contrib_unzip_destination>/modules" -G "Visual Studio 15 2017 Win64" "<opencv_unzip_destination>/opencv/sources"
CMake will produce a report on the dependencies that it did or did not find. OpenCV has many optional dependencies, so do not panic (yet) about missing dependencies. However, if the build does not finish successfully, try installing missing dependencies. (Many are available as prebuilt binaries.) Then, repeat this step.
Now that our build system is configured, we can compile OpenCV. Open
<build_folder>/OpenCV.sln
in Visual Studio. Select the
Release
configuration
and build the solution. (You may get errors if you select another build configuration besides
Release
, because most Python installations do not include debug libraries
.)
We should ensure that our Python installation does not already include some other version of OpenCV. Find and delete any OpenCV files in your Python
DLLs
folder and your Python
site-packages
folder. For example, the paths to these files might match the
C:\Python37\DLLs\opencv_*.dll
,
C:\Python37\Lib\site-packages\opencv
, and
C:\Python37\Lib\site-packages\cv2.pyd
patterns.
Finally, we need to install OpenCV to a location where Python and other processes can find it. To do this, right-click on the OpenCV solution's
INSTALL
project (in the
Solution Explorer
pane of Visual Studio) and build it. When this build finishes, quit Visual Studio.
Edit the system's
Path
variable and append
;<build_folder>\install\x86\vc15\bin
(for a 32-bit build) or
;<build_folder>\install\x64\vc15\bin
(for a 64-bit build), which is the location where the OpenCV
DLL f
iles are located
. Also, append
;<tbb_unzip_destination>\lib\ia32\vc14
(32-bit) or
;<tbb_unzip_ destination>\lib\intel64\vc14
(64-bit), which is the location where the TBB
DLL
files are located.
Log out and log back in (or reboot).
The
OpenCV
Python module is located at a path such as
C:\Python37\Lib\site-packages\cv2.pyd
. Python will find it there, so you do not need to take any further steps.
If you are using Visual Studio 2015, replace vs2017 with vs2015 , replace Visual Studio 15 2017 with Visual Studio 14 2015, and replace vc15 with vc14. However, for TBB, note that the folder named vc14 contains the DLL files that work for both Visual Studio 2015 and Visual Studio 2017.
You might want to look at the code samples in <opencv_unzip_destination>/opencv/sources/samples/python.
At this point, we have everything we need to develop OpenCV applications for Windows. To also develop for Android, we need to set up Android Studio, as described in the Setting up Android Studio and OpenCV section, later in this chapter.
Setting up Python and OpenCV on Mac
Mac comes with Python preinstalled. However, the preinstalled Python has been customized by Apple for the system's internal needs. Normally, we should not install any libraries on top of Apple's Python. If we do, our libraries might break during system updates, or worse, they might conflict with preinstalled libraries that the system requires. Instead, we should install standard Python 3.7 and then install our libraries on top of it.
For Mac, there are several possible approaches to obtaining standard Python 3.7 and Python-compatible libraries, such as OpenCV. All approaches ultimately require some components to be compiled from source, using Xcode developer tools. However, depending on the approach we choose, the task of building these components is automated for us by third-party tools in various ways.
Let's begin by setting up Xcode and the Xcode command-line tools, which give us a complete C++ development environment:
Download and install Xcode from the Mac App Store or
https://developer.apple.com/xcode/
. If the installer provides an option to install c
ommand-line tools
, select it.
Open Xcode. If a license agreement is presented, accept it.
If command-
line tools were not already installed, we must install them now. Go to
Xcode
|
Preferences
|
Downloads
and click on the
Install
button next to
command-line tools
. Wait for the installation to finish. Then, quit Xcode. Alternatively, if you do not find an option to install command-line tools from inside Xcode, open the Terminal and run the following command:
$
xcode-select install
Now, we will look at ways to automate our builds using MacPorts or Homebrew. These two tools are package managers, which help us resolve dependencies and separate our development libraries from the system libraries.
Generally, I recommend MacPorts. Compared to Homebrew, MacPorts offers more patches and configuration options for OpenCV. On the other hand, Homebrew offers more timely updates for OpenCV. At the time of writing, Homebrew offers a package for OpenCV 4.0.1, but MacPorts is still lagging at OpenCV 3.4.3. Homebrew and MacPorts can coexist with the Python package manager, pip, and we can use pip to get OpenCV 4.0.1, even though MacPorts does not package this version yet. Normally, MacPorts and Homebrew should not be installed on the same machine.
Now, depending on your preference, let's proceed to either the Mac with MacPorts section or the Mac with Homebrew section.
Mac with MacPorts
MacPorts provides Terminal commands that automate the process of downloading, compiling, and installing various pieces of open source software(OSS). MacPorts also installs dependencies, as needed. For each piece of software, the dependencies and build recipe are defined in a configuration file called a Portfile. A MacPorts repository is a collection of Portfiles.
Starting from a system where Xcode and its command-line tools are already set up, the following steps will give us an OpenCV installation through MacPorts:
Download and install MacPorts from
http://www.macports.org/install.php
.
Open the Terminal and run the following command to update MacPorts:
$
sudo port selfupdate
When prompted, enter your password.
Run the following commands to install Python 3.7, pip, NumPy, SciPy, and Requests
:
$
sudo port install python37
$
sudo port install py37-pip
$
sudo port install py37-numpy
$
sudo port install py37-scipy
$
sudo port install py37-requests
The Python installation executable is named python3.7. To link the default python executable to python3.7, and to link the default pip executable to this Python pip installation, let's also run the following:
$
sudo port install python_select
$
sudo port select python python37
$
sudo port install pip_select
$
sudo port select pip pip37
At the time of writing, MacPorts only has packages for relatively old versions of wxPython and PyInstaller. Let's use the following
pip
command to install more recent versions:
$
pip install --user wxPython pyinstaller
To check whether MacPorts has an OpenCV 4 package, run
$ port list opencv
. At the time of writing, this produces the following output:
opencv
@3.4.3
graphics/opencv
Here,
@3.4.3
means that OpenCV 3.4.3 is the latest available package from MacPorts. However, if your output shows
@4.0.0
or a more recent version, you can use MacPorts to configure, build, and install OpenCV 4, by running a command such as the following:
$
sudo port install opencv +avx2 +contrib +opencl +python37
By adding
+avx2 +contrib +opencl +python37
to the command, we are specifying that we want the
opencv
variant (build configuration) with AVX2 CPU optimizations,
opencv_contrib
extra modules, OpenCL GPU optimizations, and Python 3.7 bindings. To see the full list of available variants before installing, we can enter the following:
$
port variants opencv
Depending on our customization needs, we can add other variants to the
install
command.
On the other hand, if the output from
$ port list opencv
showed that MacPorts does not have an OpenCV 4 package yet, we can instead install OpenCV 4 and the
opencv_contrib
extra modules with pip, by running the following command:
$
pip install --user opencv-contrib-python
Now, we have everything we need to develop OpenCV applications for Mac. To also develop for Android, we need to set up Android Studio, as we will describe in the following Android Studio section.
Mac with Homebrew
Like MacPorts, Homebrew is a package manager that provides Terminal commands to automate the process of downloading, compiling, and installing various pieces of open source software.
Starting from a system where Xcode and its command-line tools are already set up, the following steps will give us an OpenCV installation through Homebrew:
Open Terminal and run the following command to install Homebrew:
$
/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)
"
Unlike MacPorts, Homebrew does not automatically put its executables in
PATH
. To do so, create or edit the
~/.profile
file and add this line at the top:
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
Save the file and run this command to refresh
PATH
:
$
source ~/.profile
Note that executables installed by Homebrew now take precedence over executables installed by the system.
For Homebrew's self-diagnostic report, run:
$
brew doctor
Follow any troubleshooting advice it gives.
Now, update Homebrew:
$
brew update
Run the following command to install Python 3.7:
$
brew install python
Now, we can use Homebrew to install OpenCV and its dependencies, including NumPy. Run the following command:
$
brew install
opencv --with-contrib
Similarly, run the following command to install SciPy:
$
pip install --user scipy
At the time of writing, Homebrew does not have packages for
requests
and
pyinstaller
, and its
wxPython
package is a relatively old version, so instead, we will use
pip
to install these modules. Run the following command:
$
pip install --user requests wxPython pyinstaller
Now, we have everything we need to develop OpenCV applications for Mac. To also develop for Android, we need to set up Tegra Android Development Pack (TADP), as described in the following Tegra Android Development Pack section.
Setting up Python and OpenCV on Debian Jessie and its derivatives, including Raspbian, Ubuntu, and Linux Mint
On Debian Jessie, Raspbian, Ubuntu, Linux Mint, and their derivatives, the python executable is Python 2.7, which comes preinstalled. We can use the system package manager, apt, to install NumPy, SciPy, and Requests from the standard repository. To update the apt package index and install the packages, run the following commands in Terminal:
$
sudo apt-get update
$
sudo apt-get install python-numpy python-scipy python-requests
The standard repository's latest packaged version of wxPython varies, depending on the operating system. On Ubuntu 14.04 and its derivatives, including Linux Mint 17, the latest packaged version is wxPython 2.8. Install it by running the following command:
$
sudo apt-get install python-wxgtk2.8
On Ubuntu 18.04 and newer versions, as well as derivatives such as Linux Mint 19, the latest packaged version is wxPython 4.0. Install it by running the following command:
$
sudo apt-get install python-wxgtk4.0
