39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
Making your applications see has never been easier with OpenCV. With it, you can teach your robot how to follow your cat, write a program to correctly identify the members of One Direction, or even help you find the right colors for your redecoration.
OpenCV 3 Computer Vision Application Programming Cookbook Third Edition provides a complete introduction to the OpenCV library and explains how to build your first computer vision program. You will be presented with a variety of computer vision algorithms and exposed to important concepts in image and video analysis that will enable you to build your own computer vision applications.
This book helps you to get started with the library, and shows you how to install and deploy the OpenCV library to write effective computer vision applications following good programming practices. You will learn how to read and write images and manipulate their pixels. Different techniques for image enhancement and shape analysis will be presented. You will learn how to detect specific image features such as lines, circles or corners. You will be introduced to the concepts of mathematical morphology and image filtering.
The most recent methods for image matching and object recognition are described, and you’ll discover how to process video from files or cameras, as well as how to detect and track moving objects. Techniques to achieve camera calibration and perform multiple-view analysis will also be explained. Finally, you’ll also get acquainted with recent approaches in machine learning and object classification.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 531
Veröffentlichungsjahr: 2017
Ähnliche
Table of Contents
OpenCV 3 Computer Vision Application Programming Cookbook - Third Edition
OpenCV 3 Computer Vision Application Programming Cookbook - Third Edition
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: May 2011
Second edition: August 2014
Third edition: February 2017
Production reference: 1070217
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78646-971-7
www.packtpub.com
Credits
Author
Robert Laganiere
Copy Editor
Safis Editing
Reviewer
Luca Del Tongo
Project Coordinator
Ulhas Kambali
Commissioning Editor
Edward Gordon
Proofreader
Safis Editing
Acquisition Editor
Nitin Dasan
Indexer
Tejal Daruwale Soni
Content Development Editor
Nikhil Borkar
Graphics
Kirk D'Penha
Jason Monteiro
Technical Editor
Subhalaxmi Nadar
Production Coordinator
Shantanu Zagade
About the Author
Robert Laganiere is a professor at the School of Electrical Engineering and Computer Science of the University of Ottawa, Canada. He is also a faculty member of the VIVA research lab and is the co-author of several scientific publications and patents in content-based video analysis, visual surveillance, driver-assistance, object detection, and tracking. Robert authored the OpenCV2 Computer Vision Application Programming Cookbook in 2011 and co-authored Object Oriented Software Development published by McGraw Hill in 2001. He co-founded Visual Cortek in 2006, an Ottawa-based video analytics startup that was later acquired by http://iwatchlife.com/ in 2009. He is also a consultant in computer vision and has assumed the role of Chief Scientist in a number of startups companies such as Cognivue Corp, iWatchlife, and Tempo Analytics. Robert has a Bachelor of Electrical Engineering degree from Ecole Polytechnique in Montreal (1987) and MSc and PhD degrees from INRS-Telecommunications, Montreal (1996). Visit the author’s website at http://www.laganiere.name/.
I wish to thank all my students at the VIVA lab; I learn so much from them.
About the Reviewer
Luca Del Tongo is a computer engineer with a strong passion for algorithms, computer vision, and image processing techniques. He's the coauthor of a free e-book called Data Structures and Algorithms (DSA) with over 100k downloads so far and has published several image processing tutorials on his YouTube channel using Emgu CV. During his master's thesis, he developed an image forensic algorithm published in a scientific paper called Copy Move forgery detection and localization by means of robust clustering with J-Linkage. Currently, Luca works as a software engineer in the ophthalmology field developing corneal topography, processing algorithms, IOL calculation, and computerized chart projector. He loves to play sport and follow MOOC courses in his spare time.
You can contact him through his blog at http://blogs.ugidotnet.org/wetblog.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Customer Feedback
Thank you for purchasing this Packt book. We take our commitment to improving our content and products to meet your needs seriously—that's why your feedback is so valuable. Whatever your feelings about your purchase, please consider leaving a review on this book's Amazon page. Not only will this help us, more importantly it will also help others in the community to make an informed decision about the resources that they invest in to learn.
You can also review for us on a regular basis by joining our reviewers' club. If you're interested in joining, or would like to learn more about the benefits we offer, please contact us: [email protected].
Preface
Augmented reality, driving assistance, video monitoring; more and more applications are now using computer vision and image analysis technologies, and yet we are still in the infancy of the development of new computerized systems capable of understanding our worlds through the sense of vision. And with the advent of powerful and affordable computing devices and visual sensors, it has never been easier to create sophisticated imaging applications. A multitude of software tools and libraries manipulating images and videos are available, but for anyone who wishes to develop smart vision-based applications, the OpenCV library is the tool to use. OpenCV (Open source Computer Vision) is an open source library containing more than 500 optimized algorithms for image and video analysis. Since its introduction in 1999, it has been largely adopted as the primary development tool by the community of researchers and developers in computer vision. OpenCV was originally developed at Intel by a team led by Gary Bradski as an initiative to advance research in vision and promote the development of rich vision-based, CPU-intensive applications. After a series of beta releases, version 1.0 was launched in 2006. A second major release occurred in 2009 with the launch of OpenCV 2 that proposed important changes, especially the new C++ interface, which we use in this book. In 2012, OpenCV reshaped itself as a non-profit foundation (http://opencv.org/) relying on crowdfunding for its future development. OpenCV3 was introduced in 2013; changes were made mainly to improve the usability of library. Its structure has been revised to remove the unnecessary dependencies, large modules have been split into smaller ones and the API has been refined. This book is the third edition of the OpenCV Computer Vision Application Programming Cookbook and the first one that covers OpenCV version 3. All the programming recipes of the previous editions have been reviewed and updated. We also have added new content and new chapters to provide readers with even better coverage of the essential functionalities of the library. This book covers many of the library’s features and explains how to use them to accomplish specific tasks. Our objective is not to provide detailed coverage of every option offered by the OpenCV functions and classes but rather to give you the elements you need to build your applications from the ground up. We also explore, in this book, fundamental concepts in image analysis and we describe some of the important algorithms in computer vision. This book is an opportunity for you to get introduced to the world of image and video analysis. But this is just the beginning. The good news is that OpenCV continues to evolve and expand. Just consult the OpenCV online documentation at http://opencv.org/ to stay updated about what the library can do for you. You can also visit the author’s website at http://www.laganiere.name/ for updated information about this cookbook.
What this book covers
Chapter 1, Playing with Images, introduces the OpenCV library and shows you how to build simple applications that can read and display images. It also introduces the basic OpenCV data structures.
Chapter 2, Manipulating Pixels, explains how an image can be read. It describes different methods for scanning an image in order to perform an operation on each of its pixels.
Chapter 3, Processing the Colors of an Image, consists of recipes presenting various object-oriented design patterns that can help you to build better computer vision applications. It also discusses the concept of colors in images.
Chapter 4, Counting the Pixels with Histograms, shows you how to compute image histograms and how they can be used to modify an image. Different applications based on histograms are presented that achieve image segmentation, object detection, and image retrieval.
Chapter 5, Transforming Images with Morphological Operations, explores the concept of mathematical morphology. It presents different operators and how they can be used to detect edges, corners, and segments in images.
Chapter 6, Filtering the Images, teaches you the principle of frequency analysis and image filtering. It shows how low-pass and high-pass filters can be applied to images and presents the concept of derivative operators.
Chapter 7, Extracting Lines, Contours, and Components, focuses on the detection of geometric image features. It explains how to extract contours, lines and connected components in an image.
Chapter 8, Detecting Interest Points, describes various feature point detector in images.
Chapter 9, Describing and Matching Interest Points, explains how descriptors of interest points can be computed and used to match points between images.
Chapter 10, Estimating Projective Relations in Images, explores the projective relations that exist between two images in the same scene. It also describes how to detect specific targets in an image.
Chapter 11, Reconstructing 3D scenes, allows you to reconstruct the 3D elements of a scene from multiple images and recover the camera pose. It also includes a description of the camera calibration process.
Chapter 12, Processing Video Sequences, provide a framework to read and write a video sequence and to process its frames. It shows you also how it is possible to extract the foreground objects moving in front of a camera.
Chapter 13, Tracking Visual Motion, addresses the visual tracking problem. It shows you how to compute the apparent motion in videos. It also explains how to track moving objects in an image sequence.
Chapter 14, Learning from Examples, introduces basic concepts in machine learning. It shows how object classifiers can be built from image samples.
What you need for this book
This cookbook is based on the C++ API of the OpenCV library. It is therefore assumed that you have some experience with the C++ language. In order to run the examples presented in the recipes and experiment with them, you need a good C++ development environment. Microsoft Visual Studio and Qt are two popular choices.
Who this book is for
This cookbook is appropriate for novice C++ programmers who want to learn how to use the OpenCV library to build computer vision applications. It is also suitable for professional software developers who wants to be introduced to the concepts of computer vision programming. It can be used as a companion book in university-level computer vision courses. It constitutes an excellent reference for graduate students and researchers in image processing and computer vision.
Sections
In this book, you will find several headings that appear frequently (Getting ready, How to do it, How it works, There's more, and See also). To give clear instructions on how to complete a recipe, we use these sections as follows:
Getting ready
This section tells you what to expect in the recipe, and describes how to set up any software or any preliminary settings required for the recipe.
How to do it…
This section contains the steps required to follow the recipe.
How it works…
This section usually consists of a detailed explanation of what happened in the previous section.
There's more…
This section consists of additional information about the recipe in order to make the reader more knowledgeable about the recipe.
See also
This section provides helpful links to other useful information for the recipe.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply e-mail [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors .
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
You can download the code files by following these steps:
You can also download the code files by clicking on the Code Files button on the book's webpage at the Packt Publishing website. This page can be accessed by entering the book's name in the Search box. Please note that you need to be logged in to your Packt account.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/OpenCV3-Computer-Vision-Application-Programming-Cookbook-Third-Edition . We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
The source code files of the examples presented in this cookbook are also hosted in the author's Github repository. You can visit the author's repository at https://github.com/laganiere to obtain the latest version of the code.
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/OpenCV3ComputerVisionApplicationProgrammingCookbookThirdEdition_ColorImages.pdf.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Chapter 1. Playing with Images
In this chapter, we will get you started with the OpenCV library. You will learn how to perform the following tasks:
Introduction
This chapter will teach you the basic elements of OpenCV and will show you how to accomplish the most fundamental image processing tasks: reading, displaying, and saving images. However, before you start with OpenCV, you need to install the library. This is a simple process that is explained in the first recipe of this chapter.
All your computer vision applications will involve the processing of images. This is why OpenCV offers you a data structure to handle images and matrices. It is a powerful data structure with many useful attributes and methods. It also incorporates an advanced memory management model that greatly facilitates the development of applications. The last two recipes of this chapter will teach you how to use this important data structure of OpenCV.
Installing the OpenCV library
OpenCV is an open source library for developing computer vision applications that can run on multiple platforms, such as Windows, Linux, Mac, Android, and iOS. It can be used in both academic and commercial applications under a BSD license that allows you to freely use, distribute, and adapt it. This recipe will show you how to install the library on your machine.
Getting ready
When you visit the OpenCV official website at http://opencv.org/ , you will find the latest release of the library, the online documentation describing the Application Programming Interface (API), and many other useful resources on OpenCV.
How to do it...
From the OpenCV website, find the latest available downloads and select the one that corresponds to the platform of your choice (Windows, Linux/Mac, or iOS). Once the OpenCV package is downloaded, run the WinZip self-extractor and select the location of your choice. An opencv directory will be created; it is a good idea to rename it in a way that will show which version you are using (for example, in Windows, your final directory could be C:\opencv-3.2). This directory will contain a collection of files and directories that constitute the library. Notably, you will find the sources directory that will contain all the source files (yes, it is open source!).
In order to complete the installation of the library and have it ready for use, you need to take an important step: generate the binary files of the library for the environment of your choice. This is indeed the point where you have to make a decision on the target platform you wish to use to create your OpenCV applications. Which operating system do you prefer to use? Which compiler should you select? Which version? 32-bit or 64-bit? As you can see, there are many possible options, and this is why you have to build the library that fits your needs.
The Integrated Development Environment (IDE) you will use in your project development will also guide you to make these choices. Note that the library package also comes with precompiled binaries that you can directly use if they correspond to your situation (check the build directory adjacent to the sources directory). If one of the precompiled binaries satisfies your requirements, then you are ready to go.
One important remark, however. Since version 3, OpenCV has been split into two major components. The first one is the main OpenCV source repository that includes the mature algorithms. This is the one you have downloaded. A separate contribution repository also exists, and it contains the new computer vision algorithm, recently added by the OpenCV contributors. If your plan is to use only the core functions of OpenCV, you do not need the contrib package. But if you want to play with the latest state-of-the-art algorithms, then there is a good chance that you will need this extra module. As a matter of fact, this cookbook will show you how to use several of these advanced algorithms. You therefore need the contrib modules to follow the recipes of this book. So you have to go to https://github.com/opencv/opencv_contrib and download OpenCV's extra modules (download the ZIP file). You can unzip the extra modules into the directory of your choice; these modules should be found at opencv_contrib-master/modules. For simplicity, you can rename this directory as contrib and copy it directly inside the sources directory of the main package. Note that you can also pick the extra modules of your choice and only save them; however, you will probably find it easier, at this point, to simply keep everything.
You are now ready to proceed with the installation. To build the OpenCV binaries, it is highly suggested that you use the CMake tool, available at http://cmake.org . CMake is another open source software tool designed to control the compilation process of a software system using platform-independent configuration files. It generates the required makefile or solution files needed for compiling a software library in your environment. Therefore, you have to download and install CMake. Also see the There's more... section of this recipe for an additional software package, the Visualization Toolkit (VTK), that you may want to install before compiling the library.
You can run cmake using a command-line interface, but it is easier to use CMake with its graphical interface (cmake-gui). In the latter case, all you need to do is specify the folder containing the OpenCV library source and the one that will contain the binaries. Now click on Configure and select the compiler of your choice:
Once this initial configuration is completed, CMake will provide you with a number of configuration options. You have to decide, for example, whether you want to have the documentation installed or whether you wish to have some additional libraries installed. Unless you know what you are doing, it is probably better to leave the default options as they are. However, since we want to include the extra modules, we have to specify the directory where they can be found:
Once the extra module path is specified, click on Configure again. You are now ready to generate the project files by clicking on the Generate button. These files will allow you to compile the library. This is the last step of the installation process, which will make the library ready to be used in your development environment. For example, if you select MS Visual Studio, then all you need to do is open the top-level solution file that CMake has created for you (the OpenCV.sln file). You then select the INSTALL project (under CMakeTargets) and issue the Build command (use right-click).
To get both a Release and Debug build, you will have to repeat the compilation process twice, one for each configuration. If everything goes well, you will have an install directory (under build) created. This directory will contain all the binary files of the OpenCV library to be linked with your application as well as the dynamic library files that your executables have to call at runtime. Make sure you set your system's PATH environment variable (from Control Panel) such that your operating system would be able to find the .dll files when you run your applications (for example, C:\opencv-3.2\build \install\x64\vc14\bin). You should also define the environment variable, OPENCV_DIR pointing to the INSTALL directory. This way, CMake will be able to find the library when configuring future projects.
In Linux environments, you can use Cmake to generate the required Makefiles; you then complete the installation by executing a sudo make install command. Alternatively, you could also use the packaging tool apt-get which can automatically perform a complete installation of the library. For Mac OS, you should use the Homebrew package manager. Once installed, you just have to type brew install opencv3 --with-contrib in order to have the complete library installed (run brew info opencv3 to view all possible options).
How it works...
OpenCV is a library that is in constant evolution. With version 3, the library continues to expand offering a lot of new functionalities with enhanced performances. The move to having a full C++ API, which was initiated in version 2, is now almost complete, and more uniform interfaces have been implemented. One of the major changes introduced in this new version is the restructuring of the modules of the library in order to facilitate its distribution. In particular, a separate repository containing the most recent algorithms has been created. This contrib repository also contains non-free algorithms that are subject to specific licenses. The idea is for OpenCV to be able to offer state-of-the-art functionalities that developers and researchers want to share while still being able to offer a very stable and well-maintained core API. The main modules are therefore the ones you get when you download the library at http://opencv.org/. The extra modules must be downloaded directly from the development repository hosted on GitHub ( https://github.com/opencv/ ). Since these extra modules are in constant development, you should expect more frequent changes to the algorithms they contain.
The OpenCV library is divided into several modules. For example, the opencv_core module contains the core functionalities of the library; the opencv_imgproc module includes the main image processing functions; the opencv_highgui module offers the image and video reading and writing functions along with some user interface functions; and so on. To use a particular module, you have to include the corresponding top-level header file. For instance, most applications that use OpenCV start with the following declarations:
#include <opencv2/core.hpp> #include <opencv2/imgproc.hpp> #include <opencv2/highgui.hpp>As you learn to work with OpenCV, you will discover more and more functionalities available in its numerous modules.
There's more...
The OpenCV website at http://opencv.org/ contains detailed instructions on how to install the library. It also contains complete online documentation that includes several tutorials on the different components of the library.
The Visualization Toolkit and the cv::viz module
In some applications, computer vision is used to reconstruct the 3D information of a scene from images. When working with 3D data, it is often useful to be able to visualize the results in some 3D virtual world. As you will learn in Chapter 11 , Reconstructing 3D Scenes, the cv::viz module offers many useful functions that allow you to visualize scene objects and cameras in 3D. However, this module is built on top of another open source library: VTK. Therefore, if you want to use the cv::viz module, you need to install VTK on your machine before compiling OpenCV.
VTK is available at http://www.vtk.org/. All you have to do is download the library and use CMake in order to create the binaries for your development environment. In this book, we used version 6.3.0. In addition, you should define the VTK_DIR environment variable, pointing to the directory containing the built files. Also, in the configuration options proposed during the OpenCV installation process with CMake, make sure that the WITH_VTK option is checked.
The OpenCV developer site
OpenCV is an open source project that welcomes user contributions. The library is hosted on GitHub, a web service that offers version control and source code management tools based on Git. You can access the developer site at https://github.com/opencv/opencv/wiki . Among other things, you can access the currently developed version of OpenCV. The community uses Git as their version control system. Git is also a free open source software system; it is probably the best tool you can use to manage your own source code.
Note
Downloading the example source code of this book: The source code files of the examples presented in this cookbook are also hosted on GitHub. Please visit the author's repository at https://github.com/laganiere to obtain the latest version of the code. Note that you can download the example code files for all the Packt books you have purchased from your account at http://www.packtpub.com . If you have purchased this book elsewhere, you can visit http://www.packtpub.com/support and register yourselves there to have the files e-mailed directly to you.
See also
Loading, displaying, and saving images
It is now time to run your first OpenCV application. Since OpenCV is about processing images, this task will show you how to perform the most fundamental operations needed in the development of imaging applications. These are loading an input image from a file, displaying an image on a window, applying a processing function, and saving the output image.
Getting ready
Using your favorite IDE (for example, MS Visual Studio or Qt), create a new console application with a main function that is ready to be filled.
How to do it...
The first thing to do is to include the header files, declaring the classes and functions you wish to use. Here, we simply want to display an image, so we need the core header that declares the image data structure and the highgui header file that contains all the graphical interface functions:
#include <opencv2/core.hpp> #include <opencv2/highgui.hpp>Our main function starts by declaring a variable that will hold the image. Under OpenCV, this is done by defining an object of the cv::Mat class:
cv::Mat image; // create an empty imageThis definition creates an image of size 0x0. This can be confirmed by accessing the cv::Mat size attributes:
std::cout << "This image is " << image.rows << " x " << image.cols << std::endl;Next, a simple call to the reading function will read an image from a file, decode it, and allocate the memory:
image= cv::imread("puppy.bmp"); // read an input imageYou are now ready to use this image. However, you should first check whether the image has been correctly read (an error will occur if the file is not found, is corrupted, or is not in a recognizable format). The validity of the image is tested using the following code:
if (image.empty()) { // error handling // no image has been created... // possibly display an error message // and quit the application ... }The empty method returns true if no image data has been allocated.
The first thing you might want to do with this image is display it. You can do this using the functions of the highgui module. Start by declaring the window on which you want to display the images, then specify the image to be shown on this special window:
// define the window (optional) cv::namedWindow("Original Image"); // show the image cv::imshow("Original Image", image);As you can see, the window is identified by a name. You can reuse this window to display another image later, or you can create multiple windows with different names. When you run this application, you will see an image window, as follows:
Now, you would normally apply some processing to the image. OpenCV offers a wide selection of processing functions, and several of them are explored in this book. Let's start with a very simple one that flips an image horizontally. Several image transformations in OpenCV can be performed in-place, meaning the transformation is applied directly on the input image (no new image is created). This is the case for the flipping method. However, we can always create another matrix to hold the output result, and this is what we will do:
cv::Mat result; // we create another empty image cv::flip(image,result,1); // positive for horizontal // 0 for vertical, // negative for bothThe result is displayed on another window:
cv::namedWindow("Output Image"); // the output window cv::imshow("Output Image", result);Since it is a console window that will terminate when it reaches the end of the main function, we add an extra highgui function to wait for a user key before we end the program:
cv::waitKey(0); // 0 to indefinitely wait for a key pressed // specifying a positive value will wait for // the given amount of msecYou can then see that the output image is displayed in a distinct window, as shown in the following screenshot:
Finally, you will probably want to save the processed image on your disk. This is done using the following highgui function:
cv::imwrite("output.bmp", result); // save resultThe file extension determines which codec will be used to save the image. Other popular supported image formats are JPG, TIFF, and PNG.
How it works...
All classes and functions in the C++ API of OpenCV are defined within the cv namespace. You have two ways to access them. First, precede the main function's definition with the following declaration:
using namespace cv;Alternatively, prefix all OpenCV class and function names with the namespace specification, that is, cv::, as we will do in this book. The use of this prefix makes the OpenCV classes and functions easier to identify within your code.
The highgui module contains a set of functions that allow you to easily visualize and interact with your images. When you load an image with the imread function, you also have the option to read it as a gray-level image. This is very advantageous since several computer vision algorithms require gray-level images. Converting an input color image on the fly as you read it will save you time and minimize your memory usage. This can be done as follows:
// read the input image as a gray-scale image image= cv::imread("puppy.bmp", cv::IMREAD_GRAYSCALE);This will produce an image made of unsigned bytes (unsigned char in C++) that OpenCV designates with the constant CV_8U. Alternatively, it is sometimes necessary to read an image as a three-channel color image even if it has been saved as a gray-level image. This can be achieved by calling the imread function with a positive second argument:
// read the input image as a 3-channel color image image= cv::imread("puppy.bmp", cv::IMREAD_COLOR);This time, an image made of 3 bytes per pixel will be created and designated as CV_8UC3 in OpenCV. Of course, if your input image has been saved as a gray-level image, all three channels will contain the same value. Finally, if you wish to read the image in the format in which it has been saved, then simply input a negative value as the second argument. The number of channels in an image can be checked using the channels method:
std::cout << "This image has " << image.channels() << " channel(s)";Pay attention when you open an image with imread without specifying a full path (as we did here). In such a case, the default directory will be used. When you run your application from the console, this directory is obviously the current console's directory. However, if you run the application directly from your IDE, the default directory will most often be the one that contains your project file. Consequently, make sure that your input image file is located in the right directory.
When you use imshow to display an image made up of integers (designated as CV_16U for 16-bit unsigned integers or as CV_32S for 32-bit signed integers), the pixel values of this image will be divided by 256 first. This is done in an attempt to make it displayable with 256 gray shades. Similarly, an image made up of floating points will be displayed by assuming a range of possible values between 0.0 (displayed as black) and 1.0 (displayed as white). Values outside this defined range are displayed in white (for values above 1.0) or black (for values below 0.0).
The highgui module is very useful to build quick prototypal applications. When you are ready to produce a finalized version of your application, you will probably want to use the GUI module offered by your IDE in order to build an application with a more professional look.
Here, our application uses both input and output images. As an exercise, you should rewrite this simple program such that it takes advantage of the function's in-place processing, that is, by not declaring the output image and writing it instead:
cv::flip(image,image,1); // in-place processingThere's more...
The highgui module contains a rich set of functions that help you interact with your images. Using these, your applications can react to mouse or key events. You can also draw shapes and write text on images.
Clicking on images
You can program your mouse to perform specific operations when it is over one of the image windows you created. This is done by defining an appropriate callback function. A callback function is a function that you do not explicitly call but which is called by your application in response to specific events (here, the events that concern the mouse interacting with an image window). To be recognized by applications, callback functions need to have a specific signature and must be registered. In the case of a mouse event handler, the callback function must have the following signature:
void onMouse( int event, int x, int y, int flags, void* param);The first parameter is an integer that is used to specify which type of mouse event has triggered the call to the callback function. The other two parameters are simply the pixel coordinates of the mouse location when the event has occurred. The flags are used to determine which button was pressed when the mouse event was triggered. Finally, the last parameter is used to send an extra parameter to the function in the form of a pointer to any object. This callback function can be registered in the application through the following call:
cv::setMouseCallback("Original Image", onMouse, reinterpret_cast<void*>(&image));In this example, the onMouse function is associated with the image window called Original Image, and the address of the displayed image is passed as an extra parameter to the function. Now, if we define the onMouse callback function as shown in the following code, then each time the mouse is clicked, the value of the corresponding pixel will be displayed on the console (here, we assume that it is a gray-level image):
void onMouse( int event, int x, int y, int flags, void* param) { cv::Mat *im= reinterpret_cast<cv::Mat*>(param); switch (event) { // dispatch the event case cv::EVENT_LBUTTONDOWN: // left mouse button down event // display pixel value at (x,y) std::cout << "at (" << x << "," << y << ") value is: " << static_cast<int>( im->at<uchar>(cv::Point(x,y))) << std::endl; break; } }Note that in order to obtain the pixel value at (x,y), we used the at method of the cv::Mat object; this is discussed in Chapter 2 , Manipulating Pixels. Other possible events that can be received by the mouse event callback function include cv::EVENT_MOUSEMOVE, cv::EVENT_LBUTTONUP, cv::EVENT_RBUTTONDOWN, and cv::EVENT_RBUTTONUP.
Drawing on images
OpenCV also offers a few functions to draw shapes and write text on images. The examples of basic shape-drawing functions are circle, ellipse, line, and rectangle. The following is an example of how to use the circle function:
cv::circle(image, // destination image cv::Point(155,110), // center coordinate 65, // radius 0, // color (here black) 3); // thicknessThe cv::Point structure is often used in OpenCV methods and functions to specify a pixel coordinate. Note that here we assume that the drawing is done on a gray-level image; this is why the color is specified with a single integer. In the next recipe, you will learn how to specify a color value in the case of color images that use the cv::Scalar structure. It is also possible to write text on an image. This can be done as follows:
cv::putText(image, // destination image "This is a dog.", // text cv::Point(40,200), // text position cv::FONT_HERSHEY_PLAIN, // font type 2.0, // font scale 255, // text color (here white) 2); // text thicknessCalling these two functions on our test image will then result in the following screenshot:
Note that you have to include the top-level module header opencv2/imgproc.hpp for these examples to work.
See also
Exploring the cv::Mat data structure
In the previous recipe, you were introduced to the cv::Mat data structure. As mentioned, this is a key component of the library. It is used to manipulate images and matrices (in fact, an image is a matrix from a computational and mathematical point of view). Since you will be using this data structure extensively in your application development processes, it is imperative that you become familiar with it. Notably, in this recipe, you will learn that this data structure incorporates an elegant memory management mechanism.
How to do it...
Let's write the following test program that will allow us to test the different properties of the cv::Mat data structure:
#include <iostream> #include <opencv2/core.hpp> #include <opencv2/highgui.hpp> // test function that creates an image cv::Mat function() { // create image cv::Mat ima(500,500,CV_8U,50); // return it return ima; } int main() { // create a new image made of 240 rows and 320 columns cv::Mat image1(240,320,CV_8U,100); cv::imshow("Image", image1); // show the image cv::waitKey(0); // wait for a key pressed // re-allocate a new image image1.create(200,200,CV_8U); image1= 200; cv::imshow("Image", image1); // show the image cv::waitKey(0); // wait for a key pressed // create a red color image // channel order is BGR cv::Mat image2(240,320,CV_8UC3,cv::Scalar(0,0,255)); // or: // cv::Mat image2(cv::Size(320,240),CV_8UC3); // image2= cv::Scalar(0,0,255); cv::imshow("Image", image2); // show the image cv::waitKey(0); // wait for a key pressed // read an image cv::Mat image3= cv::imread("puppy.bmp"); // all these images point to the same data block cv::Mat image4(image3); image1= image3; // these images are new copies of the source image image3.copyTo(image2); cv::Mat image5= image3.clone(); // transform the image for testing cv::flip(image3,image3,1); // check which images have been affected by the processing cv::imshow("Image 3", image3); cv::imshow("Image 1", image1); cv::imshow("Image 2", image2); cv::imshow("Image 4", image4); cv::imshow("Image 5", image5); cv::waitKey(0); // wait for a key pressed // get a gray-level image from a function cv::Mat gray= function(); cv::imshow("Image", gray); // show the image cv::waitKey(0); // wait for a key pressed // read the image in gray scale image1= cv::imread("puppy.bmp", CV_LOAD_IMAGE_GRAYSCALE); image1.convertTo(image2,CV_32F,1/255.0,0.0); cv::imshow("Image", image2); // show the image cv::waitKey(0); // wait for a key pressed return 0; }Run this program and take a look at the images it produces:
How it works...
The cv::Mat data structure is essentially made up of two parts: a header and a data block. The header contains all of the information associated with the matrix (size, number of channels, data type, and so on). The previous recipe showed you how to access some of the attributes of this structure contained in its header (for example, by using cols, rows, or channels). The data block holds all the pixel values of an image. The header contains a pointer variable that points to this data block; it is the data attribute. An important property of the cv::Mat data structure is the fact that the memory block is only copied when explicitly requested for. Indeed, most operations will simply copy the cv::Mat header such that multiple objects will point to the same data block. This memory management model makes your applications more efficient while avoiding memory leaks, but its consequences need to be understood. The examples of this recipe illustrate this fact.
By default, the cv::Mat objects have a zero size when they are created, but you can also specify an initial size as follows:
// create a new image made of 240 rows and 320 columns cv::Mat image1(240,320,CV_8U,100);In this case, you also need to specify the type of each matrix element-CV_8U here, which corresponds to 1-byte pixel (grayscale) images. The U letter here means it is unsigned. You can also declare signed numbers using S. For a color image, you would specify three channels (CV_8UC3). You can also declare integers (signed or unsigned) of size 16 and 32 (for example, CV_16SC3). You also have access to 32-bit and 64-bit floating-point numbers (for example, CV_32F).
Each element of an image (or a matrix) can be composed of more than one value (for example, the three channels of a color image); therefore, OpenCV has introduced a simple data structure that is used when pixel values are passed to functions. This is the cv::Scalar structure, which is generally used to hold one or three values. For example, to create a color image initialized with red pixels, write the following code:
// create a red color image // channel order is BGR cv::Mat image2(240,320,CV_8UC3,cv::Scalar(0,0,255));Similarly, the initialization of the gray-level image could have also been done using this structure by writing cv::Scalar(100).
The image size often needs to be passed to functions as well. We have already mentioned that the cols and rows attributes can be used to get the dimensions of a cv::Mat instance. The size information can also be provided through the cv::Size structure that simply contains the height and width of the matrix. The size() method allows you to obtain the current matrix size. This is the format that is used in many methods where a matrix size must be specified.
For example, an image could be created as follows:
// create a non-initialized color image cv::Mat image2(cv::Size(320,240),CV_8UC3);The data block of an image can always be allocated or reallocated using the create method. When an image has already been previously allocated, its old content is deallocated first. For reasons of efficiency, if the new proposed size and type matches the already existing size and type, then no new memory allocation is performed:
// re-allocate a new image // (only if size or type are different) image1.create(200,200,CV_8U);When no more references point to a given cv::Mat object, the allocated memory is automatically released. This is very convenient because it avoids the common memory leak problems often associated with dynamic memory allocation in C++. This is a key mechanism in OpenCV (introduced in version 2) that is accomplished by having the cv::Mat class implement reference counting and shallow copy. Therefore, when an image is assigned to another one, the image data (that is, the pixels) is not copied; both images will point to the same memory block. This also applies to images either passed or returned by a value. A reference count is kept such that the memory will be released only when all the references to the image are destructed or assigned to another image:
// all these images point to the same data block cv::Mat image4(image3); image1= image3;Any transformation applied to one of the preceding images will also affect the other images. If you wish to create a deep copy of the content of an image, use the copyTo method. In this case, the create method is called on the destination image. Another method that produces a copy of an image is the clone method, which creates a new identical image as follows:
// these images are new copies of the source image image3.copyTo(image2); cv::Mat image5= image3.clone();In the example of this recipe, we applied a transformation to image3. The other images also contain this image; some of them share the same image data, while others hold a copy of this image. Check the displayed images and find out which ones were affected by the image3 transformation.
If you need to copy an image into another image that does not necessarily have the same data type, use the convertTo method:
// convert the image into a floating point image [0,1] image1.convertTo(image2,CV_32F,1/255.0,0.0);In this example, the source image is copied into a floating-point image. The method includes two optional parameters: a scaling factor and an offset. Note that both the images must, however, have the same number of channels.
The allocation model for the cv::Mat objects also allows you to safely write functions (or class methods) that return an image:
cv::Mat function() { // create image cv::Mat ima(240,320,CV_8U,cv::Scalar(100)); // return it return ima; }We can also call this function from our main function as follows:
// get a gray-level image cv::Mat gray= function();If we do this, the gray variable will then hold the image created by the function without extra memory allocation. Indeed, as we explained, only a shallow copy of the image will be transferred from the returned cv::Mat instance to the gray image. When the ima local variable goes out of scope, this variable is deallocated. However, since the associated reference counter indicates that its internal image data is being referred to by another instance (that is, the gray variable), its memory block is not released.
It's worth noting that in the case of classes, you should be careful and not return image class attributes. Here is an example of an error-prone implementation:
class Test { // image attribute cv::Mat ima; public: // constructor creating a gray-level image Test() : ima(240,320,CV_8U,cv::Scalar(100)) {} // method return a class attribute, not a good idea... cv::Mat method() { return ima; } };Here, if a function calls the method of this class, it obtains a shallow copy of the image attributes. If this copy is modified later, the class
