Mastering openFrameworks: Creative Coding Demystified E-Book

Denis Perevalov

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 461

Veröffentlichungsjahr: 2013

Beschreibung

openFrameworks is a powerful programming toolkit and library designed to assist the creative process through simplicity and intuitiveness. It's a very handy software library written in C++ to reduce the software development process, helping you to kick-start creative coding. With the help of C++ and shaders support, openFrameworks allows for the processing of all kinds of media information with your custom-developed algorithms at the lowest possible level, with the fastest speed.

"Mastering openFrameworks: Creative Coding Demystified" will introduce you to a world of creative coding projects, including interactive installations, audio-visual, and sound art projects. You will learn how to make your own projects using openFrameworks. This book focuses on low-level data processing, which allows you to create really unique and cutting-edge installations and projects.

"Mastering openFrameworks: Creative Coding Demystified" provides a complete introduction to openFrameworks, including installation, core capabilities, and addons. Advanced topics like shaders, computer vision, and depth cameras are also covered.

We start off by discussing the basic topics such as image and video loading, rendering and processing, playing sound samples, and synthesizing new sounds. We then move on to cover 3D graphics, computer vision, and depth cameras. You will also learn a number of advanced topics such as video mapping, interactive floors and walls, video morphing, networking, and using geometry shaders.

You will learn everything you need to know in order to create your own projects; create projects of all levels, ranging from simple creative-code experiments, to big interactive systems consisting of a number of computers, depth cameras, and projectors.

Leseprobe

Table of Contents

Mastering openFrameworks: Creative Coding Demystified

Credits

Foreword

About the Author

Acknowledgement

About the Reviewers

www.PacktPub.com

Support files, eBooks, discount offers and more

Why Subscribe?

Free Access for Packt account holders

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Downloading the color graphics PDF

Errata

Piracy

Questions

1. openFrameworks Basics

About openFrameworks

Use cases

Installing openFrameworks and running your first example

Installing on Windows

Microsoft Visual Studio

Code::Blocks (Windows)

Installing on Mac OS with Xcode

Installing on Linux with Code::Blocks

openFrameworks' folders

The examples folder

The apps folder

The addons folder

File structure of a project

Code structure of a project

main.cpp

testApp.h

testApp.cpp

setup()

update()

draw()

Other functions

Creating a new project

Creating a project from an existing example

Creating a project using Project Generator

Creating your first project – the Pendulum example

Running the book's examples

Basic utility functions

Summary

2. Drawing in 2D

Drawing basics

The background color of the screen

Pulsating background example

Geometric primitives

The simplest example of a flower

Controlling the drawing of primitives

Using ofPoint

Operations with points

Using control points example

Coordinate system transformations

Flower with petals example

Colors

Operations with colors

Color modifications example

Drawing with an uncleared background

Using FBO for offscreen drawings

Spirals example

Playing with numerical instability

Screen grabbing

Additional topics

Summary

3. Building a Simple Particle System

The basics of particle systems

Interaction types

Computing particles' physics

Rendering particles

Creating a particle system – summary

A single particle

Control parameters

Defining the particle functions

Implementing a particle in the project

An emitter

The attraction, repulsion, and spinning forces

Graphical user interface

Additional topics

Summary

4. Images and Textures

Raster and vector images

Loading and drawing an image

Rotating images

Color modulation

Transparency

Creating and modifying images

Creating images

Modifying images

Working with the color of a single pixel

A simple geometrical distortion example

The functions for manipulating the image as a whole

Using ofTexture for memory optimization

Image warping and video mapping

Using images for internal calculations

An image as a mask

An image as a palette

Summary

5. Working with Videos

Video basics

Playing a video file

Controlling the video playback

Processing a single video frame

The vertical lines image example

The replacing colors example

Processing multiple frames

Radial slit-scan example

Horizontal slit-scan

Discussing color interpolation

Processing a live video from the camera

The video synthesizer example

Using image sequence

Using image sequence example

Summary

6. Working with Sounds

Sound basics

Playing sound samples

The bouncing ball example

The singing voices example

Generating sounds

The PWM synthesis example

Image-to-sound transcoder example

Using a microphone

The loop sampler example

Saving a recorded sample to the file

Getting spectral data from sound

Dancing cloud example

Summary

7. Drawing in 3D

3D basics

Representation of 3D objects

3D scene rendering

Simple 3D drawing

The triangles cloud example

Using ofMesh

Enabling lighting and setting normals

Computing normals using the setNormals() function

Drawing sharp edges

Drawing line segments and points

Drawing line segments

Drawing points

Coloring the vertices

Texturing

Working with vertices

The oscillating plane example

The twisting knot example

Additional topics

Summary

8. Using Shaders

Basics of shaders

Types of shaders

When to use shaders

Structure of a shader's code

A simple fragment shader example

Creating the fragment shader

The vertex shader

Embedding shaders in our project

Debugging shaders

Creating video effects with fragment shaders

Passing a float parameter to a shader

A simple geometrical distortion example

Passing the float array to the shader

Using Perlin noise in shaders

A liquify distortion example

Processing several images

A masking example

An audio-reactive project example

Deforming objects with a vertex shader

Vertex shader

Fragment shader

Using vertex shader in our project

Using a geometry shader

The furry carpet example

Additional topics

Summary

9. Computer Vision with OpenCV

Understanding computer vision and OpenCV

Using ofxOpenCv

Working with ofxCv images

Image initializing

Algebraic operations with images

Drawing functions

Access to pixels

Working with color planes and color spaces conversion

Motion detection from movies

Discussing the algorithm's parameters

Motion detection from live video

Image filtering

The image filtering example

Geometrical transformations of images

Perspective distortion removing example

Searching for objects in an image

Using the ofxCvContourFinder class for finding contours

An example for searching bright objects in video

Using OpenCV functions

Optical flow

Video morphing example

Using optical flow for morphing

Applying morphing to another image

Summary

10. Using Depth Cameras

Depth camera basics

Active infrared stereo cameras

Installing the ofxOpenNI addon

ofxOpenNI examples

Working with examples of depth images

Hand-tracking examples

User tracking examples

Creating interactive surface

Running the project

Additional topics

Summary

11. Networking

Networking basics

Using OSC protocol

Sending data

Receiving data

Typical schemes of OSC usage

Using TCP protocol for streaming images

The streaming images example

Summary

A. Working with Addons

Addons basics

Addons in openFrameworks

Installing a non-core addon

Linking addons to a new project

Using Project Generator

Linking an addon to an existing project

List of selected addons

Summary

B. Perlin Noise

Perlin noise basics

Using the ofNoise() function

Space-coherent noise

Summary

Index

Mastering openFrameworks: Creative Coding Demystified

Mastering openFrameworks: Creative Coding Demystified

Copyright © 2013 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: September 2013

Production Reference: 1160913

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham B3 2PB, UK.

ISBN 978-1-84951-804-8

www.packtpub.com

Cover Image by Asher Wishkerman (<[email protected]>)

Credits

Author

Denis Perevalov

Reviewers

Mathias Paumgarten

Tim Pulver

Acquisition Editor

Nikhil Karkal

Lead Technical Editor

Sweny M. Sukumaran

Technical Editors

Sharvari Baet

Aparna Kumari

Hardik B. Soni

Copy Editors

Aditya Nair

Brandt D'Mello

Gladson Monteiro

Adithi Shetty

Project Coordinators

Shiksha Chaturvedi

Hardik Patel

Proofreader

Mario Cecere

Indexer

Priya Subramani

Production Coordinator

Arvindkumar Gupta

Cover Work

Arvindkumar Gupta

Foreword

Confuсius said, "Choose a job you love, and you will never have to work a day in your life". Probably this piece of wisdom is meant right for you. Maybe you have already found your vocation. Maybe you are just looking for it. Anyway there's something that made you take this book. If so, let me tell you a little story.

My name is Igor Tatarnikov, also known as Sodazot. I'm an artist, although not by my education, but rather by my way of life. I keep on looking for a new and interesting occupation to take up. I've had different hobbies. I've tried a lot of jobs.

Three years ago I didn't even think about the interactive video, although my major occupation at that time was making video clips. I used to employ the stop-motion technique combined with computer animation. I also did some filming. By the by I grew more and more interested in the live VJ performances and generative video. I began experimenting with different techniques and posted my clips on the Internet.

At the same time there appeared the first available depth camera - Microsoft Kinect on offer. After it, things went crazy. Hundreds of video clips with reviews and tests of this camera's possibilities appeared every day on the Net. I took an interest in it all and kept track of the news. Watching the possibilities displayed I got a lot of new creative ideas. And finally my friends presented me the gadget. Of course I was happy, but at first I lost my head and didn't know what to do with it.

Also, it was not long since one person had written me an e-mail asking for permission to use my experimental video clip in his lecture for the students. I gladly agreed and since then we've been communicating by e-mail. It happened that this man lived in Ekaterinburg and we were thousands of kilometers apart.

As we communicated, we found a lot of common interests and decided to do something together. My head was boiling with ideas. I drew lots of sketches, shared them all with my friend, and he knew how to put them to life using the openFrameworks toolkit. We took counsel and the discussions brought us even more new ideas.

So we created our first commercial project for a special event at one of the Moscow cinemas. It was a funny video installation, where the picture responded to the visitor's movements. Coming up to the screen, a visitor saw himself as a funny character. The visitors of the event liked it very much and our big customers were happy—they played before the screen of the installation like children. We enjoyed the result of the work and it inspired us to create something new.

We've been working together for two years since and we've made several interesting and successful projects, using openFrameworks as our main tool. This is how our visual laboratory Kuflex was created. Our projects now grow more complicated and interesting, and our team became more and more numerous. We also work with musicians, artists, dancers, architects, and we have recently created a project with a real symphony orchestra.

If you still haven't guessed, I'll tell you that the friend with whom everything started is the author of this book Denis Perevalov.

The content of the book is based on our experience of creating interactive installations and performances. It teaches you the openFrameworks' multimedia capabilities and the principles of their usage for building interactive projects, which work with video, 3D graphics, sound, and cameras. The core of the book consists of the real working examples of projects for openFrameworks. Some of them are based on our works, the others were designed exclusively for this book. Besides, across the text you will find hints that will help you avoid many pitfalls in the practical use of openFrameworks.

Study openFrameworks, invent, and create your own projects and soon you'll realize that you can implement practically everything you can imagine using it.

Igor (Sodazot) Tatarnikov,

Artist

About the Author

Denis Perevalov is a computer vision research scientist. He works at the Institute of Mathematics and Mechanics of the Ural Branch of the Russian Academy of Sciences (Ekaterinburg, Russia). He is the co-author of two Russian patents on robotics computer vision systems and an US patent on voxel graphics. Since 2010 he has taught openFrameworks in the Ural Federal University. From 2011 he has been developing software for art and commercial interactive installations at kuflex.com using openFrameworks. He is the co-founder of interactive technologies laboratory expo32.ru (opened in 2012).

Acknowledgement

I would like to thank my family—wife Svetlana and son Timofey for their patience and suggestions. And many thanks to my parents and grandparents for great pirozhki, which was a necessary part of the book-writing process.

My creative coding experience and the desire to write the book appeared, thankfully, to artist Igor Sodazot, who invented and designed most of the interactive installations, which I program for him using openFrameworks. He is the coauthor of most of the book's examples and its video/audio contents.

Thanks to my scientific supervisor Victor Borisovich Kostousov for expending so many efforts to shape my scientific style of thinking and writing.

I would like to thank my colleagues working at interactive media art, experimental music, and dance fields—Prof. Yoichi Nagashima, Tatyana Komarova, Ekaterina Zharinova, and my first curator Ksenia Fedorova, for their teaching and influence.

And big thanks to my friends and scientific colleagues for supporting me and helping me with ideas: Nikolay Mikhalev, Sergey Samuraev, Kirill Kostousov, Fedor Kornilov, Elizaveta Sayfutdinova, and Prof. Pavel Konstantinovich Kuznetzov.

This book would be impossible without hard work on proof-reading by Angelina Poptzova, and technical reviewing by Mathias Paumgarten and Tim Pulver.

Thanks to Packt Publishing, who made this book possible.

Thanks to the openFrameworks' creators and openFrameworks community for developing this amazing toolkit.

All the book's examples were developed together with Igor Sodazot, except the Dancing cloud example, which is based on the idea of nCode installation by Andrey Krel, Igor Sodazot, Denis Perevalov, and Pavel Tikhonenko (2011, Moscow).

All the video and audio content for the examples were made by/with Igor Sodazot, except:

Image sunflower.png, which was contributed by ©iStockphoto.com/Andrew Johnson

Music track surface.wav, which was contributed by Ilya Orange (soundcloud.com/ilyaorange)

About the Reviewers

Mathias Paumgarten is a creative developer from Austria. He is currently living and working in Santa Monica, California.

Starting with a background in Flash development, Mathias found his passion for code-driven animation at a very young age. Over the years while working for and at several agencies he has broadened his skillset by leaving the web platform and working on installations using low-level languages such as C/C++.

After graduating with a Bachelor's degree at the University of Applied Sciences, Salzburg, Austria, he decided to leave Austria while focusing on modern web technologies such as HTML5 and JavaScript, currently working as a frontend JavaScript developer.

Mathias has worked for several renowned agencies such as B-Reel, Soap Creative, and Firstborn working on projects for Sony, Fox Entertainment, Pepsi Co., and many more.

After receiving recognitions such as FWA and other awards, Mathias has also contributed to publications such as HTML5 Games Most Wanted.

Tim Pulver is an interaction design student from Potsdam, Germany. As a teenager he was fascinated by the demo scene and how people were able to transform code into something beautiful. He now uses Processing, openFrameworks, and Arduino to create interactive installations, data-visualizations, and user-interface prototypes.

One of his recent projects is interactive fulldome data visualization, where users can playfully explore global crop production.

In another project, Tim wrote a program that translated an image of an eye based on its structure into unique jewelry, which was printed out using a 3D printer.

He likes the idea of sharing and free culture. In 2011, he founded the electronic music netlabel Yarn Audio, which supports sharing and remixing of released music. All the cover artwork for this netlabel has been generated using custom made tools, too. You can contact him at http://www.timpulver.de.

I would like to thank my family for their support and Hanna Schatz, Paul Vollmer, Kim Albrecht, Fabian Althaus, and Martin von Lupin for great collaboration.

www.PacktPub.com

Support files, eBooks, discount offers and more

You might want to visit www.PacktPub.com for support files and downloads related to your book.

Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.

At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

http://PacktLib.PacktPub.com

Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books.

Why Subscribe?

Fully searchable across every book published by PacktCopy and paste, print and bookmark contentOn demand and accessible via web browser

Free Access for Packt account holders

If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access.

Preface

openFrameworks is a simple and powerful C++ toolkit designed to develop real-time projects with focus on generating and processing graphics and sound. Nowadays, this is a popular platform for experiments in generative and sound art and creating interactive installations and audiovisual performances.

Mastering openFrameworks: Creative Coding Demystified covers programming openFrameworks 0.8.0 for Windows, Mac OS X, and Linux. It provides a complete introduction to openFrameworks, including installation, core capabilities, and addons. Advanced topics like shaders, computer vision, and depth cameras are also covered.

You will learn everything you need to know to create your own projects, ranging from simple generative art experiments to big interactive systems consisting of a number of computers, depth cameras, and projectors.

This book focuses on low-level data processing, which allows you to create really unique and cutting-edge works.

What this book covers

Chapter 1, openFrameworks Basics, covers installing openFrameworks, the structure of openFrameworks projects, and creating the pendulum-simulation project.

Chapter 2, Drawing in 2D, explains the basics of two-dimensional graphics, including drawing geometric primitives, working with colors and drawing in the offscreen buffer. It also contains a generative art example of using numerical instability for drawing.

Chapter 3, Building a Simple Particle System, teaches the basics of particle system modeling and drawing. By the end of this chapter, you will build a fully featured project that can be used as a sketch for further experiments with particles.

Chapter 4, Images and Textures, covers the principles of working with images, including loading images from file; rendering it on the screen with different sizes, color, and transparency; creating new images; and modifying existing images. It also touches the basics of image warping and video mapping.

Chapter 5, Working with Videos, covers basic and advanced topics on playing, layering, and processing videos, including playing video files, processing live video grabbed from a camera, and working with image sequences. This chapter contains an implementation of the slit-scan effect and a simple video synthesizer, which uses a screen-to-camera feedback loop to create vivid effects on prerecorded videos.

Chapter 6, Working with Sounds, explains how to play sound samples, synthesize new sounds, and get sounds from the microphone. It includes the project wherein we generate music using bouncing-ball simulation, the PWM synthesizer, and the image-to-sound transcoding. Finally, it teaches us how to use spectrum analysis for creating an audio-reactive visual project.

Chapter 7, Drawing in 3D, covers representing, modifying, and drawing 3D objects. It includes examples of drawing a sphere-shaped cloud of triangles, an oscillating surface, and a twisting 3D knot.

Chapter 8, Using Shaders, explains how to use fragment, vertex, and geometry shaders for creating 2D video effects and 3D object deformations.

Chapter 9, Computer Vision with OpenCV, teaches the basics of computer vision using the OpenCV library. It explains how to perform filtering and correct perspective distortions in images and how to look for motion areas and detect bright objects in the videos. It includes an advanced example of using optical flow for video morphing.

Chapter 10, Using Depth Cameras, covers using depth cameras in openFrameworks projects using the ofxOpenNI addon. It includes an example of the projector-camera interactive system, which lets us draw abstract images on the wall. The example can be used as a sketch for creating interactive walls, tables, and floors.

Chapter 11, Networking, covers how to use OSC and TCP protocols in your openFrameworks projects for creating distributed projects that run on several computers. It includes an image-streaming example.

Appendix A, Working with Addons, teaches the basic principles of addons, explains how to link addons to your projects, and discusses some of the most useful addons.

Appendix B, Perlin Noise, explains the principles of using Perlin noise, which is employed in many of the examples in the book.

What you need for this book

For working with this book's examples and creating your openFrameworks projects, you need a computer with the Windows, Mac OS X, or Linux operating system.

You will also need to install some development environment (Visual Studio C++ Express, Xcode, or Code::Blocks) and openFrameworks itself. In the first chapter of the book, you will find detailed instructions for installation. All required software are free.

Some examples can require additional equipment:

The video synthesizer example section in Chapter 5, Working with Videos, and the The streaming images example section in Chapter 11, Networking, need a webcam for grabbing live video. If you are using a laptop, it most probably has a built-in webcam.The loop sampler example section in Chapter 6, Working with Sounds, needs a microphone. If you are using a laptop, it most probably has a built-in microphone.The furry carpet example section in Chapter 8, Using Shaders, uses a geometry shader, and therefore needs a modern video card.The Creating interactive surface section in Chapter 10, Using Depth Cameras, needs a depth camera like Microsoft Kinect, Asus Xtion, or PrimeSense Carmine. Having a projector would be ideal, but is not compulsory.

Who this book is for

If you are a visual artist, designer, or programmer interested in creative coding with openFrameworks, this book is for you. Basic knowledge of programming, such as C++, Java, Python, or ActionScript, would be helpful.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Downloading the color graphics PDF

For downloading the colored graphics of this book visit: http://www.packtpub.com/sites/default/files/downloads/8048OS_ColoredImages.pdf

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the erratasubmissionform link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[email protected]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.

Chapter 1. openFrameworks Basics

In this chapter you will get acquainted with openFrameworks, learn its specifics and cases when you should use it. Also you will study how to install openFrameworks, run its examples, and make your first openFrameworks project:

Installing openFrameworks and running your first exampleFile structure of a projectCode structure of a projectCreating a first project – the Pendulum exampleRunning the book's examples

About openFrameworks

openFrameworks is an open source C++ toolkit for creative coding. It was initially released by Zachary Lieberman in 2005. Today openFrameworks is one of the main creative coding platforms, which is actively developed by Zachary Lieberman, Theodore Watson, and Arturo Castro with help from the openFrameworks community.

Note

The current openFrameworks' version is 0.8.0.

The toolkit is indebted to two significant precursors: the Processing development environment, created by Casey Reas, Ben Fry, and the Processing community; and the ACU Toolkit, a privately distributed C++ library developed by Ben Fry and others in the MIT Media Lab's Aesthetics and Computation Group.

openFrameworks' website is http://openframeworks.cc. It contains latest downloads, documentation, tutorials, and forums.

The main purpose of openFrameworks is to provide users with an easy access to multimedia, computer vision, networking, and other capabilities in C++ by gluing many open libraries into one package. Namely, it acts as a wrapper for libraries such as OpenGL, FreeImage, and OpenCV. The term wrapper means that openFrameworks provides you with new functions and classes, and gives hints on a project structure, but does not limit you. Namely, you can still use all of the C++ capabilities, and directly call functions from all of the linked libraries without using the wrapper's classes.

openFrameworks is cross-platform compatible with Windows, Mac OS X, Linux, iOS, and Android as the supported platforms. It means that if you develop a project for one of the platforms, you can copy the source files and compile the project for any other platform from the list. In the book we will cover developing a project for Windows, Mac OS X, and Linux only. Though many of the examples considered will work on mobile platforms too.

There are many great projects made with openFrameworks. Here are a few "classical" ones:

Funky Forest by Emily Gobeille and Theodore Watson, 2007 – the interactive forest installationBody Paint by Mehmet Akten, 2009 – drawing on the wall by moving the user's bodyHand from Above by Chris O'Shea, 2009 – outdoor installation working on a big billboard and interacting with pedestrians

Use cases

openFrameworks has the following architectural specifics:

Its core is based around multimedia, including 2D and 3D graphics, images, video, and sound. So openFrameworks is especially appropriate for developing multimedia projects working in real-time environments.It works using C++ language, which implies that the code is compiled into native machine instructions and hence works very fast. So it lets you create computing-intensive, ground-breaking projects, using the top capabilities of modern computing technologies.

Such specifics determine cases when you should and should not use openFrameworks for a project development.

You definitely can employ openFrameworks when:

You need to make a creative coding project, such as an interactive audio-visual installation or performance, which works with multimedia in a nontrivial and custom way. Namely, such a project would render a custom particle system, apply effects such as video morphing and slit-scan, or even perform data transcoding.You need to create a project, which performs intensive data analysis, for example, analyzing data from depth cameras.

Maybe you should not use openFrameworks when:

The project is centered on working with visual controls such as buttons, checkboxes, lists, and sliders. In this case the better option is in using developing platforms like QT, Cocoa, or .Net.

Note

openFrameworks contains a number of classes implementing visual controls like buttons and sliders, which are great for creating simple graphical user interfaces. But currently visual interface is not the main focus of the openFrameworks' evolution.

The project does not use multimedia or intensive computations a lot. For example, if you just want to send simple commands to a robot, it is definitely simpler to use Processing.

Tip

Though openFrameworks is an open source project, currently you can use it for developing commercial projects (see details in the openFrameworks license at http://www.openframeworks.cc/about/license.html). To protect the project's content, to add licensing, and to create an installer, you should use special additional software. Note that all of this software is included in iOS and Android development kits, so commercial developing for mobile platforms is quite easy.

Installing openFrameworks and running your first example

Now we will consider how to install openFrameworks in Windows, Mac OS X, and Linux, and execute one of the openFrameworks' examples.

Note

Historically (and currently) openFrameworks works best in Mac OS X. So when we use openFrameworks in interactive performances, we often do it in Mac OS X. Because performance is a short event (5 to 30 minutes), it is highly critical the software works as maximally fast and stable as it can.

For interactive installations, which are exhibited for a longer time (hours to months), and rare interruption in work is not so critical, we use any OS, depending on the available hardware.

The installation files and setup guides for all supported operating systems are located at http://www.openframeworks.cc/download/. Let's consider them in detail.

Installing on Windows

Within Windows 7 and 8 there are two choices of programming environments: Microsoft Visual Studio and Code::Blocks. Microsoft Visual Studio is one of the most popular environments in the world. It is very mature and stable. Many libraries (included and not included in openFrameworks) are adopted for Visual Studio. So if you are a novice in C++ programming, but you know that you need to link additional libraries in your project, Visual Studio is the best option for you.

Code::Blocks is a more lightweight environment and some developers prefer to use it. If you are a novice and just want to play with openFrameworks in simple projects, you should probably to do it with Code::Blocks.

Microsoft Visual Studio

The current version of Microsoft Visual Studio is 2012. In general Microsoft Visual Studio 2012 is a commercial software. But it has a lightweight free version, Microsoft Visual Studio Express, which is full enough for developing projects with openFrameworks.

The installation steps are as follows:

Install Microsoft Visual Studio Express 2012 for Windows Desktop. Navigate to http://microsoft.com, enter Visual Studio Express 2012 for Windows Desktop download in the search line, and press Enter to start searching. Click the Download button. When downloading page opens, press the Install now button. Then you will go through the process of installation. When installation is finished, run Visual Studio and register it (registration is free).Download the openFrameworks' archive. Go to http://www.openframeworks.cc/download/ and download the version for Visual Studio. The downloaded ZIP file should be named like of_v0.8.0_vs_release.zip. Unzip the downloaded file; it will be a folder containing openFrameworks. Move the folder to any location on your computer, for example, C:\openFrameworks.Let's compile and run some example to verify openFrameworks is working correctly. Navigate to the examples/3d/pointCloudExample openFrameworks folder and open pointCloudExample.sln in Visual Studio.Press F7 to compile the project.Press F5 to run the project.You will see an application window with a boy's face made from 3D points as shown in the following screenshot:Press the left mouse button and move the mouse to rotate the face, or press the right mouse button and move the mouse to scale the face.

Note

To understand the code of the example you need to learn how to work with images and 3D graphics, see Chapter 4, Images and Textures, and Chapter 7, Drawing in 3D. For creating your own point cloud, for example, from your own face, you need to learn how to work with depth cameras, see Chapter 10, Using Depth Cameras. Also check out the description of the project's files in the File structure of a project section.

Also, note the second window associated with the running application. It is colored in black and possibly contains some text. This is a console window, where openFrameworks writes various information, warnings, and errors. When you have some problems, check the contents of this window.Now press Esc for closing the application.

You have successfully run the openFrameworks example and now can develop your own projects.

Tip

If you try to run a project made with Visual Studio on some other computer, it will probably not start, and show an error message. The possible reason is that you need to install Visual Studio redistributables. To download it, go to http://microsoft.com, enter Visual C++ Redistributable for Visual Studio in the search line, and press Enter to start searching. Click on the first search result and download the installation file.

Before installing the redistributables, try to run your project. Maybe it will work.

Code::Blocks (Windows)

Code::Blocks is an open source development environment for Windows. The current version is 12.11. The installation steps are as follows:

Install Code::Blocks. Go to http://www.codeblocks.org, click on the Downloads menu item, and click on Download the binary release. Then select the file to download, which includes mingw. For example, codeblocks-12.11mingw-setup.exe.Install additions for Code::Blocks, which are needed for openFrameworks working. Go to http://www.openframeworks.cc/setup/codeblocks/ and see instructions in the Add files to MinGW section.Download the openFrameworks archive. Go to http://www.openframeworks.cc/download/ and download openFrameworks for Code::Blocks (Windows). The downloaded ZIP file should be named like of_v0.8.0_win_cb_release. Unzip the downloaded file; it will be a folder containing openFrameworks. Move the folder to any location on your computer, for example, C:\openFrameworks.Let's compile and run an example to verify openFrameworks is working correctly. Navigate to the examples/3d/pointCloudExample openFrameworks folder and open pointCloudExample.workspace in Code::Blocks.The first time you open Code::Blocks it will ask you which compiler you want to use. Be sure GNU GCC Compiler is selected and continue.Press the Build button and then the Run button:Follow the steps 6, 7, 8, and 9 from the Microsoft Visual Studio section.

Installing on Mac OS with Xcode

We will be using Mac OS X 10.8.4 and Xcode 4.6.3. The installation steps are as follows:

Install Xcode by downloading it from the Mac App Store.Download the openFrameworks archive. Go to http://www.openframeworks.cc/download/ and download openFrameworks for Xcode. The downloaded ZIP file should be named like of_v0.8.0_osx_release. Unzip the downloaded file; it will be a folder containing openFrameworks. Move the folder to any location on your computer, for example, on the desktop.Let's compile and run some examples to verify openFrameworks is working correctly. Navigate to the examples/3d/pointCloudExample openFrameworks folder and open pointCloudExample.xcodeproj in Xcode.The project in Xcode contains several schemes for building openFrameworks and the example itself. For compiling your project, you need to select the example's scheme. Check it out. If the current scheme is openFrameworks, you will see the following button:Click on it on the left-hand side and select the example project, pointCloudExample:Press Command + B to compile the project.Press the Run button.Follow the steps 6, 8, and 9 from the Microsoft Visual Studio section. (Considering step 7, describing console window, the separate console window does not appear in Mac OS X, and all such information is printed in the Xcode console.)

Tip

Downloading the color graphics PDF

For downloading the colored graphics of this book visit:

http://www.packtpub.com/sites/default/files/downloads/8048OS_ColoredImages.pdf

Installing on Linux with Code::Blocks

This section will guide you in installing openFrameworks for Code::Block on the Ubuntu version of Linux. The current version is 12.11.

Note

You can install openFrameworks not only on Ubuntu, but also on Debian and Fedora versions of Linux. See installation guides at http://www.openframeworks.cc/setup/linux-codeblocks/.

Also, you can use Eclipse development environment instead of Code::Blocks. See http://www.openframeworks.cc/setup/linux-eclipse/.

The installation steps are as follows:

Install Code::Blocks. In the main menu in Ubuntu, click on the Dash home icon, search for Ubuntu Software Center, and open it by selecting the Ubuntu Software Center icon. Search for Code Blocks here. The Code::Blocks program should be the first item listed. Click on the Install button and follow the instructions.Download the openFrameworks archive. Go to http://www.openframeworks.cc/download/ and download openFrameworks for Code::Blocks (Linux), for a 32- or 64-bit operating system. The downloaded ZIP file should be named like of_v0.8.0_linux_release.tar or of_v0.8.0_linux64_release.tar. Unzip the downloaded file; it will be a folder containing openFrameworks. Move the folder to any location on your computer.Now you should install openFrameworks by running some scripts from Terminal. Please refer to http://www.openframeworks.cc/setup/linux-codeblocks/ for detailed instructions.Follow the steps 4, 5, 6, and 7 from the Code::Blocks (Windows) section for installation steps on running an openFrameworks example.

openFrameworks' folders

Till now we have installed openFrameworks and checked its working. Let's explore the contents of its folder. It consists of a number of folders, including addons, apps, examples, libs, projectGenerator, and text files such as license.md and readme.txt.

Note

It is a good idea to read the license file license.md carefully before using openFrameworks.

Let's consider some of the most important folders in detail.

The examples folder

This folder contains simple examples covering almost all the aspects of openFrameworks, sorted by the following topics:

3d: This folder demonstrates the basics of 3D graphics and 3D math (see Chapter 7, Drawing in 3D, for more information on the topic)addons: This folder contains examples of using various addons, which are extensions of openFrameworks' core (see Appendix A, Working with Addons, for more information on the topic)communication: This folder contains examples of communicating with peripherals using serial port (most often via USB), for example, for connecting with Arduinoempty: This folder contains emptyExample, the simplest project for openFrameworks, which we will use as a starting point for developing most of the examples across the bookevents: This folder demonstrates built-in openFrameworks' event system usage, like key pressing and timing, and also working with custom event objectsgl: This folder contains advanced examples on 2D and 3D graphics, including FBO, VBO, and shaders (see Chapter 2, Drawing in 2D, Chapter 7, Drawing in 3D, and Chapter 8, Using Shaders, for detailed information on these topics)graphics: This folder demonstrates basic 2D graphics capabilities and working with images (see Chapter 2, Drawing in 2D, and Chapter 4, Images and Textures, for more information on these topics)gui: This folder exposes how to add graphical user interface, containing buttons, sliders, and checkboxes to an openFrameworks projectmath: This folder has examples on working with Perlin noise, simple particle system, and also with vector mathematics (see Appendix B, Perlin noise, and Chapter 3, Building a Simple Particle System, for more information on these topics)sound: This folder contains examples that demonstrate how to play, generate at low-level, and record sounds (see Chapter 6, Working with Sounds, for more information on the topic)utils: This folder demonstrates working with small but important topics such as converting values between different types (for example, converting int to string), working with directories, and using threadsvideo: This folder demonstrates how to play videos, process video frames, and grab live video from camera (see Chapter 5, Working with Videos, for more information on the topic)

Though most of the topics will be covered in this book, we highly recommend that you run and see as many examples of the code as you can. It is very helpful for dipping into the openFrameworks world.

The apps folder

This is a folder, in which all your projects should live. More specifically, you should place the projects inside some subfolder of the apps folder. By now this folder contains the myApps folder, and you can use it for your projects. Also you can create new folders, like Performances2014, and place corresponding projects there.

Note, if you will try to compile a project placed outside a subfolder of the apps folder, the compiler can give an error that it can not find the openFrameworks library.

The addons folder

This folder contains addons. These are the extensions of the basic openFrameworks capabilities. When you need to add some extension to openFrameworks, you will install it into this folder. We discuss the addons in detail in Appendix A, Working with Addons.

File structure of a project

All openFrameworks projects have a similar structure of folders and files. Let's consider this in detail by looking at the openFrameworks' pointCloudExample project.

Open the examples/3d/pointCloudExample folder. It consists of the following files and folders:

The bin folder contains an executable file of the project (maybe also a number of libraries in the .dll files will be there—it depends on your operating system). If you compile the project, as it is described in the Installing openFrameworks and running your first example section, most probably you will find there is an executable file named pointCloudExample_debug.

The _debug suffix means that the project was compiled in the Debug mode of compilation. This mode lets you debug the project by using breakpoints and other debugging tools.

Note

Breakpoint is a debugging tool, which pauses execution of the project in a specified line of code and lets you inspect the current values of the project's variables.

Projects compiled in the Debug mode can work very slowly. So when your project is working properly, always compile it in the Release mode of the compilation. Many examples in the book should be compiled in the Release mode for good performance. In this case, the executable file will be called without the _debug suffix, such as pointCloudExample.

Also, inside the bin folder you will find the data subfolder. This is a folder where all your contents should be located: images, videos, sounds, XML, and text files. openFrameworks projects use this folder as a default place for loading and saving the data.

In the considered project, this folder contains one image file named linzer.png.

Note

This image consists of pixels, which hold red, green, blue, and alpha color components (red, green, blue, alpha). Each image's pixel (x, y) is transformed into a 3D point (x, y, z) with color (r, g, b, a), so that the point's third coordinate z is calculated from the alpha value. As a result we obtain the resultant 3D point cloud, which is drawn on the screen. (See Chapter 4, Images and Textures, for details on getting the pixels' colors from the images.)

The src folder contains C++ source codes for your project. Often, source codes are represented in just three files: main.cpp, testApp.h, and testApp.cpp. We will consider these files a bit later, in the Code structure of a project section. Note that this folder can contain other .h and .cpp files of your project.

Also the project's folder contains a special project file for your development environment. It has extension .sln (Visual Studio), .xcodeproj (Xcode), or .workspace (Code::Blocks). This is the file which you should open in your development environment in order to edit, compile, and run the project. (In the considered example it has the name pointCloudExample.sln, pointCloudExample.xcodeproj, or pointCloudExample.workspace.)

Additionally, the project's folder can contain some other files, for example, the current project settings (the set of files depend on the development environment).

Code structure of a project

Source codes of an openFrameworks' project are placed in the project's src folder and consist of at least three files: main.cpp, testApp.h, and testApp.cpp.

Note

Remember the following convention: if some function or class name begins with of, it means that it belongs to openFrameworks. Examples are ofPoint, ofImage, and ofSetColor(). (If some name begins with ofx, it means that it is part of some openFrameworks addon, for example, ofxXmlSettings.)

main.cpp

In C++ language specification each project must have a .cpp file with the defined main() function. This function is an entry point for an operating system to start the application. In openFrameworks, the main() function is contained in the main.cpp file. The most important line of the function is the following:

ofSetupOpenGL( &window, 1024, 768, OF_WINDOW );

This ofSetupOpenGL() function calling instructs openFrameworks that you need to create a window for visual output with the width 1024 and height 768 pixels. The last parameter OF_WINDOW means that you need to create a window, which the user can move and resize on the desktop screen. If you specify the last parameters as OF_FULLSCREEN, the project will run at full screen—such a mode is important for many projects.

For example, if you need to show your project on the full screen with dimensions 1920 x 1024 pixels, you can do it by replacing the ofSetupOpenGL() call with the following line:

ofSetupOpenGL( &window, 1920, 1024, OF_FULLSCREEN );

Normally you need not change the main.cpp file at all, because the settings of screen size can be done in the testApp.cpp text, which we consider now.

Tip

Be careful! Inside the main() function most of the openFrameworks objects such as ofImage do not work properly, because paths and other variables are not set yet. So, indeed, in most cases you should keep main.cpp untouched and do all you need in testApp.cpp.

testApp.h

This file begins with #pragma once. This is a compiler directive, which should be present at the beginning of all the .h files. The next line is #include "ofMain.h". It includes openFrameworks' core classes and functions. After this, the code contains declaration of the testApp class, which is inherited from the openFrameworks' ofBaseApp class:

#pragma once #include "ofMain.h" class testApp : public ofBaseApp{ public: //openFrameworks' standard functions declarations void setup(); void update(); void draw(); //... //Declarations of custom objects for the project ofEasyCam cam; ofMesh mesh; ofImage img; };

The testApp class contains a number of functions, setup(), update(), draw(), and some others. These are the functions required for your project to work. They are defined in the ofBaseApp class and called by openFrameworks. (The linking of the testApp class to the openFrameworks engine is done within the main() function. Its last line creates an object of this class and links it to the window, controlled by openFrameworks.) We will describe the meaning of the functions in the next section.

In the end of the class definition you will see declarations of the cam, mesh, and img objects. These are custom objects defined just in this example. In your own projects, you should add declarations of your objects here too.

Note

For simplicity you can declare objects right in the testApp.cpp file, but be careful, objects of some classes like ofEasyCam, ofThread, and ofxTCPServer will not work properly and can cause the application to crash if defined as static variables not belonging to the testApp class. The reason is that openFrameworks performs some actions before the testApp class' object is created, and such classes rely on this. Note that in some examples of the book we sometimes use such declarations for simple types (float, int, ofPoint, ofImage, and others).

Let's sum up: when creating your own project you should keep declarations of the setup(), update(), draw() functions, and others untouched, and also add your objects' and functions' declarations, which are needed for your project.

testApp.cpp

The testApp.cpp file contains definitions of all functions, declared in testApp.h. Let's explain the standard functions of the testApp class.

The most important functions are setup(), update(), and draw(). setup() is called first, and then update() and draw() are called in an infinite cycle, until the user presses the Esc key to close the project:

Note

Besides pressing Esc, to finish the projects' execution, the user can just close the projects' window.

If you need the project to terminate itself, call the OF_EXIT_APP( val ) function with some integer value val.

Let's consider these functions in detail.

setup()

The setup() function is called by openFrameworks just once, at the start of the project. This is the best place for setting screen parameters such as refresh rate, load images and videos, and start processes like camera grabbing.

The typical functions for controlling screen parameters are the following:

ofSetFrameRate( rate ): This parameter sets the frame rate of screen refresh equal to the value rate of type int. Also, it controls the rate of calling update() and draw(). The typical value is 60, which corresponds to the frame rate of most TVs and projectors. The default value is zero, which means that the frame rate is as large as possible (in some cases it is unwanted).ofSetVerticalSync( v ): This parameter enables or disables synchronization of screen refresh with the video card's physical refresh, with v of typebool. Enabling this mode improves the quality of a fast-moving object's rendering, but slightly decreases the performance. By default the synchronization is enabled.ofSetFullscreen( v ): This parameter enables or disables full screen mode, with v of type bool.ofSetWindowShape( w, h ): This parameter sets the size of the output window so that the drawing area will have size width w and height h pixels.

Note that you can call these functions from other functions of the testApp class too.

update()

This function is called by openFrameworks right after the setup() call. This is the place where all computations should be performed, like changing positions of objects, analyzing data from cameras, and network exchange.

Tip

Also, drawing into offscreen buffers (FBOs) can be done here.

draw()

This function is called by openFrameworks after update(). All drawing functions should be placed here. After draw(), openFrameworks again calls update(), so we obtain a cycle of the update() and draw() methods.

The typical drawing functions are as follows:

ofSetBackground( r, g, b ), where r, g, and b are integer values from 0 to 255, specifying red, green, and blue components of screen backgroundofSetColor( r, g, b ) sets the drawing colorofLine( x1, y1, x2, y2 ) draws a line segment connecting points (x1, y1) and (x2, y2)

Other functions

The testApp.cpp file contains definitions of other functions, declared in testApp.h. These are event-driven functions; openFrameworks calls them when some event occurs, like mouse moving or keyboard pressing. Some of the most important functions are the following:

The keyPressed( key ) and keyReleased( key ) functions are called by openFrameworks when some key is pressed or released. Here key is an int value, which can be compared with char values like 'a', and with constants denoting special keys like OF_KEY_RETURN for the Return (Enter) key, OF_KEY_LEFT for the left cursor key, and so on. See the full list of special keys constants in the libs/openFrameworks/utils/ofConstants.h file.The mouseMoved( x, y ) function is called when the mouse is moved over the project's window without pressing any keys. Here x and y are the mouse pointer coordinates in pixels, with the center of the coordinates in the top-left corner of the window.The mouseReleased( x, y, button ), mouseDragged( x, y, button ), and mousePressed( x, y, button ) functions are called when a mouse button is pressed, when the mouse is moving, and when the mouse button is released, respectively. Here button equals to 0, 1, and 2 for left, center, and right mouse buttons respectively.The windowResized( w, h ) function is called when the size of the window is changed by the user or by calling the ofSetWindowShape() function. Here w and h are equal to the current width and height of the window.

Now we will discuss the ways for creating a new openFrameworks project.

Creating a new project

For developing your projects you have two possibilities: start it from an existing example or create it using the Project Generator wizard. Let's discuss both in detail.

Creating a project from an existing example

The easiest way to start your own project is to just copy some existing examples (or your own project) into the apps/myApps folder, or any other subfolder of the apps folder. Then rename the folder as you want (for example, to myInteractiveWall), and open the project using your developing environment. Now you can change the code, and run the project.

This way, almost all the examples in the book are made from the emptyExample project, located in the examples/empty folder.

Such an approach is indeed very easy. Also, it is especially useful for working with some complicated addons such as ofxOpenNI (see Chapter 10, Using Depth Cameras, for more information on this addon). Namely, you can have difficulties with linking such addons by yourself or with projectGenerator. In this case, just start your project from the existing working addon's example.

Creating a project using Project Generator

You can create a new project using the Project Generator wizard, located in the projectGenerator