39,59 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Create Deep Learning and Reinforcement Learning apps for multiple platforms with TensorFlow
Key Features
- Build TensorFlow-powered AI applications for mobile and embedded devices
- Learn modern AI topics such as computer vision, NLP, and deep reinforcement learning
- Get practical insights and exclusive working code not available in the TensorFlow documentation
Book Description
As a developer, you always need to keep an eye out and be ready for what will be trending soon, while also focusing on what's trending currently. So, what's better than learning about the integration of the best of both worlds, the present and the future? Artificial Intelligence (AI) is widely regarded as the next big thing after mobile, and Google's TensorFlow is the leading open source machine learning framework, the hottest branch of AI. This book covers more than 10 complete iOS, Android, and Raspberry Pi apps powered by TensorFlow and built from scratch, running all kinds of cool TensorFlow models offline on-device: from computer vision, speech and language processing to generative adversarial networks and AlphaZero-like deep reinforcement learning. You’ll learn how to use or retrain existing TensorFlow models, build your own models, and develop intelligent mobile apps running those TensorFlow models. You'll learn how to quickly build such apps with step-by-step tutorials and how to avoid many pitfalls in the process with lots of hard-earned troubleshooting tips.
What you will learn
- Classify images with transfer learning
- Detect objects and their locations
- Transform pictures with amazing art styles
- Understand simple speech commands
- Describe images in natural language
- Recognize drawing with Convolutional Neural Network and Long Short-Term Memory
- Predict stock price with Recurrent Neural Network in TensorFlow and Keras
- Generate and enhance images with generative adversarial networks
- Build AlphaZero-like mobile game app in TensorFlow and Keras
- Use TensorFlow Lite and Core ML on mobile
- Develop TensorFlow apps on Raspberry Pi that can move, see, listen, speak, and learn
Who this book is for
If you're an iOS/Android developer interested in building and retraining others' TensorFlow models and running them in your mobile apps, or if you're a TensorFlow developer and want to run your new and amazing TensorFlow models on mobile devices, this book is for you. You'll also benefit from this book if you're interested in TensorFlow Lite, Core ML, or TensorFlow on Raspberry Pi.
Jeff Tang fell in love with classical AI more than two decades ago. After his MS in CS, he worked on Machine Translation for 2 years and then, to survive the long AI winter, he worked on enterprise apps, voice apps, web apps, and mobile apps at startups, AOL, Baidu, and Qualcomm. He developed a top-selling iOS app with millions of downloads and was recognized by Google as a Top Android Market Developer. He reconnected with modern AI in 2015 and knew that AI will be his passion and commitment for the next two decades. One of his favorite topics is to make AI available anytime anywhere and hence the book.Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 431
Veröffentlichungsjahr: 2018
Ähnliche
Intelligent Mobile Projects with TensorFlow
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Kunal ChaudhariAcquisition Editor: Larissa PintoContent Development Editor: Flavian VazTechnical Editor: Akhil NairCopy Editor: Safis EditingProject Coordinator: Devanshi DoshiProofreader:Safis EditingIndexer: Rekha NairGraphics: Jason MonteiroProduction Coordinator: Aparna Bhagat
First published: May 2018
Production reference: 1160518
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78883-454-4
www.packtpub.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Foreword
The past decade has seen the explosion of both machine learning and smartphones; today, these technologies are finally merging, and the result is an incredible variety of applications that you would have dismissed as far future Science Fiction just a few years ago. Think about it: you have already become accustomed to talking to your phone, asking it for directions, or telling it to schedule an appointment in your agenda. Your phone's camera tracks faces and recognizes objects. Games are becoming more interesting and challenging as the bots gets smarter and smarter. And countless apps use some form of artificial intelligence under the hood, in less obvious ways, such as recommending content that you will enjoy, anticipating your next trips to tell you when to leave, suggesting what to type next, and so on.
Until recently, all the intelligence happened on the server side, which meant that the user had to be connected to the internet, ideally with a fast and stable connection. The latency and service disruptions that this implied were show-stoppers for many applications. But today the intelligence is right there in the palm of your hand, thanks to tremendous hardware improvements and better Machine Learning libraries.
Most importantly, these technologies are now completely democratized: virtually any software engineer can learn to code an intelligent mobile application based on deep neural networks, using TensorFlow, Google's powerful and open source deep learning library. Jeff Tang's great and unique book will show you how to develop on-device TensorFlow-powered iOS, Android, and Raspberry Pi apps by guiding you through many concrete examples with step-by-step tutorials and hard-earned troubleshooting tips: from image classification, object detection, image captioning, and drawing recognition to speech recognition, forecasting time series, generative adversarial networks, reinforcement learning, and even building intelligent games using AlphaZero — the improved technology built on top of AlphaGo that beat Lee Sedol and Ke Jie, the world champions of the game of Go.
This is going to be a super popular book. It's such an important topic, and it's hard to get good reliable information. So roll up your sleeves, you have an exciting journey ahead of you! What intelligent mobile application will you build?
Aurélien Géron Former lead of YouTube's video classification team and author of the book Hands-On Machine Learning with Scikit-Learn and TensorFlow (O'Reilly, 2017) Paris, May 11th, 2018
Contributors
About the author
Jeff Tang fell in love with classical AI more than two decades ago. After his MS in CS, he worked on Machine Translation for 2 years and then, to survive the long AI winter, he worked on enterprise apps, voice apps, web apps, and mobile apps at startups, AOL, Baidu, and Qualcomm. He developed a top-selling iOS app with millions of downloads and was recognized by Google as a Top Android Market Developer. He reconnected with modern AI in 2015 and knew that AI will be his passion and commitment for the next two decades. One of his favorite topics is to make AI available anytime anywhere and hence the book.
About the reviewers
Pete Warden is the technical lead of the mobile and embedded TensorFlow group in Google's Brain team.
Amita Kapoor is an associate professor in the department of electronics, SRCASW, University of Delhi since 1996. She is the recipient of the prestigious DAAD fellowship to pursue a part of her research work in Karlsruhe Institute of Technology, Germany. She had been awarded the Best Presentation Award at International Conference Photonics 2008. She is a member of different professional bodies and has more than 40 publications in international journals and conferences. Her present research areas include ML, AI, Neural Networks, Robotics, Buddhism, and Ethics in AI.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Intelligent Mobile Projects with TensorFlow
Dedication
Packt Upsell
Why subscribe?
PacktPub.com
Foreword
Contributors
About the author
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
When to read the book
Download the example code files
Conventions used
Get in touch
Reviews
Getting Started with Mobile TensorFlow
Setting up TensorFlow
Setting up TensorFlow on MacOS
Setting up TensorFlow on GPU-powered Ubuntu
Setting up Xcode
Setting up Android Studio
TensorFlow Mobile vs TensorFlow Lite
Running sample TensorFlow iOS apps
Running sample TensorFlow Android apps
Summary
Classifying Images with Transfer Learning
Transfer learning – what and why
Retraining using the Inception v3 model
Retraining using MobileNet models
Using the retrained models in the sample iOS app
Using the retrained models in the sample Android app
Adding TensorFlow to your own iOS app
Adding TensorFlow to your Objective-C iOS app
Adding TensorFlow to your Swift iOS app
Adding TensorFlow to your own Android app
Summary
Detecting Objects and Their Locations
Object detection–a quick overview
Setting up the TensorFlow Object Detection API
Quick installation and example 
Using pre-trained models
Retraining SSD-MobileNet and Faster RCNN models
Using object detection models in iOS
Building TensorFlow iOS libraries manually
Using TensorFlow iOS libraries in an app
Adding an object detection feature to an iOS app
Using YOLO2–another object-detection model
Summary
Transforming Pictures with Amazing Art Styles
Neural Style Transfer – a quick overview
Training fast neural-style transfer models
Using fast neural-style transfer models in iOS
Adding and testing with fast neural transfer models
Looking back at the iOS code using fast neural transfer models
Using fast neural-style transfer models in Android
Using the TensorFlow Magenta multi-style model in iOS
Using the TensorFlow Magenta multi-style model in Android
Summary
Understanding Simple Speech Commands
Speech recognition – a quick overview
Training a simple commands recognition model
Using a simple speech recognition model in Android
Building a new app using the model
Showing model-powered recognition results
Using a simple speech recognition model in iOS with Objective-C
Building a new app using the model
Fixing model-loading errors with tf_op_files.txt
Using a simple speech recognition model in iOS with Swift
Summary
Describing Images in Natural Language
Image captioning – how it works
Training and freezing an image captioning model
Training and testing caption generation
Freezing the image captioning model
Transforming and optimizing the image captioning model
Fixing errors with transformed models
Optimizing the transformed model
Using the image captioning model in iOS
Using the image captioning model in Android
Summary
Recognizing Drawing with CNN and LSTM
Drawing classification – how it works
Training, predicting, and preparing the drawing classification model
Training the drawing classification model
Predicting with the drawing classification model
Preparing the drawing classification model
Using the drawing classification model in iOS
Building custom TensorFlow library for iOS
Developing an iOS app to use the model
Using the drawing classification model in Android
Building custom TensorFlow library for Android
Developing an Android app to use the model
Summary
Predicting Stock Price with RNN
RNN and stock price prediction – what and how
Using the TensorFlow RNN API for stock price prediction
Training an RNN model in TensorFlow
Testing the TensorFlow RNN model 
Using the Keras RNN LSTM API for stock price prediction
Training an RNN model in Keras
Testing the Keras RNN model 
Running the TensorFlow and Keras models on iOS
Running the TensorFlow and Keras models on Android
Summary
Generating and Enhancing Images with GAN
GAN – what and why
Building and training GAN models with TensorFlow
Basic GAN model of generating handwritten digits
Advanced GAN model of enhancing image resolution
Using the GAN models in iOS
Using the basic GAN model
Using the advanced GAN model
Using the GAN models in Android
Using the basic GAN model
Using the advanced GAN model
Summary
Building an AlphaZero-like Mobile Game App
AlphaZero – how does it work?
Training and testing an AlphaZero-like model for Connect 4
Training the model
Testing the model
Looking into the model-building code
Freezing the model
Using the model in iOS to play Connect 4
Using the model in Android to play Connect 4
Summary
Using TensorFlow Lite and Core ML on Mobile
TensorFlow Lite – an overview
Using TensorFlow Lite in iOS
Running the example TensorFlow Lite iOS apps
Using a prebuilt TensorFlow Lite model in iOS
Using a retrained TensorFlow model for TensorFlow Lite in iOS
Using a custom TensorFlow Lite model in iOS
Using TensorFlow Lite in Android
Core ML for iOS – an overview
Using Core ML with Scikit-Learn machine learning 
Building and converting the Scikit Learn models
Using the converted Core ML models in iOS
Using Core ML with Keras and TensorFlow
Summary
Developing TensorFlow Apps on Raspberry Pi
Setting up Raspberry Pi and making it move
Setting up Raspberry Pi 
Making Raspberry Pi move
Setting up TensorFlow on Raspberry Pi
Image recognition and text to speech
Audio recognition and robot movement
Reinforcement learning on Raspberry Pi
Understanding the CartPole simulated environment
Starting with basic intuitive policy 
Using neural networks to build a better policy
Summary
Final words
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Artificial Intelligence (AI), the simulation of human intelligence in computers, has a long history. Since its official birth in 1956, AI has experienced several booms and busts. The ongoing AI resurgence, or the new AI revolution, started in 2012 with the breakthrough in deep learning, a branch of machine learning that is now the hottest branch of AI because of deep learning, when a deep convolutional neural network (DCNN) won the ImageNet Large-Scale Visual Recognition Challenge with an error rate of only 16.4%, compared to the second best non-DCNN entry with an error rate of 26.2%. Since 2012, improved DCNN-based entries have won the ImageNet challenge every year, and deep learning technology has been applied to many hard AI problems beyond computer vision, such as speech recognition, machine translation, and the game of Go, resulting in one breakthrough after another. In March 2016, Google DeepMind's AlphaGo, built with deep reinforcement learning, beat 18-time human world Go champion Lee Sedol 4:1. At Google I/O 2017, Google announced that they're shifting from mobile-first to AI-first world. Other leading companies such as Amazon, Apple, Facebook, and Microsoft have all invested heavily in AI and launched many AI-powered products.
TensorFlow is Google's open source framework for building machine learning AI apps. Since its initial release in November 2015, when there were already several popular open source deep learning frameworks, TensorFlow has quickly become the most popular open source deep learning framework in less than 2 years. New TensorFlow models to tackle all kinds of tasks that'd require human or even superhuman intelligence have been built on a weekly basis. Dozens of books on TensorFlow have been published. More online blogs, tutorials, courses, and videos on TensorFlow have been made available. It's obvious that AI and TensorFlow are hot, but why another book with the word "TensorFlow" in its title?
This is a unique book, and the first one that combines TensorFlow-powered AI with mobile, connecting the world of the brightest future with the world of most prosperous present. We have all witnessed and experienced the iOS and Android smart phone revolution in the past decade, and we're just starting the AI revolution that'll likely have an even deeper impact on the world around us. What can be better than a theme that integrates the best of the two worlds, a book that shows how to build TensorFlow AI apps on mobile, anytime, anywhere?
It's true that you can build AI apps using many cloud AI APIs that exist out there, and sometimes it makes sense. However, running AI apps completely on mobile devices have the benefits that you can run the apps even when no network connectivity is available, when you can't afford the round trip to a cloud server, or when users don't want to send the data on their phones to anyone else.
It's also true that there're already a few example TensorFlow iOS and Android apps in the TensorFlow open source project that can get you started with mobile TensorFlow. However, if you have ever tried to run a cool TensorFlow model that amazes you on your iOS or Android device, you'll most likely stumble upon many hiccups before you can see the model successfully running on your phone.
This book can save you a lot of time and effort by showing you how to solve all the common problems you may encounter when running TensorFlow models on mobile. You'll get to see more than 10 complete TensorFlow iOS and Android apps built from scratch in the book, running all kinds of cool TensorFlow models, including the latest and coolest Generative Adversarial Network (GAN) and AlphaZero-like models.
Who this book is for
If you're an iOS and/or Android developer interested in building and retraining others' cool TensorFlow models and running them in your mobile apps, or if you're a TensorFlow developer and want to run your new amazing TensorFlow models on mobile devices, the book is for you. If you're interested in TensorFlow Lite, Core ML, or TensorFlow on Raspberry Pi, you'll also benefit from the book.
What this book covers
Chapter 1, Getting Started with Mobile TensorFlow, discusses how to set up TensorFlow on Mac and Ubuntu and NVIDIA GPU on Ubuntu and how to set up Xcode and Android Studio. We'll also discuss the difference between TensorFlow Mobile and TensorFlow Lite and when you should use them. Finally, we'll show you how to run the sample TensorFlow iOS and Android apps.
Chapter 2, Classifying Images with Transfer Learning, covers what is transfer learning and why you should use it, how to retrain the Inception v3 and MobileNet models for more accurate and faster dog breed recognition, and how to use the retrained models in sample iOS and Android apps. Then, we'll show you how to add TensorFlow to your own iOS app, both in Objective-C and Swift, and your own Android app for dog breed recognition.
Chapter 3, Detecting Objects and Their Locations, gives a quick overview of Object Detection, and then shows you how to set up the TensorFlow Object Detection API and use it to retrain SSD-MobileNet and Faster RCNN models. We'll also show you how to use the models used in the example TensorFlow Android app in your iOS app by manually building the TensorFlow iOS library to support non-default TensorFlow operations. Finally, we'll show you how to train YOLO2, another popular object detection model, which is also used in the example TensorFlow Android app, and how to use it in your iOS app.
Chapter 4, Transforming Pictures with Amazing Art Styles, first gives an overview of neural style transfer with their rapid progress in the last few years. Then, it shows you how to train fast neural style transfer models and use them in iOS and Android apps. After that, we'll cover how to use the TensorFlow Magenta multi-style model in your own iOS and Android apps to easily create amazing art styles.
Chapter 5, Understanding Simple Speech Commands, outlines speech recognition and shows you how to train a simple speech commands recognition model. We'll then show you how to use the model in Android as well as in iOS using both Objective-C and Swift. We'll also cover more tips on how to fix possible model loading and running errors on mobile.
Chapter 6, Describing Images in Natural Language, describes how image captioning works, and then it covers how to train and freeze an image captioning model in TensorFlow. We'll further discuss how to transform and optimize the complicated model to get it ready for running on mobile. Finally, we'll offer complete iOS and Android apps using the model to generate natural language description of images.
Chapter 7, Recognizing Drawing with CNN and LSTM, explains how drawing classification works, and discusses how to train, predict, and prepare the model. Then, we'll show you how to build another custom TensorFlow iOS library to use the model in a fun iOS doodling app. Finally, we'll show you how to build a custom TensorFlow Android library to fix a new model loading error and then use the model in your own Android app.
Chapter 8, Predicting Stock Price with RNN, takes you through RNN and how to use it to predict stock prices. Then, we'll inform you of how to build an RNN model with the TensorFlow API to predict stock prices, and how to build a RNN LSTM model with the easier-to-use Keras API to achieve the same goal. We'll test and see whether such models can beat a random buy or sell strategy. Finally, we'll show you how to run the TensorFlow and Keras models in both iOS and Android apps.
Chapter 9, Generating and Enhancing Images with GAN, gives an overview of what GAN is and why it has such great potential. Then, it oultines how to build and train a basic GAN model that can be used to generate human-like handwritten digits and a more advanced model that can enhance low resolution images to high resolution ones. Finally, we'll cover how to use the two GAN models in your iOS and Android apps.
Chapter 10, Building AlphaZero-like Mobile Game App, begins with how the latest and coolest AlphaZero works, and how to train and test a AlphaZero-like model to play a simple but fun game called Connect 4 in Keras with TensorFlow as backend. We'll then show you the complete iOS and Android apps to use the model and play the game Connect 4 on your mobile devices.
Chapter 11, Using TensorFlow Lite and Core ML on Mobile, demonstrates TensorFlow Lite and then shows you how to use a prebuilt TensorFlow model, a retrained TensorFlow model for TensorFlow Lite, and a custom TensorFlow Lite model in iOS. We'll also show you how to use TensorFlow Lite in Android. After that, we'll give an overview of Apple's Core ML and show you how to use Core ML with standard machine learning using Scikit-Learn. Finally, we'll cover how to use Core ML with TensorFlow and Keras.
Chapter 12, Developing TensorFlow Apps on Raspberry Pi, first looks at how to set up Raspberry Pi and make it move, and how to set up TensorFlow on Raspberry Pi. Then, we'll cover how to use the TensorFlow image recognition and audio recognition models, along with text to speech and robot movement APIs, to build a Raspberry Pi robot that can move, see, listen, and speak. Finally, we'll discuss in detail how to use OpenAI Gym and TensorFlow to build and train a powerful neural network-based reinforcement learning policy model from scratch in a simulated environment to make the robot learn to keep its balance.
To get the most out of this book
We recommend that you start with reading the first four chapters in order, along with running the accompanying iOS and Android apps available from the book's source code repository at http://github.com/jeffxtang/mobiletfbook. That'll help you ensure that you have the development environments all set up for TensorFlow mobile app development and that you know how to integrate TensorFlow into your own iOS and/or Android apps. If you're an iOS developer, you'll also learn how to use Objective-C or Swift with TensorFlow, and when and how to use the TensorFlow pod or the manual TensorFlow iOS library.
Then, if you need to build a custom TensorFlow Android library, go to Chapter 7, Recognizing Drawing with CNN and LSTM, and if you want to learn how to use a Keras model in your mobile app, check out Chapter 8, Predicting Stock Price with RNN, and Chapter 10, Building an AlphaZero-like Mobile Game App.
If you're more interested in TensorFlow Lite or Core ML, read Chapter 11, Using TensorFlow Lite and Core ML on Mobile, and if you're most interested in TensorFlow on Raspberry Pi, or reinforcement learning in TensorFlow, jump to Chapter 12, Developing TensorFlow Apps on Raspberry Pi.
Other than that, you can go through chapters 5 to 10 in order to see how to train different kinds of CNN, RNN, LSTM, GAN, and AlphaZero models and how to use them on mobile, maybe running the iOS and/or Android apps for each chapter before looking into the detailed implementation. Alternatively, you can jump directly to any chapter with the model you're most interested in; just be aware that a later chapter may refer to an earlier chapter for some duplicated details, such as steps of adding a TensorFlow custom iOS library to your iOS app, or fixing some model loading or running errors by building a TensorFlow custom library. However, rest assured that you won't be lost, or at least we've done our best to provide user-friendly and step-by-step tutorials, with occasional references to some steps of previous tutorials, to help you avoid all possible pitfalls you may encounter when building mobile TensorFlow apps, while also avoiding repeating ourselves.
When to read the book
AI, or its hottest branch machine learning, or its hottest subbranch deep learning, has enjoyed rapid progress in recent years. New releases of TensorFlow, backed by Google and with the most popular developer community of all open source machine learning frameworks, has also been launched at a faster speed. When we started writing the book in December 2017, the latest TensorFlow release was 1.4.0, released on November 2, 2017, and after that 1.5.0 was released on January 26, 2018, 1.6.0 on February 28, 2018, 1.7.0 on March 29, 2018, and 1.8.0 on April 27, 2018. All the iOS, Android, and Python code in the book has been tested with all those TensorFlow versions. Still, by the time you read the book, the latest TensorFlow version likely will be later than 1.8.0.
It turns out you don't need to worry too much about new releases of TensorFlow; the code in the book will most likely run seamlessly on the latest TensorFlow releases. During our testing run the apps on TensorFlow 1.4, 1.5, 1.6, 1.7, and 1.8, we've made no code changes at all. It's likely that in a later version, more TensorFlow operations will be supported by default, so you won't need to build a custom TensorFlow library, or you'll be able to build the custom TensorFlow library in a simpler way.
Of course, there's no guarantee that all the code will run without any changes in all future TensorFlow versions, but with all the detailed tutorials and troubleshooting tips covered in the book, no matter when you read the book, now or months later, you should have a smooth time reading the book and running the apps in the book with TensorFlow 1.4-1.8 or later.
Another concern is about the choice between TensorFlow Mobile and TensorFlow Lite. The book covers TensorFlow Mobile in most chapters (1 to 10). TensorFlow Lite may be the future of running TensorFlow on mobile, it's still in developer preview as of Google I/O 2018—that's why Google expects you to "use TensorFlow Mobile to cover production cases." Even after TensorFlow Lite is officially released, according to Google, "TensorFlow Mobile isn't going away anytime soon"—in fact, with the latest TensorFlow 1.8.0 version we tested before the book's publication, we found that using TensorFlow Mobile gets even simpler.
If the day that TensorFlow Lite fully replaces TensorFlow Mobile in all use cases, with Lite's better performance and smaller size, does finally come, the skills you'll learn from the book will only better prepare you for that day. In the meantime, before that unforeseeable future arrives, you can read the book and get to know how to use the big brother such as TensorFlow Mobile to run all those amazing and powerful TensorFlow models in your mobile apps.
Download the example code files
You can download the example code files for this book from your account at www.packtpub.com. If you purchased this book elsewhere, you can visit www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packtpub.com
.
Select the
SUPPORT
tab.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box and follow the onscreen instructions.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on the Packt GitHub account at https://github.com/PacktPublishing/Intelligent-Mobile-Projects-with-TensorFlow. In case there's an update to the code, it will be updated on the existing GitHub repository.
We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Getting Started with Mobile TensorFlow
This chapter covers how to get your development environments set up for building all the iOS or Android apps with TensorFlow that are discussed in the rest of the book. We won't discuss in detail all the supported TensorFlow versions, OS versions, Xcode, and Android Studio versions that can be used for development, as that kind of information can easily be found on the TensorFlow website (http://www.tensorflow.org) or via Google. Instead, we'll just talk briefly about sample working environments in this chapter so that we can dive in quickly to look at all the amazing apps you can build with the environments.
If you already have TensorFlow, Xcode, and Android Studio installed, and can run and test the sample TensorFlow iOS and Android apps, and if you already have an NVIDIA GPU installed for faster deep learning model training, you can skip this chapter. Or you can jump directly to the section that you're unfamiliar with.
We're going to cover the following topics in this chapter (how to set up the Raspberry Pi development environment will be discussed in Chapter 12, Developing TensorFlow Apps on Raspberry Pi):
Setting up TensorFlow
Setting up Xcode
Setting up Android Studio
TensorFlow Mobile vs TensorFlow Lite
Running sample TensorFlow iOS apps
Running sample TensorFlow Android apps
Setting up TensorFlow
TensorFlow is the leading open source framework for machine intelligence. When Google released TensorFlow as an open source project in November 2015, there were already several other similar open source frameworks for deep learning: Caffe, Torch, and Theano. By Google I/O 2018 on May 10, TensorFlow on GitHub has reached 99k stars, an increase of 14k stars in 4 months, while Caffe has increased only 2k to 24k stars. Two years later, it's already the most popular open source framework for training and deploying deep learning models (it also has good support for traditional machine learning). As of January 2018, TensorFlow has close to 85k stars (https://github.com/tensorflow/tensorflow) on GitHub, while the other three leading open source deep learning frameworks, Caffe (https://github.com/BVLC/caffe), CNTK (https://github.com/Microsoft/CNTK), and Mxnet (https://github.com/apache/incubator-mxnet) have over 22k, 13k, and 12k stars, respectively.
We assume you already have a basic understanding of TensorFlow, but if you don't, you should check out the Get Started (https://www.tensorflow.org/get_started) and Tutorials (https://www.tensorflow.org/tutorials) parts of the TensorFlow website or the Awesome TensorFlow tutorials (https://github.com/jtoy/awesome-tensorflow). Two good books on the topic are Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow, 2nd Edition by Sebastian Raschka and Vahid Mirjalili, and Hands-On Machine Learning with Scikit-Learn and TensorFlow: Concepts, Tools, and Techniques to Build Intelligent Systems by Aurélien Géron.
TensorFlow can be installed on MacOS, Ubuntu or Windows. We'll cover the steps to install TensorFlow 1.4 from its source on MacOS X El Capitan (10.11.6), macOS Sierra (10.12.6), and Ubuntu 16.04. If you have a different OS or version, you can refer to the TensorFlow Install (https://www.tensorflow.org/install) documentation for more information. By the time you read this book, it's likely a newer TensorFlow version will come out. Although you should still be able to run the code in the book with the newer version, it's not a guarantee, which is why we use the TensorFlow 1.4 release source code to set up TensorFlow on Mac and Ubuntu; that way, you can easily test run and play with the apps in the book.
Overall, we'll use TensorFlow on Mac to develop iOS and Android TensorFlow apps, and TensorFlow on Ubuntu to train deep learning models used in the apps.
Setting up TensorFlow on MacOS
Generally, you should use a virtualenv, Docker, or Anaconda installation to install TensorFlow in an isolated environment. But as we have to build iOS and Android TensorFlow apps using the TensorFlow source code, we might as well build TensorFlow itself from source, in which case, using the native pip installation choice could be easier than other options. If you want to experiment with different TensorFlow versions, we recommend you install other TensorFlow versions using one of the virtualenv, Docker, and Anaconda options. Here, we'll have TensorFlow 1.4 installed directly on your MacOS system using the native pip and Python 2.7.10.
Follow these steps to download and install TensorFlow 1.4 on MacOS:
Download the TensorFlow 1.4.0 source code (
zip
or
tar.gz
) from the TensorFlow releases page on GitHub:
https://github.com/tensorflow/tensorflow/releases
Uncompress the downloaded file and drag the
tensorflow-1.4.0
folder to your home directory
Make sure you have Xcode 8.2.1 or above installed (if not, read the
Setting up Xcode
section first)
Open a new Terminal window, then
cd tensorflow-1.4.0
Run
xcode-select --install
to install command-line tools
Run the following commands to install other tools and packages needed to build TensorFlow:
sudo pip install six numpy wheel brew install automake brew install libtool ./configure brew upgrade bazel
Build from the TensorFlow source with CPU-only support (we'll cover the GPU support in the next section) and generate a pip package file with the
.whl
file extension:
bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Install the TensorFlow 1.4.0 CPU package:
sudo pip install --upgrade /tmp/tensorflow_pkg/tensorflow-1.4.0-cp27-cp27m-macosx_10_12_intel.whl
If you encounter any error during the process, googling the error message, to be honest, should be the best way to fix it, as we intend in this book to focus on the tips and knowledge, not easily available elsewhere, gained from our long hours of building and debugging practical mobile TensorFlow apps. One particular error you may see is the Operation not permitted error when running the sudo pip install commands. To fix it, you can disable your Mac's System Integrity Protection (SIP) by restarting the Mac and hitting the Cmd + R keys to enter the recovery mode, then under Utilities-Terminal, running csrutil disable before restarting Mac. If you're uncomfortable with disabling SIP, you can follow the TensorFlow documentation to try one of the more complicated installation methods such as virtualenv.
If everything goes well, you should be able to run on your Terminal Window, Python or preferably IPython, then run import tensorflow as tf and tf.__version__ to see 1.4.0 as output.
Setting up TensorFlow on GPU-powered Ubuntu
One of the benefits of using a good deep learning framework, such as TensorFlow, is its seamless support for using a Graphical Processing Unit (GPU) in model training. It'd be a lot faster training a non-trivial TensorFlow-based model on a GPU than on a CPU, and currently NVIDIA offers the best and most cost-effective GPUs supported by TensorFlow. And Ubuntu is the best OS for running NVIDIA GPUs with TensorFlow. You can easily buy one GPU for a few hundred bucks and install it on an inexpensive desktop with an Ubuntu system. You can also install NVIDIA GPU on Windows but TensorFlow's support for Windows is not as good as that for Ubuntu.
To train the models deployed in the apps in this book, we use NVIDIA GTX 1070, which you can purchase on Amazon or eBay for about $400. There's a good blog by Tim Dettmers on which GPUs to use for deep learning (http://timdettmers.com/2017/04/09/which-gpu-for-deep-learning/). After you get such a GPU and install it on your Ubuntu system, and before you install the GPU-enabled TensorFlow, you need to install NVIDIA CUDA 8.0 (or 9.0) and the cuDNN (CUDA-Deep Neural Network library) 6.0 (or 7.0), both are supported by TensorFlow 1.4.
Follow these steps to install CUDA 8.0 and cuDNN 6.0 on Ubuntu 16.04 (you should be able to download and install CUDA 9.0 and cuDNN 7.0 in a similar way):
Find the NVIDIA CUDA 8.0 GA2 release at
https://developer.nvidia.com/cuda-80-ga2-download-archive
, and make the selections shown in the following screenshot:
Download the base installer as shown in the following screenshot:
Open a new Terminal and run the following commands (you'll also need to add the last two commands to your
.bashrc
file for the two environment variables to take effect next time you launch a new Terminal):
sudo dpkg -i /home/jeff/Downloads/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb sudo apt-get update sudo apt-get install cuda-8-0 export CUDA_HOME=/usr/local/cuda export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
Download the NVIDIA cuDNN 6.0 for CUDA 8.0 at
https://developer.nvidia.com/rdp/cudnn-download
– you'll be asked to sign up (for free) with an NVIDIA Developer account first before you can download it, as shown in
the next screenshot
(choose the highlighted
cuDNN v6.0 Library for Linux
):
Unzip the downloaded file, assuming it's under the default
~/Downloads
directory, and you'll see a folder named
cuda
, with two subfolders named
include
and
lib64
Copy the cuDNN
include
and
lib64
files to the CUDA_HOME's
lib64
and
include
folders:
sudo cp ~/Downloads/cuda/lib64/* /usr/local/cuda/lib64 sudo cp ~/Downloads/cuda/include/cudnn.h /usr/local/cuda/include
Now we're ready to install the GPU-enabled TensorFlow 1.4 on Ubuntu (the first two steps given here are the same as those described in the section Setting up TensorFlow on MacOS):
Download the TensorFlow 1.4.0 source code (
zip
or
tar.gz
) from the TensorFlow releases page on GitHub:
https://github.com/tensorflow/tensorflow/releases
Uncompress the downloaded file and drag the folder to your home directory
Download the bazel installer,
bazel-0.5.4-installer-linux-x86_64.sh
at
https://github.com/bazelbuild/bazel/releases
Open a new Terminal Window, then run the following commands to install the tools and packages needed to build TensorFlow:
sudo pip install six numpy wheel cd ~/Downloads chmod +x bazel-0.5.4-installer-linux-x86_64.sh ./bazel-0.5.4-installer-linux-x86_64.sh --user
Build from the TensorFlow source with GPU-enabled support and generate a pip package file with the
.whl
file extension:
cd ~/tensorflow-1.4.0 ./configure bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
Install the TensorFlow 1.4.0 GPU package:
sudo pip install --upgrade /tmp/tensorflow_pkg/tensorflow-1.4.0-cp27-cp27mu-linux_x86_64.whl
Now, if all goes well, you can launch IPython and enter the following scripts to see the GPU information TensorFlow is using:
In [1]: import tensorflow as tf In [2]: tf.__version__ Out[2]: '1.4.0' In [3]: sess=tf.Session() 2017-12-28 23:45:37.599904: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:892] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero 2017-12-28 23:45:37.600173: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties: name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.7845 pciBusID: 0000:01:00.0 totalMemory: 7.92GiB freeMemory: 7.60GiB 2017-12-28 23:45:37.600186: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
Congratulations! You're now ready to train the deep learning models used in the apps in this book. Before we start having fun with our new toy and use it to train our cool models and then deploy and run them on mobile devices, let's first see what it takes to be ready for developing mobile apps.
Setting up Xcode
Xcode is used to develop iOS apps and you need a Mac computer and a free Apple ID to download and install it. If your Mac is relatively older and with OS X El Capitan (version 10.11.6), you can download Xcode 8.2.1 at https://developer.apple.com/download/more. Or if you have macOS Sierra (version 10.12.6) or later, installed, you can download Xcode 9.2 or 9.3, the latest version as of May 2018, from the preceding link. All the iOS apps in the book have been tested in both Xcode 8.2.1, 9.2, and 9.3.
To install Xcode, simply double-click the downloaded file and follow the steps on the screen. It's pretty straightforward. You can now run apps on the iOS simulator that comes with Xcode or your own iOS device. Starting Xcode 7, you can run and debug your iOS apps on an iOS device for free, but if you want to distribute or publish your apps, you need to enroll in the Apple Developer Program for $99 a year as an individual: https://developer.apple.com/programs/enroll.
Although you can test run many apps in the book with the Xcode simulator, some apps in the book require the camera on your actual iOS device to take a picture before processing it with a deep learning model trained with TensorFlow. In addition, it's generally better to test a model on an actual device for accurate performance and memory usage: a model that runs fine in the simulator may crash or run too slow in an actual device. So it's strongly recommended or required that you test and run the iOS apps in the book on your actual iOS device at least once, if not always.
This book assumes you're comfortable with iOS programming, but if you're new to iOS development, you can learn from many excellent online tutorials such as the iOS tutorials by Ray Wenderlich (https://www.raywenderlich.com). We won't cover complicated iOS programming; we'll mainly show you how to use the TensorFlow C++ API in our iOS apps to run the TensorFlow trained models to perform all kinds of intelligent tasks. Both Objective-C and Swift code, the two official iOS programming languages from Apple, will be used to interact with the C++ code in our mobile AI apps.
Setting up Android Studio
Android Studio is the best tool for Android app development, and TensorFlow has great support for using it. Unlike Xcode, you can install and run Android Studio on Mac, Windows, or Linux. For detailed system requirements, see the Android Studio website (https://developer.android.com/studio/index.html). Here, we'll just cover how to set up Android Studio 3.0 or 3.0.1 on Mac – all the apps in the book have been tested on both versions.
First, download the Android Studio 3.0.1, or the latest version if it's newer than 3.0.1 and if you don't mind fixing possible minor issues, from the preceding link. You can also download 3.0.1 or 3.0 from its archives at https://developer.android.com/studio/archive.html.
Then, double-click the downloaded file and drag and drop the Android Studio.app icon to Applications. If you have a previously installed Android Studio, you'll get a prompt asking you if you want to replace it with the newer one. You can just select Replace.
Next, open Android Studio and you need to provide the path to the Android SDK, which by default is in ~/Library/Android/sdk if you have a previous version of Android Studio installed, or you can select Open an existing Android Studio project, then go to the TensorFlow 1.4 source directory created in the section Setting up TensorFlow on MacOS, and open the tensorflow/examples/android folder. After that, you can download the Android SDK by either clicking the link to an Install Build Tools message or going to Android Studio's Tools | Android | SDK Manager, as shown in the following screenshot. From the SDK Tools tab there, you can check the box next to a specific version of Android SDK Tools and click the OK button to install that version:
Finally, as TensorFlow Android apps use the native TensorFlow library in C++ to load and run TensorFlow models, you need to install the Android Native Development Kit (NDK), which you can do either from the Android SDK Manager shown in the preceding screenshot, or by downloading NDK directly from https://developer.android.com/ndk/downloads/index.html. Both the NDK version r16b and r15c have been tested to run the Android apps in the book. If you download the NDK directly, you may also need to set the Android NDK location after opening your project and selecting Android Studio's File | Project Structure, as shown in the following screenshot:
With both Android SDK and NDK installed and set up, you're ready to test run sample TensorFlow Android apps.
