29,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Fachliteratur
- Sprache: Englisch
Geospatial development links your data to places on the Earth’s surface. Its analysis is used in almost every industry to answer location type questions. Combined with the power of the Python programming language, which is becoming the de facto spatial scripting choice for developers and analysts worldwide, this technology will help you to solve real-world spatial problems.
This book begins by tackling the installation of the necessary software dependencies and libraries needed to perform spatial analysis with Python. From there, the next logical step is to prepare our data for analysis; we will do this by building up our tool box to deal with data preparation, transformations, and projections. Now that our data is ready for analysis, we will tackle the most common analysis methods for vector and raster data. To check or validate our results, we will explore how to use topology checks to ensure top-quality results. This is followed with network routing analysis focused on constructing indoor routes within buildings, over different levels.
Finally, we put several recipes together in a GeoDjango web application that demonstrates a working indoor routing spatial analysis application. The round trip will provide you all the pieces you need to accomplish your own spatial analysis application to suit your requirements.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 317
Veröffentlichungsjahr: 2015
Ähnliche
Table of Contents
Python Geospatial Analysis Cookbook
Python Geospatial Analysis Cookbook
Copyright © 2015 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: November 2015
Production reference: 1251115
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78355-507-9
www.packtpub.com
Credits
Author
Michael Diener
Reviewers
Jáchym Čepický
Richard Marsden
Commissioning Editor
Ashwin Nair
Acquisition Editor
Rebecca Youé
Content Development Editor
Athira Laji
Technical Editor
Chinmay S. Puranik
Copy Editor
Sonia Michelle Cheema
Project Coordinator
Bijal Patel
Proofreader
Safis Editing
Indexer
Hemangini Bari
Graphics
Disha Haria
Production Coordinator
Nilesh Mohite
Cover Work
Nilesh Mohite
About the Author
Michael Diener graduated from Simon Fraser University, British Columbia, Canada, in 2001 with a bachelor of science degree in geography. He began working in 1995 with Environment Canada as a GIS (Geographic Information Systems) analyst and has continued to work with GIS technologies ever since.
In 2008, he founded a company called GOMOGI that is focused on building web and mobile GIS application with open source tools. In 2011, the focus changed to indoor wayfinding and navigation solutions and building the indrz platform that Michael had envisioned.
From time to time, Michael also holds seminars for organizations wanting to explore or discover the possibilities of how GIS can increase productivity and help better answer spatial questions. He is also the creative head of new product development in his company. His technical skills include working with Python to solve a wide range of spatial problems on a daily basis. Through the years, he has developed many spatial applications with Python, including indrz and golfgis, which are two of the products built by his company, GOMOGI.
He is also lecturer of GIS at the Alpen Adria University, Klagenfurt, where he enjoys teaching students the wonderful powers of GIS and explaining how to solve spatial problems with open source GIS and Python.
I would like to dedicate this book to my family, beginning with my loving wife, Silvia, who encouraged and supported me through the writing of this book. I must also thank my three boys, Noah, Levi, and Jasper, for sacrificing many evenings and weekends of playtime for daddy’s book time. My parents, Birte and Hartwig, also deserve a big thank you for always believing in me—thank you for everything.
About the Reviewers
Jáchym Čepický is an open source GIS software consultant, developer, and user.
Richard Marsden has over 15 years of professional software development experience. After starting in the fields of geophysics and oil exploration, he has spent the last 10 years running the Winwaed Software Technology LLC independent software vendor. Winwaed specializes in geospatial tools and applications, including web applications, and operates the http://www.mapping-tools.com website for tools and add-ins for geospatial products such as Caliper Maptitude and Microsoft MapPoint.
Richard has been a technical reviewer for Python Geospatial Development and Python Geospatial Analysis Essentials both by Erik Westra, and Mastering Python Forensics by Dr. Michael Spreitzenbarth and Dr. Johann Uhrmann all by Packt Publishing.
www.PacktPub.com
Support files, eBooks, discount offers, and more
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at <[email protected]> for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www2.packtpub.com/books/subscription/packtlib
Do you need instant solutions to your IT questions? PacktLib is Packt’s online digital book library. Here, you can search, access, and read Packt’s entire library of books.
Why Subscribe?
Free Access for Packt account holders
If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view 9 entirely free books. Simply use your login credentials for immediate access.
Preface
Geospatial analysis is not special; it is just different when compared to other types of analysis such as financial market analysis. We work with geometry objects, such as lines, points, and polygons, and connect these geometries to attributes such as business data. We ask "where" question, such as "Where is the nearest pub?", "Where are all my customers located?", and "Where is my competition located?". The other location questions include, "Will this new building cast a shadow over the park?", "What is the shortest way to school?", "What is the safest way to school for my kids?", "Will this building block my view of the mountains?", and "Where is the optimal place to build my next store?". Identify the areas that fire trucks can reach from their station in 5 min, 10 min, or 20 min, and so on.
One thing all these questions have in common is the fact that you need to know where certain objects are located in order to answer them. Without the spatial component, you cannot answer such questions and this is what geospatial analysis is all about.
Geospatial features are laid over each other and patterns or trends are easily identified. This ability to see a pattern or trend is geospatial analysis in its simplest form.
Throughout this book, simple and complex code recipes are provided as small working models that can easily be integrated or expanded into a larger project or model.
Analysis is the fun part of GIS, and involves visualizing relationships, identifying trends, and seeing patterns that are not visible in a spreadsheet.
The Python programming language is clean, clear, and concise, making it great for beginners. It also has advanced powers for professionals to help them quickly code solutions to complex problems. Python makes visualization quick and easy for experts or beginners who work with geospatial data. It's that simple.
What this book covers
Chapter 1, Setting Up Your Geospatial Python Environment, explores setting up your computer to handle all software requirements in one go, such as pyproj, NumPy, and Shapely. All your development software needs to enable spatial analysis or geoprocessing on Windows and Linux are met in this chapter.
Chapter 2, Working with Projections, explains how to deal with spatial data that's projected or unprojected. You can learn and discover how to transform your data into a correct projection to prepare for an analysis.
Chapter 3, Moving Spatial Data from One Format to Another, explains how geospatial data comes in many different formats and also how messaging data from one format to another is a daily chore. In this chapter, you will find out about the most common data management tasks.
Chapter 4, Working with PostGIS, shows you how most of our geospatial data is stored in a spatial database and using, accessing, manipulating this data with Python is what this chapter is about.
Chapter 5, Vector Analysis, introduces a very common geospatial data format, that is, the vector data format. To execute analysis functions on vector data, we will explore patterns used to create new data by snapping, clipping, cutting, and overlaying vector datasets followed by determining the 3D ground distance and total elevation gain.
Chapter 6, Overlay Analysis, explains how to combine spatial data to create new data by using the process of overlaying two sets of data over each other.
Chapter 7, Raster Analysis, shows you how to create an elevation profile and quick ways to merge images to perform raster analysis functions on your data.
Chapter 8, Network Routing Analysis, shows you how finding the nearest anything is a common geospatial analysis feature. This chapter will disclose how to go about solving an indoor network type problem and demonstrate some common use cases for wayfinding inside buildings.
Chapter 9, Topology Checking and Data Validation, covers data quality and connections. In this chapter, you will learn how to verify your data for errors using custom topological functions.
Chapter 10, Visualizing Your Analysis, explains how geospatial data is inherently visual and you will learn about presenting your analysis on a web map and a 3D web.
Chapter 11, Web Analysis with GeoDjango, builds on Chapter 8, Network Routing Analysis, where you will create an indoor routing web application. You will easily be able to route a person from point A to point B within a building with real 3D network data. These key features will be presented by bringing together all the parts of the recipes you have learned so far.
Appendix A, Other Geospatial Python Libraries, explains how Python flourishes with geospatial libraries, and you will also find a listing of many popular libraries that are used for data analysis, regardless of whether they're spatial or not. This may trigger your interest.
Appendix B, Mapping Icon Libraries, quickly goes over the icon libraries out there that play a special role in the python geospatial working environment.
What you need for this book
To work with this book, you should be familiar with the programming language Python and the concepts involved in programming. This means that you should be able to install Python 2.7.x on your machine (Windows, Linux, or OS X) if it's not already installed. The concepts related to GIS (Geographic Information Systems) are definitely helpful but not necessary. A primer to this book could be Learning Geospatial Analysis with Python, Joel Lawhead or Python Geospatial Development, Eric Westra, both by Packt Publishing.
Who this book is for
If you are a student, teacher, programmer, geospatial or IT administrator, GIS analyst, researcher, or scientist looking to learn about spatial analysis, then this book is for you. Anyone trying to answer simple to complex spatial analysis questions will get a working demonstration of the power of Python with the help of real-world data. Some of you may be beginners but most of you will probably have a basic understanding of geospatial analysis and programming.
Sections
In this book, you will find several headings that appear frequently (Getting ready, How to do it..., How it works..., There's more..., and See also).
To give clear instructions on how to complete a recipe, we use these sections as follows:
Getting ready
This section tells you what to expect in the recipe, and describes how to set up any software or any preliminary settings required for the recipe.
How to do it…
This section contains the steps required to follow the recipe.
How it works…
This section usually consists of a detailed explanation of what happened in the previous section.
There's more…
This section consists of additional information about the recipe in order to make the reader more knowledgeable about the recipe.
See also
This section provides helpful links to other useful information for the recipe.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title through the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.
Downloading the color images of this book
We also provide you with a PDF file that has color images of the screenshots/diagrams used in this book. The color images will help you better understand the changes in the output. You can download this file from https://www.packtpub.com/sites/default/files/downloads/5079OS_ColorImage.pdf.
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website, or added to any list of existing errata, under the Errata section of that title.
Piracy
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
Questions
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.
Chapter 1. Setting Up Your Geospatial Python Environment
In this chapter, we will cover the following topics:
Introduction
This chapter will get the grunt work done for you so that you can freely and actively complete all the recipes in this book. We will start off by installing, each of the libraries you will be using, one by one. Once each step is completed, we will test each library installation to make sure it works. Since this book is directed toward those of you already working with spatial data, you can skip this chapter if you have it installed already. If not, you will find the installation instructions here useful as a reference.
The choice of Python libraries is based on industry-proven reliability and functionality. The plethora of functions in Python libraries has led to a flourishing GIS support on many top desktop GIS systems, such as QGIS and ESRI ArcGIS.
Also included in this book is an installer.sh bash file. The installer.sh file can be used to install the Python libraries that are available for your virtual environment from pip and other dependencies via the apt-get command. The installer.sh bash file is executed from the command line and installs almost everything in one go, so please take a look at it. For those of you who are starting with Python for the first time, follow the instructions in this chapter and your machine will be set up to complete different recipes.
Installations can sometimes be tricky even for advanced users, so you will find some of the most common pitfalls and hook-ups described in this chapter.
The development of these recipes was completed on a fresh Linux/Ubuntu 14.04 machine. Therefore, the code examples, if not otherwise specified, are Linux/Ubuntu-specific with Windows notes wherever necessary, unless otherwise specified.
Installing virtualenv and virtualenvwrapper
This recipe will enable you to manage different versions of different libraries for multiple projects. We use virtualenv to create virtual Python environments to host collections of project-specific libraries in an isolated directory. For example, you may have an old legacy project using Django 1.4, whereas a new project requires you use Django version 1.8. With virtualenv, you can have both versions of Django installed on the same machine, and each project can access the appropriate version of Django without any conflicts or problems.
Without virtualenv, you are forced to either upgrade the old project or find a workaround to implement the new features of the other version, therefore limiting or complicating the new project.
The virtualenv allows you to simply switch between different Python virtual environments for your individual projects. This has the added benefit that you can easily and quickly set up a new machine for testing or help a new developer get their machine up and running as fast as possible.
Getting ready
Before anything, we are going to assume that you already have a Linux/Ubuntu machine or a virtualbox instance running Linux/Ubuntu so you can follow these instructions.
Tip
I also suggest trying out Vagrant (http://www.vagrantup.com), which uses virtualbox to box and standardize your development environment.
Ubuntu 14.04 comes with Python 2.7.6 and Python 3.4 preinstalled; the other libraries are your responsibility as explained in the following sections.
Windows users need to download and install Python 2.7.x from the Python home page at https://www.python.org/downloads/windows/; please download the newest version of the 2.7.x series since this book is written with 2.7.X in mind. The installer includes a bundled version of pip, so make sure you install it!
Take a close look at the correct version to download, making sure that you get either the 32-bit or 64-bit download. You cannot mix and match the versions, so be careful and remember to install the correct version.
A great site for other kinds of Windows binaries can be found at http://www.lfd.uci.edu/~gohlke/pythonlibs/. Wheel files are the new norms of installations and can be executed from the command line as follows:
Note
On Windows, make sure that your Python interpreter is set up on your system path. This way, you can simply call Python directly from the command prompt using the C:\Users\Michael> python filename.py command. If you need more help, information can be found by following one of the online instructions at https://pypi.python.org/pypi/pip.
As of Python 2.7.9 and later versions, pip is available on installation.
Python 3 would be awesome to use, and for many Python GIS libraries, it is ready for show time. Unfortunately, not all GIS libraries jive with Python 3 (pyproj) as one would love to see at the time of writing this. If you want, feel free to go for Python 3.x and give it a go. A great webpage to check the compatibility of a library can be found at https://caniusepython3.com/.
To install virtualenv, you need to have a running installation of Python and pip. The pip package manager manages and installs Python packages, making our lives easier. Throughout this book, if we need to install a package, pip will be our tool of choice for this job. The official installation instructions for pip can be found at https://pip.pypa.io/en/latest/installing.html. To install pip from the command line, we first need to install easy_install. Let's try it out from the Terminal:
With this one line, you have both pip and easy_install installed.
Note
What is sudo?
sudo is a program for Unix-like computer operating systems that allows users to run programs with the security privileges of another user (normally, the super user or root). Its name is a concatenation of su (substitute user) and do (take action). Take a look at http://en.wikipedia.org/wiki/Sudo for more information on this.
The command sudo means to run an execution as a super user. If this fails, you will need to get theez_setup.py file, which is available at https://bootstrap.pypa.io/ez_setup.py. After downloading the file, you can run it from the command line:
Now pip should be up and running and you can execute commands to complete the installations of virtualenv and virtualenvwrapper. The virtualenvwrapper creates shortcuts that are faster ways to create or delete your virtual environments. You can test it as follows:
How to do it...
The steps to install your Python virtualenv and virtualenvwrapper packages are as follows:
Note
We use easy_install instead of pip because with Ubuntu 14.04, the virtualenvwrapper.sh file is unfortunately not located at /usr/local/bin/virtualenvwrapper.sh where it should be according to the online documentation.
To use virtualenvwrapper the next time you start up your machine, we need to set it up so that your bash terminal runs the virtualenvwrapper.sh script when your computer starts.
First, put it in your ~/.bashrc file:How it works...
Step one shows how pip installs the virtualenv package into your system-wide Python installation. Step two shows how the virtualenvwrapper helper package is installed with easy_install because the virtualenvwrapper.sh file is not created using the pip installer. This will help us create, enter, and generally, work or switch between Python virtual environments with ease. Step three assigns the WORKON_HOME variable to a directory where we want to have all of our virtual environments. Then, we'll create a new directory to hold all the virtual environments. In step four, the command source is used to execute the shell script to set up the virtualenvwrapper package. In step five, we see how to actually create a new virtualenv called pygeoan_cb in our /home/mdiener/venvs directory. This final step automatically starts our virtualenv session.
Once the virtualenv session starts, we can now see the name of virtualenv in brackets like this:
To exit virtualenv, simply type the following code:
Now, your command line should be back to normal as shown here:
To reactivate virtualenv, simply type:
Tip
The workon command has Tab completion. So, simply type workon, and then the first letter of the name of the virtual environment you want to enter, such as py. Hit Tab and it will autocomplete the name.
Inside the /venvs folder, you will find specific individual virtual environments for each project in the form of a subfolder. The virtualenvwrapper package will always create a new folder for each new project you create. You can, therefore, easily delete a folder and it will remove your virtual environment.
To quickly print a list all of the installed libraries to a file, we'll use the pip command:
This will create a text file called requirements.txt in the current folder. The text file contains a list of all the installed Python packages inside the Python virtual environment currently running.
To create a new virtualenv from a requirements file, use the following command:
There's more…
For those of you who are just starting out with geospatial Python development, it should be noted that you should keep your project-specific code at another location outside your Python virtual environment folder. For example, I always have each project-related code contained in a separate folder called 01_projects, which is my main folder. The path to my projects folder is /home/mdiener/01_projects, and the structure of two of my projects is as follows:
All virtual environments are located under /home/mdiener/venvs/. Usually, I give them the same name as a project to keep things organized, as follows:
Installing pyproj and NumPy
The pyproj is a wrapper around the PROJ.4 library that works with projections and performs transformations (https://pypi.python.org/pypi/pyproj/) in Python. All your geographic information should be projected into one of the many coordinate systems supported by the European Petroleum Survey Group (EPSG). This information is necessary for the systems to correctly place data at the appropriate location on Earth. The geographic data can then be placed on top of each other as layers upon layers of data in order to create maps or perform analysis. The data must be correctly positioned or we won't be able to add, combine, or compare it to other data sources spatially.
Data comes from many sources and, often, a projection is not the same as a dataset. Even worse, the data could be delivered with a description from a data provider stating it's in projection UTM31 when, in reality, the data is in projection UTM34! This can lead to big problems later on when trying to get your data to work together as programs will throw you some ugly error messages.
NumPy is the scientific backbone of number crunching arrays and complex numbers that are used to power several popular geospatial libraries including GDAL (geospatial abstraction library). The power of NumPy lies is in its support for large matrices, arrays, and math functions. The installation of NumPy is, therefore, necessary for the other libraries to function smoothly, but is seldom used directly in our quest for spatial analysis.
Getting ready
Fire up your virtual environment, if it is not already running, using the following standard start command:
Your prompt should now look like this:
Note
If workon for some reason does not start your virtual environment, you can start it simply by executing source /home/mdiener/venvs/pygeoan_cb/bin/activate from the command line; try the steps listed in the Installing virtualenv and virtualenvwrapper recipe again to get it going.
Now, we need to install some Python tools for development that allow us to install NumPy, so run this command:
You are now ready to move on and install pyproj and NumPy inside your running virtual environment.
How to do it...
Simply fire upvirtualenv and we will use the pip installer to do all the heavy lifting as follows:
Windows users can grab the .whl file for NumPy and execute it using following command:
Windows users can use the following command to install pyproj:
No errors, I hope. You have now successfully installed NumPy and pyproj.
Note
All sorts of errors could show up, so please take a look at the respective installation links to help you solve them:
For pyproj: https://pypi.python.org/pypi/pyproj/
For NumPy: http://www.numpy.org
How it works...
This easy installation works using the standard pip installation method. No tricks or special commands are needed. You need to simply execute the pip install <library_name> command and you are off to the races.
Tip
Library names can be found by visiting the https://pypi.python.org/pypi web page if you are unsure of the exact name you want to install.
Installing shapely, matplotlib, and descartes
A large part of geospatial analysis and visualization is made possible using Shapely, matplotlib, GDAL, OGR, and descartes, which are installed later. Most of the recipes here will use these libraries extensively so setting them up is necessary to complete our exercises.
Shapely (http://toblerity.org/shapely) provides pure spatial analysis of geometries using the Cartesian coordinate system as is used by AutoCAD, for those of you familiar with CAD-like programs. The benefit of using a flat coordinate system is that all the rules of Euclidean geometry and analytic geometry are applied. For a quick refresher in the coordinate systems that we all learned in school, here is a little image to quickly jolt your memory.
Note
Description: A Cartesian coordinate system demonstrating a flat plane to plot and measure geometry.
Illustration 1: Source: http://en.wikipedia.org/wiki/Cartesian_coordinate_system.
The classic overlay analysis and other geometric computations is where Shapely shines using the GEOS library as its workhorse in the background.
As for matplotlib (http://matplotlib.org/), it is the plotting engine that renders nice graphs and data to your screen as an image or scalable vector graphic (svg
