SciPy Recipes - L. Felipe Martins - E-Book

SciPy Recipes E-Book

L. Felipe Martins

0,0
27,59 €

oder
-100%
Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.

Mehr erfahren.
Beschreibung

With the SciPy Stack, you get the power to effectively process, manipulate, and visualize your data using the popular Python language. Utilizing SciPy correctly can sometimes be a very tricky proposition. This book provides the right techniques so you can use SciPy to perform different data science tasks with ease.
This book includes hands-on recipes for using the different components of the SciPy Stack such as NumPy, SciPy, matplotlib, and pandas, among others. You will use these libraries to solve real-world problems in linear algebra, numerical analysis, data visualization, and much more. The recipes included in the book will ensure you get a practical understanding not only of how a particular feature in SciPy Stack works, but also of its application to real-world problems. The independent nature of the recipes also ensure that you can pick up any one and learn about a particular feature of SciPy without reading through the other recipes, thus making the book a very handy and useful guide.

Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:

EPUB

Veröffentlichungsjahr: 2017

Bewertungen
0,0
0
0
0
0
0
Mehr Informationen
Mehr Informationen
Legimi prüft nicht, ob Rezensionen von Nutzern stammen, die den betreffenden Titel tatsächlich gekauft oder gelesen/gehört haben. Wir entfernen aber gefälschte Rezensionen.



SciPy Recipes
A cookbook with over 110 proven recipes for performing mathematical and scientific computations
L. Felipe Martins
Ruben Oliva Ramos
V Kishore Ayyadevara

BIRMINGHAM - MUMBAI

SciPy Recipes

Copyright © 2017 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: December 2017

Production reference: 1191217

Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.

ISBN 978-1-78829-146-0

www.packtpub.com

Credits

Authors

L. Felipe Martins Ruben Oliva Ramos V Kishore Ayyadevara

Copy Editor

Safis Editing

Reviewer

Juan Tomás Oliva Ramos

Project Coordinator

Nidhi Joshi

Commissioning Editor

Amey Varangaonkar

Proofreader

Safis Editing

Acquisition Editor

Tushar Gupta

Indexer

Pratik Shirodkar

Content Development Editor

Aishwarya Pandere

Graphics

Tania Dutta

Technical Editor

Prasad Ramesh

Production Coordinator

Arvindkumar Gupta

About the Authors

L. Felipe Martins has a PhD in applied mathematics from Brown University and is currently an associate professor in the Department of Mathematics at Cleveland State University. His main research areas are applied probability and scientific computing. He has taught applied mathematics courses at all levels, including linear algebra, differential equations, probability, and optimization, and uses Python as an instructional tool in all courses. He is the author of two books, IPython Notebook Essentials and Mastering Python Data Analysis.

Ruben Oliva Ramos is a computer systems engineer from Tecnologico de Leon Institute, with a master's degree in computer and electronic systems engineering and a specialization in teleinformatics and networking from the University of Salle Bajio in Leon, Guanajuato, Mexico. He has more than 5 years of experience of developing web applications to control and monitor devices connected with Arduino and Raspberry Pi, using web frameworks and cloud services to build the Internet of Things applications.

He is a mechatronics teacher at the University of Salle Bajio and teaches students of the master's degree in design and engineering of mechatronics systems. Ruben also works at Centro de Bachillerato Tecnologico Industrial 225 in Leon, Guanajuato, Mexico, teaching subjects such as electronics, robotics and control, automation, and microcontrollers on the Mechatronics Technician career course; he is a consultant and developer for projects in areas such as monitoring systems and datalogger data using technologies (such as Android, iOS, Windows Phone, HTML5, PHP, CSS, Ajax, JavaScript, Angular, and ASP.NET), databases (such as SQLite, MongoDB, and MySQL), web servers (such as Node.js and IIS), hardware programming (such as Arduino, Raspberry Pi, Ethernet Shield, GPS, GSM/GPRS, and ESP8266), and control and monitor systems for data acquisition and programming.

Ruben is the author of the following books by Packt: Internet of Things Programming with JavaScript, Advanced Analytics with R and Tableau, and Raspberry Pi 3 Home Automation Projects.

He is also involved in monitoring, controlling, and acquiring of data with Arduino and Visual Basic .NET for Alfaomega.

I would like to thank my savior and lord, Jesus Christ, for giving me the strength and courage to pursue this project; to my dearest wife, Mayte; our two lovely sons, Ruben and Dario; my dear father, Ruben; my dearest mom, Rosalia; my brother, Juan Tomas; and my sister, Rosalia, whom I love, for all their support while reviewing this book, for allowing me to pursue my dream and tolerating me not being with them after my busy days. I'm very grateful to Packt Publishing for giving me the opportunity to collaborate as an author and reviewer, and to belong to this honest and professional team.

V Kishore Ayyadevarahas over 9 years of experience of using analytics to solve business problems and setting up analytical work streams through his work at American Express, Amazon, and more recently a retail analytics consulting start-up. He is an MBA graduate from IIM Calcutta and also an electronics and communications engineer.

He worked in the fields of credit risk analytics, supply chain analytics, and consulting for multiple FMCG companies to identify ways to improve their profitability.

His interests lie in translating a business problem into a data-related problem by demystifying complexity in data science and identifying ways to further embed analytics in business.

I would like to dedicate my work on this book to my dear parents—Hema and Subrahmanyeswara Rao; my lovely wife—Sindhura and my dearest daughter, Hemanvi. This work would not have been possible without their support and encouragement.

About the Reviewer

Juan Tomás Oliva Ramos is an environmental engineer from the University of Guanajuato, Mexico, with a master's degree in administrative engineering and quality. He has more than 5 years of experience in managing and developing patents, technological innovation projects, and developing technological solutions through statistical control of processes.

He has been a teacher of statistics, entrepreneurship, and technological development of projects since 2011. He has always maintained an interest in improvement and innovation in processes through technology. He became an entrepreneur mentor and technology management consultant. Juan started a new department of technology management and entrepreneurship at Instituto Tecnologico Superior de Purisima del Rincon, Mexico.

He has worked on the book Wearable Designs for Smart Watches, Smart TVs, and Android Mobile Devices.

Juan has also developed prototypes using programming and automation technologies to improve operations, which have been registered for patents.

I want to thank my beautiful wife, Brenda, our two magic princesses (Maria Regina and Maria Renata) and our next member (Angel Tadeo), despite the distance they are always in my heart and in my mind. I thank Packt and Nidhi Joshi for giving me the opportunity to review this amazing book and to collaborate with a group of committed people.

www.PacktPub.com

For support files and downloads related to your book, please visit www.PacktPub.com. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.comand as a print book customer, you are entitled to a discount on the eBook copy.

Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.

https://www.packtpub.com/mapt

Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.

Why subscribe?

Fully searchable across every book published by Packt

Copy and paste, print, and bookmark content

On demand and accessible via a web browser

Customer Feedback

Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1788291468.

If you'd like to join our team of regular reviewers, you can email us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!

Table of Contents

Preface

What this book covers

What you need for this book

Who this book is for

Conventions

Reader feedback

Customer support

Downloading the example code

Errata

Piracy

Questions

Getting to Know the Tools

Introduction

Installing Anaconda on Windows

How to do it...

Installing Anaconda on macOS

How to do it...

Installing Anaconda on Linux

How to do it...

Checking the Anaconda installation

How to do it...

Installing SciPy from a binary distribution on Windows

How to do it...

Installing Python

Installing the SciPy stack

Installing SciPy from a binary distribution on macOS

How to do it...

Installing the Xcode command-line tools

Installing Homebrew

Installing Python 3

Installing the SciPy stack

Installing SciPy from source on Linux

How to do it...

Installing Python 3

Installing the SciPy stack

Installing optional packages with conda

Getting ready

How to do it...

Installing packages with pip

How to do it...

Setting up a virtual environment with conda

Getting ready

How to do it...

Creating a virtual environment for development with conda 

Getting ready

How to do it...

Creating a conda environment with a different version of a package

Getting ready

How to do it...

Using conda environments to run different versions of Python

Getting ready

How to do it...

Creating virtual environments with venv

How to do it...

Running SciPy in a script

Getting ready

How to do it...

Running SciPy in Jupyter

Getting ready

How to do it...

Running SciPy in Spyder

Getting ready

How to do it...

Running SciPy in PyCharm

Getting started

How to do it...

Getting Started with NumPy

Introduction

Creating NumPy arrays

How to do it…

Creating an array from a list

Specifying the data type for elements in an array

Creating an empty array with a given shape

Creating arrays of zeros and ones with a single value

Creating arrays with equally spaced values

Creating an array by repeating elements

Creating an array by tiling another array

Creating an array with the same shape as another array

Using object arrays to store heterogeneous data

See also

Querying and changing the shape of an array

How to do it...

Storing and retrieving NumPy arrays

How to do it...

Storing a NumPy array in text format

Storing a NumPy array in CSV format

Loading an array from a text file

Storing a single array in binary format

Storing several arrays in binary format

Loading arrays stored in NPY binary format

Indexing

How to do it...

Accessing sub arrays using slices

Selecting subarrays using an index list

Indexing with Boolean arrays

Operations on arrays

How to do it...

Computing a function for all elements of an array

Doing array operations

Computing matrix products

Using masked arrays to represent invalid data

How to do it...

Creating a masked array from an explicit mask

Creating a masked array from a condition

Using object arrays to store heterogeneous data

How to do it...

Defining, symbolically, a function operating on arrays

Getting ready

How to do it...

How it works...

Using Matplotlib to Create Graphs

Introduction

Creating two-dimensional plots of functions and data

Getting ready

How to do it…

How it works…

Generating multiple plots in a single figure

Getting ready

How to do it…

How it works…

Setting line styles and markers

Getting ready

How to do it…

How it works…

Using different backends to display graphs

Getting ready

How to do it…

How it works…

Saving plots to disk

Getting ready

How to do it…

How it works…

Annotating graphs

Getting ready

How to do it…

How it works…

Generating histograms and box plots

Getting ready

How to do it…

How it works…

Creating three-dimensional plots

Getting ready

How to do it…

How it works…

Generating interactive displays in the Jupyter Notebook

Getting ready

How to do it…

How it works…

Object-oriented graph creation using Artist objects

Getting ready

How to do it…

How it works…

Creating a map with cartopy

Getting ready

How to do it…

How it works…

Data Wrangling with pandas

Creating Series objects

Getting ready

How to do it...

How it works...

Creating DataFrame objects

Getting ready

How to do it...

How it works...

Inserting and deleting columns to a DataFrame

Getting ready

How to do it...

How it works...

Inserting and deleting rows to a DataFrame

Getting ready

How to do it...

How it works...

Selecting items by row indexes and column labels

Getting ready

How to do it...

How it works...

Selecting items by integer location

Getting ready

How to do it...

How it works...

Selecting items using mixed indexing

Getting ready

How to do it...

How it works...

Accessing, selecting, and modifying data

Getting ready

How to do it...

How it works...

Selecting rows using Boolean selection

How to do it...

Reading and storing data in different formats

Getting ready

How to do it...

Working with CSV, text/tabular, and format data

How it works...

Reading a CSV file into a DataFrame

Specifying the index column when reading a CSV file

Reading and writing data in Excel format

Reading and writing JSON files

Reading HTML data from the web

Accessing CSV data on the web

Reading and writing from/to SQL databases

Data displays employing different kinds of visual representation

Getting ready

How to do it...

How it works...

How to apply numerical functions and operations to Series and DataFrame objects

Getting ready

How to do it...

How it works...

Computing statistical functions on Series and DataFrame objects

Getting ready

How to do it...

Retrieving summary descriptive statistics

How it works...

Calculating the mean

Calculating variance and standard deviation

How to sort data in Series and DataFrame objects

Getting ready

How to do it...

How it works...

Performing merging, joins, concatenation, and grouping

Getting ready

How to do it...

How it works...

Merging data from multiple pandas objects

Matrices and Linear Algebra

Introduction

Matrix operations and functions on two-dimensional arrays

How to do it…

Solving linear systems using matrices

How it works…

How to do it…

Calculating the null space of a matrix 

How to do it…

Calculating the LU decompositions of a matrix 

How to do it…

Calculating the QR decomposition of a matrix

How to do it…

Calculating the eigenvalue and eigenvector of a matrix

How to do it…

Diagonalizing a matrix

How to do it…

Calculating the Jordan form of a matrix

How to do it…

Calculating the singular value decomposition of a matrix

How to do it…

Creating a sparse matrix

How to do it…

Computations on top of a sparse matrix

How to do it…

Solving Equations and Optimization

Introduction

Non-linear equations and systems

Getting ready

How to do it...

How it works...

System of equations and how to solve it

Getting ready

How to do it...

How it works...

Choosing the solver used to find the solution of equations

Getting ready

How to do it...

How it works...

Solving constrained non-linear optimization problems in several variables

Getting ready

How to do it...

How it works...

Solving one-dimensional optimization problems

Getting ready

How to do it...

How it works...

Solving multidimensional non-linear equations using the Newton-Krylov method

Getting ready

How to do it...

Solving multidimensional non-linear equations using the Anderson method

Getting ready

How to do it...

How it works...

Finding the best linear fit for a set of data

Getting ready

How to do it...

How it works ...

Doing non-linear regression for a set of data

Getting ready

How to do it...

How it works...

Regression

Getting ready

How to do it...

How it works...

Constants and Special Functions

Introduction

Physical and mathematical constants available in SciPy

Getting ready...

How to do it...

Using constants in the CODATA database

Getting ready

How to do it...

Bessel functions

Getting ready...

How to do it...

Error functions

Getting ready...

How to do it...

Orthogonal polynomials functions

Getting ready...

How to do it...

Gamma function

Getting ready...

How to do it...

How it works...

The Riemann zeta function

Getting ready

How to do it...

How it works...

Airy and Bairy functions

Getting ready...

How to do it...

The Bessel and Struve functions

Getting ready...

How to do it...

How it works...

There's more

Calculus, Interpolation, and Differential Equations

Introduction

Integration

Getting ready

How to do it…

How it works...

Computing integrals using the Newton-Cotes method

Computing integrals using a Gaussian quadrature

Getting ready

How to do it...

How it works...

Computing integrals with weighting functions

Getting ready

How to do it...

Computing multiple integrals

Getting ready

How to do it...

Interpolation

Getting ready

How to do it...

Computing a polynomial interpolation for a set of data points

Getting ready

How to do it…

How it works…

Univariate interpolation

Getting ready

How to do it…

Finding a cubic spline that interpolates a set of data

Getting ready

How to do it…

How it works...

Defining a B-spline for a given set of control points

Getting ready

How to do it…

How it works ...

Differentiation

Getting ready

How to do it...

How it works...

Solving a one-dimensional ordinary differential equation

Getting ready

How to do it...

How it works...

Solving a system of ordinary differential equations

Getting ready

How to do it...

How it works...

Solving differential equations and systems with parameters

Getting ready

How to do it...

How it works...

Using ode and the objected-oriented interface to solve differential equations

Getting ready

How to do it...

How it works ...

Statistics and Probability

Introduction

Computing the probability mass function of a discrete random variable

Binomial discrete distribution

Multinomial discrete distribution

How to do it...

Visualizing the probability mass function

Computing the probability density function of a continuous random variable

How to do it...

Computing the cumulative distribution function for a random variable

How to do it...

Computing the values of inverse probabilities associated with a random variable

How to do it...

Computing the average, standard deviation, and higher moments of a distribution

How to do it...

Average and standard deviation of a distribution

Calculating the moments of a distribution

Computing probabilities associated with the multivariate Gaussian distribution

How to do it...

Getting started with simulation

Computing the summary statistics of a dataset

How to do it...

Advanced Computations with SciPy

Discrete Fourier transforms

How to do it…

How it works…

Computing the discrete Fourier transform (DFT) of a data series using the FFT algorithm

How to do it…

How it works…

Computing the inverse DFT of a data series

How to do it…

How it works…

Computing signal construction

How to do it…

Getting started with filters

How to do it…

How it works…

Computing the DFT for two-dimensional data

How to do it…

How it works…

How to find the DFT of the derivative of a function

How to do it…

Computing the convolution of two functions

How to do it…

Mathematical imaging

How to do it…

Computing pairwise distances from a dataset, using different distance metrics

How to do it…

How to identify neighborhoods and nearest neighbors for a dataset and a given metric

How to do it…

How it works…

Nearest neighbors regression

How it works…

Preface

The SciPy stack is a collection of open source libraries of the powerful Python scripting language, together with its interactive shells. This environment offers a cutting-edge platform for numerical computation, programming, visualization, and publishing, and is used by some of the world's leading mathematicians, scientists, and engineers. It works on any operating system that supports Python and is very easy to install and completely free of charge! It can effectively transform into a data-processing and system-prototyping environment.

The SciPy stack is a popular Python ecosystem used for mathematical and scientific computing tasks. It can be used to perform a variety of data science tasks, from data manipulation to visualization. Utilizing the offerings of SciPy to perform your data science tasks is a very tricky proposition.

This book will show you how you can put to use the various functionalities offered by the SciPy stack in the most efficient way possible. With the help of this book, you will solve real-world problems in linear algebra, numerical analysis, visualization, and much more, including independent recipes drawn from the fields of statistics, scientific computation, and visualization. You will master the different tasks associated with using SciPy and its related libraries, such as NumPy, Matplotlib, pandas and more, in the best way. This book will ensure that you have a practical understanding of not onlyhow a particular feature in SciPy stack works but also its applications in real-world problems.

What this book covers

Chapter 1, Getting to Know the Tools, explains how to install and configure all the packages required to set up and configure an environment dedicated to scientific computing in Python. The chapter considers several different setup options in the three main operating systems available to users: Windows, macOS, and Linux.

Chapter 2, Getting Started with NumPy, presents the essential recipes for efficient use of NumPy, the Python package for numerical computations on which SciPy is based.

Chapter 3, Using Matplotlib to Create Graphs, is a thorough discussion of Matplotlib, the plotting library included with NumPy and SciPy, concentrating on the skills required to display the results of technical computations.

Chapter 4, Data Wrangling with pandas, shows how to use pandas, a powerful package for data handling and analysis in Python.

Chapter 5, Matrices and Linear Algebra, covers performing the various matrix data manipulation techniques such as basic matrix operations, solving linear systems, finding eigenvalues and eigenvectors, calculating the singular value decomposition, and sparse matrix manipulation techniques that are potentially used in recommender systems using SciPy.

Chapter 6, Solving Equations and Optimization, discusses the solutions of numerical equations and systems of equations, as well as the solution of maximization/minimization problems.

Chapter 7, Constants and Special Functions, presents the numerical constants and special functions that are available in SciPy.

Chapter 8, Calculus, Interpolation, and Differential Equations, shows how to solve essential calculus problems, including integration, differentiation, interpolation, and differential equations.

Chapter 9, Statistics and Probability, covers the various statistics and probability measures such as PMF, PDF, CDF, and multivariate Gaussian distributions using SciPy.

Chapter 10, Advanced Computations with SciPy, discusses the advanced computations available in SciPy that are of a more specific nature.

What you need for this book

To get the most from this book, the reader needs to know the basics of Python; it's not necessary that the reader has the ability to program because the first chapter explains how to install the plugins needed to work with SciPy. The following are the software and OS requirements:

SciPy 1.0.0

NumPy v1.13

Matplotlib 2.1.0

Python 2.7, 3.5, and 3.6

Python Data Analysis Library (v0.21.0)

SymPy 1.1.1

Released libraries

OS required: Windows, Mac, or Linux

Who this book is for

Python developers, aspiring data scientists, and analysts who want to get started with scientific computing using Python will find this book to be a useful resource. If you want to learn how to manipulate and visualize your data using the SciPy stack, this book will also help you. A basic understanding of Python programming is all you need to get started.

This book is for readers who want learn more about SciPy in specific topics and gain the basic knowledge required to solve problems. The following are the objectives:

Tackle sophisticated problems in scientific computing with the SciPy stack

Get a solid foundation in scientific computing with Python and open source software

Present common tasks related to SciPy and associated libraries such as NumPy, pandas, and Matplotlib

Perform mathematical operations and work with the statistical and probability functions in SciPy

Empower users to further explore the library and find solutions to their own computational needs

Discuss best practices and efficient methods in the solution of computational problems

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of. To send us general feedback, simply email [email protected], and mention the book's title in the subject of your message. If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Downloading the example code

You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files emailed directly to you. You can download the code files by following these steps:

Log in or register to our website using your email address and password.

Hover the mouse pointer on the

SUPPORT

tab at the top.

Click on

Code Downloads & Errata

.

Enter the name of the book in the

Search

box.

Select the book for which you're looking to download the code files.

Choose from the drop-down menu where you purchased this book from.

Click on

Code Download

.

Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:

WinRAR / 7-Zip for Windows

Zipeg / iZip / UnRarX for Mac

7-Zip / PeaZip for Linux

The code bundle for the book is also hosted on GitHub at https://github.com/PacktPublishing/SciPy-Recipes. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title. To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.

Piracy

Piracy of copyrighted material on the internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the internet, please provide us with the location address or website name immediately so that we can pursue a remedy. Please contact us at [email protected] with a link to the suspected pirated material. We appreciate your help in protecting our authors and our ability to bring you valuable content.

Questions

If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.

Getting to Know the Tools

In this chapter, we cover the following recipes:

Installing Anaconda on Windows

Installing Anaconda on macOS

Installing Anaconda on Linux

Checking the Anaconda installation

Installing SciPy from a binary distribution on Windows

Installing SciPy from a binary distribution on macOS

Installing SciPy from source on Linux

Installing optional packages with

conda

Installing packages with

pip

Setting up a virtual environment with

conda

Creating a virtual environment for development with

conda

Creating a

conda

environment with a different version of a package

Using

conda

environments to run different versions of Python

Creating Virtual environments with venv

Running SciPy in a script

Running SciPy in Jupyter

Running SciPy in Spyder

Running SciPy in PyCharm

Introduction

In this chapter, we discuss the available options for setting up and running the SciPy stack and associated tools. We present solutions for all the major platforms and consider different scenarios. Readers are advised to browse through all installation options before deciding which option fits their workflow and computational needs. After reading this chapter, the reader will understand all different options for setting up a full-fledged environment in Python for computational and data science.

The recipes in this chapter assume the use of the following tools:

The Command Prompt, also known as Terminal in Linux and macOS. Each operating system has a different way of accessing the default Command Prompt:

In Windows, open the search bar and type

cmd

.

In macOS, the Terminal app is in the

Applications-Utilities

folder.

In Linux, the Command Prompt may be called

xterm

or

Terminal

. In Ubuntu, it can also be started by pressing

Ctrl

+

Alt

+

T

.

A text editor. Sublime Text is a popular multi-platform programmer's editor with many nice features, available at:

https://www.sublimetext.com

. Sublime Text is commercial software, but a trial version is available. Alternatives available for each operating system are as follows:

Windows

: Notepad is pre-installed on Windows. A free Notepad alternative that adds nice features for free is Notepad++, which can be downloaded from

https://notepad-plus-plus.org.

macOS

: TextEdit is pre-installed and can be found in the

Applications

folder. An alternative is

nano

, a simple text editor that can be started from a Terminal window.

Linux

: Usually ships with at least one of the following:

gedit

,

nano

, or

vim

, all of which can all be launched from a Terminal window.

If you decide to use the pre-assembled Anaconda distribution, you will also need to download it from the following site: https://www.continuum.io/downloads.

Choose the latest 64-bit Python 3 distribution, unless you have an older computer with a 32-bit architecture.

Installing Anaconda on Windows

In this recipe, we will show you how to install Anaconda on a Windows system.

How to do it...

Double-click the downloaded

.exe

installer

Accept the software license

When prompted for the kind of installation you want, select

Just Me

, and then click

Next

Accept the default installation folder

In the next option box, select both

Add Anaconda to my PATH environment variable

and

Register Anaconda as my default Python 3.x

Click

Install

to finish the installation

Anaconda will be installed in the

C:\Users\

username

\Anaconda3

folder

Optionally, proceed to the

Checking the Anaconda installation

Installing Anaconda on macOS

In this recipe, we will show you how to install Anaconda on a macOS system.

How to do it...

Double-click the downloaded installer file, which is a file with a

.pkg

extension

Click the

Continue

button to view

ReadMe

and accept the software license

When prompted, select the

Install for me only

option and click

Continue

Review the installation options and click the

Install

button

Wait until the installation finishes, and then click the

Close

button to quit the installer

Anaconda will be installed in the

anaconda

subfolder of your home folder

Optionally, proceed to the

Checking the Anaconda installation

recipe

Installing Anaconda on Linux

In this recipe, we will show you how to install Anaconda on a Linux system.

How to do it...

Open a Terminal window on the folder containing the installer and run the following command, replacing the version number,

x.x.x

,with the corresponding value for file you downloaded:

bash Anaconda3-x.x.x-Linux-x86_64.sh

Review the license agreement and accept it. Enter

yes

when prompted to continue the installation

When asked if you want to prepend the Anaconda3 installation location to your path, answer

yes

Wait until the installer stops

Anaconda will be installed in the

anaconda3

subfolder of your home folder.

Optionally, proceed to the

Checking the Installation

recipe

Checking the Anaconda installation

This recipe shows you how to do some basic checking. We will verify that the software can be started and that the correct version is being used.

How to do it...

Open a new Terminal window and run the following command:

python3

Verify the information displayed in the Terminal. It will look like the following:

Python 3.6.0 |Anaconda 4.3.1 (x86_64)| (default, Dec 23 2016, 13:19:00)

[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>>

Notice that, in the output, we can check that we are indeed running the Python 3 version distributed with Anaconda.

You have now successfully installed Anaconda on your computer.

Installing SciPy from a binary distribution on Windows

Windows does not ship with any version of Python pre-installed, which actually makes things easier when we want to install our own version of Python.

On the other hand, the installation of a full SciPy stack in Windows is somewhat more complex, due to conflicts that exist between the Python distribution and certain Windows libraries. We indicate an installation route that has been tested several times, but some trial and error may be necessary due to changes in the distribution.

How to do it...

To make the instructions easier to follow, the installation procedure is broken down into two stages:

Installing Python

Install the SciPy stack

Installing Python

Go to

https://www.python.org

and download the Python 3 binary distribution for Windows

Once the download finishes, double-click on the installation file to start the setup

Check the box

Add Python 3.x to PATH

Click the

Install Now

option

Select the

Disable the path length limit

option, if available, on the last installation screen

Close the installation screen

These steps will install Python in the folder:

C:\Users\username\Appdata\Local\Programs\Python\Python3x

To test the installation, start a Command Prompt window and enter the following command:

python3

If all is correct, the Python command-line interpreter will start and display information about the version of Python being run. For now, just exit the interpreter by entering, at the >>>Python prompt, the following statement:

quit()

Now, let's check if pip was correctly installed. Enter the following at the command line:

pip --version

This should print information about the currently installed version of pip3, including the location where packages will be installed. As long as no errors are reported, the installation is correct.

Installing the SciPy stack

To install SciPy, we need to first download the versions of the library that have been built specifically for Windows. They can be found at the following site: http://www.lfd.uci.edu/~gohlke/pythonlibs/.

This page contains a long list of pre-compiled Python packages for Windows. Search the page for numpy-mkl and scipy and look for a package that matches your operating system and Python distribution. In my case, I found the following two files:

numpy-1.12/1+mkl-cp36-cp36m-win_amd64.whl

scipy-0.19.0-cp36-cpm36m-win_amd64.whl

Notice that the package names refer to version 3.6 and a 64-bit architecture. Make sure the versions you download match your Python 3 distributions. Open a command window on the directory where the files were saved and enter the following two commands, in the following order:

pip install numpy-1.12/1+mkl-cp36-cp36m-win_amd64.whl

pip install scipy-0.19.0-cp36-cpm36m-win_amd64.whl

After installing NumPy and SciPy, pip can be used to install the other packages directly by running the commands shown as follows:

pip install matplotlib

pip install ipython jupyter

pip install pandas sympy nose

Let's now test the installation. First, start Python 3 and execute the following statements at the >>> Python prompt:

import numpy

import scipy

import matplotlib

import pandas

import IPython

import sympy

If you can run all these commands and there are no errors, the installation of the packages is correct. Exit the Python shell by running the following statement:

quit()

Now, back in the command window, run the following command:

ipython

This will start IPython and display information about the installed version. For now, simply exit IPython by running the following at the prompt:

quit()

Finally, let's test the Jupyter Notebook. At the command line, run the following command:

jupyter notebook

If all is correct, this will start the Jupyter Notebook in your browser after a few seconds. This finishes the installation of Python and the SciPy stack in Windows.

Installing SciPy from a binary distribution on macOS

macOS currently ships with version 2.7 of Python pre-installed. In this recipe, we will show you how to install Python 3 on the Mac without making changes to the original Python distribution. The easiest way to achieve this is to use Homebrew, a package manager for macOS.

How to do it...

The full installation instructions are broken down into the following stages:

Installing the Xcode command-line tools

Installing Homewbrew

Installing Python 3

Installing the SciPy stack

Installing the Xcode command-line tools

Xcode is the free development environment for macOS distributed by Apple. If you already have Xcode installed on your computer, you can skip this step. If you don't, open a Terminal window and run the following command:

xcode-select --install

If you get an error message, then the command-line tools are already installed and you can go to the next step. Otherwise, a window will pop up asking for confirmation. Press the Install button and, when prompted, accept the license agreement.

To check that the command-line tools were correctly installed, run the following command in the Terminal:

gcc -v

This command prints information about the gcc compiler present in your computer, which will be similar to the output shown as follows:

Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1

Apple LLVM version 8.1.0 (clang-802.0.38)

Target: x86_64-apple-darwin16.4.0

Thread model: posix

InstalledDir: /Library/Developer/CommandLineTools/usr/bin

If you get no error message, the command-line tools are properly installed.

Installing Homebrew

Homebrew is a package manager for macOS that makes it easier to install and remove software packages without interfering with system software that ships with the computer. It installs package files to the /usr/local directory and makes no changes to system folders. Although it is possible to install Python on the Mac from the source, using Homebrew considerably simplifies the setup process.

To install Homebrew, open a Terminal window and run the following command. Please note that the whole command should be typed in a single Terminal line:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Follow the on-screen instructions and confirm that you want to install Homebrew. Enter the administrative password for your computer if prompted. On a personal computer, this is usually the same as your login password.

To check that Homebrew was successfully installed, run the following command:

brew -v

This command outputs information about the current Homebrew installation, which looks like the following example:

Homebrew 1.1.13

Homebrew/homebrew-core (git revision c80e; last commit 2017-04-26)

If you get a similar message and no errors, Homebrew is properly installed.

Installing Python 3

Once Homebrew is set up, install Python 3 by running the following command from a Terminal window:

brew install python3

The installation process will start and may take a few minutes. When it is finished, run the following from the command line:

python3

If the installation is correct, this will print information about the Python interpreter, shown as follows:

Python 3.x.x (default, Apr 4 2017, 09:40:21)

[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>>

You can check that you are indeed running the Python distribution that you installed by checking the version number, indicated by 3.x.x in the preceding sample. You can now exit Python by running the following command at the >>>Python prompt:

quit()

We are going to use the pip3Python package manager to install the SciPy stack. To check that pip3 was correctly installed, run the following statement from the command line:

pip3 --version

This will print to the Terminal information about the currently installed version of pip, as shown in the following example:

pip 9.0.1 from /home/fmartins/python3/lib/python3.x/site-packages (python 3.x)

Verify that you are indeed running the version of pip3 associated with your installation of Python by checking the version number, indicated by 3.x in the preceding sample output. If no error message is issued, the setup was completed correctly, and you can proceed to install SciPy.

Installing the SciPy stack

To install the SciPy stack, execute each of the following commands on a Terminal window:

pip3

install

--

user

numpy

scipy

matplotlib

pip3 install

--

user

ipython

jupyter

pip3 install --user pandas

sympy

nose

We now need to adjust the PATH variable in the .bash_profile file. Notice that you might not have a .bash_profile yet. If you do, it is important to make a backup copy of it by running the following commands at the command line:

cd

cp .bash_profile .bash_profile.bak

If you get a message stating that .bash_profile does not exits, do not worry. We will create one now.

Start your text editor and open the.bash_profile file. For example, to use nano, a simple text editor included with macOS, run the following in a Terminal window:

cd

nano .bash_profile

This will create .bash_profile if it still does not exist. Add the following line at the end of file, where you need to replace 3.x by the version of Python you have installed:

export PATH="$HOME/Library/Python/3.x/bin:$PATH"

Save the file, close the editor, and run the following command from the Terminal window:

source .bash_profile

This completes the installation of a basic SciPy stack. To test the setup, start Python 3 in the Terminal window by running the following command:

python3

To check that all packages we need were installed, execute the following lines at the >>> Python prompt. Notice that there will be no output, and, as long as there are no errors, the installation is correct:

import numpy

import scipy

import matplotlib

import IPython

import pandas

import sympy

import nose

You can now exit the Python shell by running the following statement at the prompt:

quit()

Let's now check that IPython is accessible from the command line. Run the following line from the Terminal window:

ipython

This will start IPython, an alternative Python shell with many added features that is required to run Jupyter. A message similar to the following will be printed:

Python 3.x.x (default, Apr 4 2017, 09:40:21)

Type 'copyright', 'credits' or 'license' for more information

IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]:

If you get an error when trying to start IPython, the PATH variable was probably not set correctly in the .bash_profile file. You can check the current value of the PATH variable by entering the echo $PATH command. If the path is not correct, edit .bash_profile as indicated previously to correct the error.

Exit Python by entering, the following command in the IPython prompt:

quit()

Let's now test if we can run the Jupyter Notebook. Run the following command from the Terminal window:

jupyter notebook

If all is correct, you will see a series of startup messages in the Terminal, and the Jupyter Notebook will eventually start on your browser. If you get an error message, you probably have an incorrectly configured PATH variable. Check the preceding tip box for instructions on how to fix it.

This finishes the installation of Python and the SciPy stack in macOS. Please proceed to the Setting up a virtual environment section.

Installing SciPy from source on Linux

Since Linux is an umbrella name for a number of distinct operating system configurations, there is no binary distribution that fits all possible Linux flavors.

All modern distributions of Linux come with Python pre-installed. In this recipe, we describe a setup procedure for a local Python 3 installation from source that works on two of the most popular Linux distributions, Ubuntu and Debian. The installation will be located in the user's home directory, and will not conflict with any pre-installed version of Python that exists in the system.

How to do it...

The installation procedure is broken down in the following stages:

Installing Python 3

Installing the SciPy Stack

Installing Python 3

Start by opening a Terminal window and running the following commands, one at a time. You will be required to enter the administrative password for your system which, on a personal computer, is usually your own login password:

sudo apt-get install build-essential

sudo apt-get install sqlite3 libsqlite3-dev

sudo apt-get install bzip2 libbz2-dev

sudo apt-get install libreadline-dev libncurses5-dev

sudo apt-get install libssl-dev tcl-dev tk-dev python3-tk

sudo apt-get install zlib1g-dev liblzma-dev

Next, download a source distribution of Python from the site https://python.org.

Make a note of the file name, which will look like the following: Python-3.x.x.tar.xz, where x.x is a pair of numbers that specify the build for this distribution. You should download the highest version available, which should be above 3.6.0.

Now, go to the directory where the downloaded file was saved and run the following command, replacing x.x with the corresponding build number of your file:

tar xvf Python-3.x.x.tar.xz

To build the distribution, execute the following commands, again replacing x.xwiththe correct build number:

cd Python-3.x.x

./configure --prefix=$HOME/python3

make

The build process will take a while to finish, depending on the configuration and speed of your system.

The following step is optional. If you want to run the battery of tests included with the source distribution, run the following command from the Terminal window:

make test

Depending on how fast your computer is, this may take a long time. At the end of the tests, a report of the build process will be generated. Do not worry if you get a few messages about skipped tests.

Next, install the code in its target directory, by running the following command:

make install

We now need to adjust the PATH environment variable. Start by making a backup copy of your shell configuration file by running the following commands:

cd

cp .bashrc .bashrc.python.bak

Start your text editor, open the .bashrc file, and add the following line at the end of the file:

export PATH=$HOME/python3/bin:$PATH

Save the file, close the editor and, back at the Terminal window, run the following command:

source ~/.bashrc

To test the installation, start Python from the command line by running the following command:

python3

If the installation is correct, Python will start and print information about the interpreter, as follows:

Python 3.x.x (default, Apr 4 2017, 09:40:21)

[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.38)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

Check that the correct version of Python is being started by verifying that the version number 3.x.x coincides with that of the distribution you downloaded.

If you get an error when trying to start Python, or the wrong version of Python starts, the PATH variable was probably not set correctly in the .bash_rc file. You can check the current value of PATH by entering the echo $PATH command. If the path is not correct, edit.bashrc as indicated previously.

Exit Python by running the following command at the >>> Python prompt:

quit()

As a final test, run the following statement from the command line to check that Python's package manager was installed:

pip3 --version

This will print information about the version of pip3 that you are running. If no error message is issued, the setup was completed correctly and you can proceed to installing SciPy.

Installing the SciPy stack

Open a Terminal window and enter each of the following commands in succession:

pip3 install --user numpy scipy matplotlib

pip3 install --user ipython jupyter PyQt5

pip3 install --user pandas sympy nose

Make a backup copy of the .bashrc file and open it with a text editor. For example, to use nano, run the following commands in a Terminal window:

cd

cp .bashrc .bashrc.scipy.bak

nano .bashrc

Add the following line at the end of .bashrc:

export PATH="$HOME/.local/bin:$PATH"

Save the file, close the editor, and run the following command from the Terminal window:

source .bashrc

Let's now test the installation. Start Python by running the following command in a Terminal window:

python3

Execute the following lines at the >>>Python prompt. There will be no output and, as long as there are no errors, the installation is correct:

import numpy

import scipy

import matplotlib

import IPython

import pandas

import sympy

import nose

Exit Python by running the following at the prompt:

quit()

Back at the Terminal prompt, run the following command:

ipython

This will start IPython, an alternative Python shell with many added features, and print a startup message. You can check that the Python shell is running the expected version of Python from the top line of the startup message. Exit IPython by entering, at the prompt, the following command:

quit()

Back at the Command Prompt, run the following from the command line:

jupyter notebook

If all is correct, the Jupyter Notebook will start on your browser after a while. You have now concluded the installation of Python 3 and the SciPy stack in Linux.