Learn Python Programming, 3rd edition E-Book

Fabrizio Romano

• Herausgeber: Packt Publishing
• Kategorie: Fachliteratur
• Sprache: Englisch

Seitenzahl: 811

Veröffentlichungsjahr: 2021

Beschreibung

Learn Python Programming, Third Edition is both a theoretical and practical introduction to Python, an extremely flexible and powerful programming language that can be applied to many disciplines. This book will make learning Python easy and give you a thorough understanding of the language. You'll learn how to write programs, build modern APIs, and work with data by using renowned Python data science libraries.

This revised edition covers the latest updates on API management, packaging applications, and testing. There is also broader coverage of context managers and an updated data science chapter.

The book empowers you to take ownership of writing your software and become independent in fetching the resources you need. You will have a clear idea of where to go and how to build on what you have learned from the book.

Through examples, the book explores a wide range of applications and concludes by building real-world Python projects based on the concepts you have learned.

Leseprobe

Learn Python Programming

Third Edition

An in-depth introduction to the fundamentals of Python

Fabrizio Romano

Heinrich Kruger

BIRMINGHAM—MUMBAI

"Python" and the Python Logo are trademarks of the Python Software Foundation.

Learn Python Programming

Third Edition

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Producer: Tushar Gupta

Acquisition Editor – Peer Reviews: Suresh Jain, Saby Dsilva

Project Editor: Namrata Katare

Development Editor: Lucy Wan

Copy Editor: Safis Editing

Technical Editor: Aditya Sawant

Proofreader: Safis Editing

Indexer: Tejal Daruwale Soni

Presentation Designer: Pranit Padwal

First published: December 2015

Second edition: June 2018

Third edition: October 2021

Production reference: 3160223

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80181-509-3

www.packt.com

"To Elisa, the love of my lives, and to all those who gift the world their beautiful smiles."

Fabrizio Romano

"To my wife, Debi, without whose love, support, and endless patience I would not have been able to do this."

Heinrich Kruger

Contributors

About the authors

Fabrizio Romano was born in Italy in 1975. He holds a master's degree in Computer Science Engineering from the University of Padova. He's been working as a professional software developer since 1999. Fabrizio has been part of Sohonet's Product Team since 2016. In 2020, the Television Academy honored them with an Emmy Award in Engineering Development for advancing remote collaboration.

I would like to thank everyone at Packt, and the reviewers, who helped us in making this book a success. I also want to thank Heinrich Kruger and Tom Viner for joining me in this adventure. My deepest gratitude goes to my wife-to-be, Elisa, who never made me feel like I was neglecting her, even though I was. Thank you for your love and support.

Heinrich Kruger was born in South Africa in 1981. He holds a master's degree in Computer Science from Utrecht University in the Netherlands. He has been working as a professional software developer since 2014 and has worked alongside Fabrizio in the Product Team at Sohonet since 2017.

I want to thank Fabrizio for asking me to help him with this book. It's been a great experience working with you, my friend. I would also like to thank Tom Viner and Dong-hee Na for their helpful comments, as well as everyone at Packt who helped us along the way. Most of all, I thank my wife, Debi, for all her love, encouragement, and support.

About the reviewers

Tom Viner is a principal software developer living in London. He has over 13 years' experience in building web applications and has been using Python and Django for 10 years. He has special interests in open-source software, web security, and test-driven development.

I would like to thank Fabrizio Romano and Heinrich Kruger for inviting me to review this book.

Dong-hee Na is a software engineer and an open-source enthusiast. He works at Line Corporation as a backend engineer. He has professional experience in machine learning projects based on Python and C++. As for his open-source work, he focuses on the compiler and interpreter area, especially for Python-related projects. He has been a CPython core developer since 2020.

Contents

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

A Gentle Introduction to Python

A proper introduction

Enter the Python

About Python

Portability

Coherence

Developer productivity

An extensive library

Software quality

Software integration

Satisfaction and enjoyment

What are the drawbacks?

Who is using Python today?

Setting up the environment

Python 2 versus Python 3

Installing Python

Setting up the Python interpreter

About virtual environments

Your first virtual environment

Installing third-party libraries

Your friend, the console

How to run a Python program

Running Python scripts

Running the Python interactive shell

Running Python as a service

Running Python as a GUI application

How is Python code organized?

How do we use modules and packages?

Python's execution model

Names and namespaces

Scopes

Objects and classes

Guidelines for writing good code

Python culture

A note on IDEs

Summary

Built-In Data Types

Everything is an object

Mutable or immutable? That is the question

Numbers

Integers

Booleans

Real numbers

Complex numbers

Fractions and decimals

Immutable sequences

Strings and bytes

Encoding and decoding strings

Indexing and slicing strings

String formatting

Tuples

Mutable sequences

Lists

Bytearrays

Set types

Mapping types: dictionaries

Data types

Dates and times

The standard library

Third-party libraries

The collections module

namedtuple

defaultdict

ChainMap

Enums

Final considerations

Small value caching

How to choose data structures

About indexing and slicing

About names

Summary

Conditionals and Iteration

Conditional programming

A specialized else: elif

The ternary operator

Looping

The for loop

Iterating over a range

Iterating over a sequence

Iterators and iterables

Iterating over multiple sequences

The while loop

The break and continue statements

A special else clause

Assignment expressions

Statements and expressions

Using the walrus operator

A word of warning

Putting all this together

A prime generator

Applying discounts

A quick peek at the itertools module

Infinite iterators

Iterators terminating on the shortest input sequence

Combinatoric generators

Summary

Functions, the Building Blocks of Code

Why use functions?

Reducing code duplication

Splitting a complex task

Hiding implementation details

Improving readability

Improving traceability

Scopes and name resolution

The global and nonlocal statements

Input parameters

Argument-passing

Assignment to parameter names

Changing a mutable object

Passing arguments

Positional arguments

Keyword arguments

Iterable unpacking

Dictionary unpacking

Combining argument types

Defining parameters

Optional parameters

Variable positional parameters

Variable keyword parameters

Positional-only parameters

Keyword-only parameters

Combining input parameters

More signature examples

Avoid the trap! Mutable defaults

Return values

Returning multiple values

A few useful tips

Recursive functions

Anonymous functions

Function attributes

Built-in functions

Documenting your code

Importing objects

Relative imports

One final example

Summary

Comprehensions and Generators

The map, zip, and filter functions

map

zip

filter

Comprehensions

Nested comprehensions

Filtering a comprehension

Dictionary comprehensions

Set comprehensions

Generators

Generator functions

Going beyond next

The yield from expression

Generator expressions

Some performance considerations

Don't overdo comprehensions and generators

Name localization

Generation behavior in built-ins

One last example

Summary

OOP, Decorators, and Iterators

Decorators

A decorator factory

Object-oriented programming (OOP)

The simplest Python class

Class and object namespaces

Attribute shadowing

The self argument

Initializing an instance

OOP is about code reuse

Inheritance and composition

Accessing a base class

Multiple inheritance

Method resolution order

Class and static methods

Static methods

Class methods

Private methods and name mangling

The property decorator

The cached_property decorator

Operator overloading

Polymorphism – a brief overview

Data classes

Writing a custom iterator

Summary

Exceptions and Context Managers

Exceptions

Raising exceptions

Defining your own exceptions

Tracebacks

Handling exceptions

Not only for errors

Context managers

Class-based context managers

Generator-based context managers

Summary

Files and Data Persistence

Working with files and directories

Opening files

Using a context manager to open a file

Reading and writing to a file

Reading and writing in binary mode

Protecting against overwriting an existing file

Checking for file and directory existence

Manipulating files and directories

Manipulating pathnames

Temporary files and directories

Directory content

File and directory compression

Data interchange formats

Working with JSON

Custom encoding/decoding with JSON

I/O, streams, and requests

Using an in-memory stream

Making HTTP requests

Persisting data on disk

Serializing data with pickle

Saving data with shelve

Saving data to a database

Summary

Cryptography and Tokens

The need for cryptography

Useful guidelines

Hashlib

HMAC

Secrets

Random numbers

Token generation

Digest comparison

JSON Web Tokens

Registered claims

Time-related claims

Authentication-related claims

Using asymmetric (public key) algorithms

Useful references

Summary

Testing

Testing your application

The anatomy of a test

Testing guidelines

Unit testing 

Writing a unit test

Mock objects and patching

Assertions

Testing a CSV generator

Boundaries and granularity

Testing the export function

Final considerations

Test-driven development

Summary

Debugging and Profiling

Debugging techniques

Debugging with print

Debugging with a custom function

Using the Python debugger

Inspecting logs

Other techniques

Reading tracebacks

Assertions

Where to find information

Troubleshooting guidelines

Where to inspect

Using tests to debug

Monitoring

Profiling Python

When to profile

Measuring execution time

Summary

GUIs and Scripting

First approach: scripting

The imports

Parsing arguments

The business logic

Second approach: a GUI application

The imports

The layout logic

The business logic

Fetching the web page

Saving the images

Alerting the user

How can we improve the application?

Where do we go from here?

The turtle module

wxPython, Kivy, and PyQt

The principle of least astonishment

Threading considerations

Summary

Data Science in Brief

IPython and Jupyter Notebook

Using Anaconda

Starting a Notebook

Dealing with data

Setting up the Notebook

Preparing the data

Cleaning the data

Creating the DataFrame

Unpacking the campaign name

Unpacking the user data

Cleaning everything up

Saving the DataFrame to a file

Visualizing the results

Where do we go from here?

Summary

Introduction to API Development

What is the Web?

How does the Web work?

Response status codes

Type hinting: An overview

Why type hinting?

Type hinting in a nutshell

APIs: An introduction

What is an API?

What is the purpose of an API?

API protocols

API data-exchange formats

The railway API

Modeling the database

Main setup and configuration

Adding settings

Station endpoints

Reading data

Creating data

Updating data

Deleting data

User authentication

Documenting the API

Consuming an API

Calling the API from Django

Where do we go from here?

Summary

Packaging Python Applications

The Python Package Index

The train schedule project

Packaging with setuptools

Required files

pyproject.toml

License

README

Changelog

setup.cfg

setup.py

MANIFEST.in

Package metadata

Accessing metadata in your code

Defining the package contents

Accessing package data files

Specifying dependencies

Entry points

Building and publishing packages

Building

Publishing

Advice for starting new projects

Alternative tools

Further reading

Summary

Other Books You May Enjoy

Index

Landmarks

Cover

Index

Share Your Thoughts

Once you've read Learn Python Programming, Third edition, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application. 

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781801815093

Submit your proof of purchaseThat’s it! We’ll send your free PDF and other benefits to your email directly

1

A Gentle Introduction to Python

"Give a man a fish and you feed him for a day. Teach a man to fish and you feed him for a lifetime."

– Chinese proverb

According to Wikipedia, computer programming is:

"...the process of designing and building an executable computer program to accomplish a specific computing result or to perform a specific task. Programming involves tasks such as: analysis, generating algorithms, profiling algorithms' accuracy and resource consumption, and the implementation of algorithms in a chosen programming language (commonly referred to as coding)."

(https://en.wikipedia.org/wiki/Computer_programming)

In a nutshell, computer programming, or coding, as it is sometimes known, is telling a computer to do something using a language it understands.

Computers are very powerful tools, but unfortunately, they can't think for themselves. They need to be told everything: how to perform a task; how to evaluate a condition to decide which path to follow; how to handle data that comes from a device, such as a network or a disk; and how to react when something unforeseen happens, in the case of, say, something being broken or missing.

You can code in many different styles and languages. Is it hard? We would say yes and no. It's a bit like writing—it is something that everybody can learn. But what if you want to become a poet? Writing alone is not enough. You have to acquire a whole other set of skills, and this will involve a longer and greater effort.

In the end, it all comes down to how far you want to go down the road. Coding is not just putting together some instructions that work. It is so much more!

Good code is short, fast, elegant, easy to read and understand, simple, easy to modify and extend, easy to scale and refactor, and easy to test. It takes time to be able to write code that has all these qualities at the same time, but the good news is that you're taking the first step towards it at this very moment by reading this book. And we have no doubt you can do it. Anyone can; in fact, we all program all the time, only we aren't aware of it. Take the following example…

Say you want to make instant coffee. You have to get a mug, the instant coffee jar, a teaspoon, water, and the kettle. Even if you're not aware of it, you're evaluating a lot of data. You're making sure that there is water in the kettle and that the kettle is plugged in, that the mug is clean, and that there is enough coffee in the jar. Then, you boil the water and maybe, in the meantime, you put some coffee in the mug. When the water is ready, you pour it into the mug, and stir.

So, how is this programming?

Well, we gathered resources (the kettle, coffee, water, teaspoon, and mug) and we verified some conditions concerning them (the kettle is plugged in, the mug is clean, and there is enough coffee). Then we started two actions (boiling the water and putting coffee in the mug), and when both of them were completed, we finally ended the procedure by pouring water into the mug and stirring.

Can you see the parallel? We have just described the high-level functionality of a coffee program. It wasn't that hard because this is what the brain does all day long: evaluate conditions, decide to take actions, carry out tasks, repeat some of them, and stop at some point.

All you need now is to learn how to deconstruct all those actions you do automatically in real life so that a computer can actually make some sense of them. You need to learn a language as well so that the computer can be instructed.

So, this is what this book is for. We'll show you one way in which you can code successfully, and we'll try to do that by means of many simple but focused examples (our favorite kind).

In this chapter, we are going to cover the following:

Python's characteristics and ecosystemGuidelines on how to get up and running with Python and virtual environmentsHow to run Python programsHow to organize Python code and its execution model

A proper introduction

We love to make references to the real world when we teach coding; we believe they help people to better retain the concepts. However, now is the time to be a bit more rigorous and see what coding is from a more technical perspective.

When we write code, we are instructing a computer about the things it has to do. Where does the action happen? In many places: the computer memory, hard drives, network cables, the CPU, and so on. It's a whole world, which most of the time is the representation of a subset of the real world.

If you write a piece of software that allows people to buy clothes online, you will have to represent real people, real clothes, real brands, sizes, and so on and so forth, within the boundaries of a program.

In order to do so, you will need to create and handle objects in the program being written. A person can be an object. A car is an object. A pair of trousers is an object. Luckily, Python understands objects very well.

The two main features any object has are properties and methods. Let's take the example of a person as an object. Typically, in a computer program, you'll represent people as customers or employees. The properties that you store against them are things like a name, a Social Security number, an age, whether they have a driving license, an email, gender, and so on. In a computer program, you store all the data needed in order to use an object for the purpose that needs to be served. If you are coding a website to sell clothes, you probably want to store the heights and weights as well as other measures of your customers so that the appropriate clothes can be suggested to them. So, properties are characteristics of an object. We use them all the time: Could you pass me that pen? —Which one? —The black one. Here, we used the color (black) property of a pen to identify it (most likely it was being kept alongside different colored pens for the distinction to be necessary).

Methods are things that an object can do. As a person, I have methods such as speak, walk, sleep, wake up, eat, dream, write, read, and so on. All the things that I can do could be seen as methods of the objects that represent me.

So, now that you know what objects are, that they expose methods that can be run and properties that you can inspect, you're ready to start coding. Coding, in fact, is simply about managing those objects that live in the subset of the world that we're reproducing in our software. You can create, use, reuse, and delete objects as you please.

According to the Data Model chapter on the official Python documentation (https://docs.python.org/3/reference/datamodel.html):

"Objects are Python's abstraction for data. All data in a Python program is represented by objects or by relations between objects."

We'll take a closer look at Python objects in Chapter 6, OOP, Decorators, and Iterators. For now, all we need to know is that every object in Python has an ID (or identity), a type, and a value.

Once created, the ID of an object is never changed. It's a unique identifier for it, and it's used behind the scenes by Python to retrieve the object when we want to use it. The type also never changes. The type states what operations are supported by the object and the possible values that can be assigned to it. We'll see Python's most important data types in Chapter 2, Built-In Data Types. The value can be changed or not: if it can, the object is said to be mutable; otherwise, it is said to be immutable.

How, then, do we use an object? We give it a name, of course! When you give an object a name, then you can use the name to retrieve the object and use it. In a more generic sense, objects, such as numbers, strings (text), and collections, are associated with a name. Usually, we say that this name is the name of a variable. You can see the variable as being like a box, which you can use to hold data.

So, you have all the objects you need; what now? Well, we need to use them, right? We may want to send them over a network connection or store them in a database. Maybe display them on a web page or write them into a file. In order to do so, we need to react to a user filling in a form, or pressing a button, or opening a web page and performing a search. We react by running our code, evaluating conditions to choose which parts to execute, how many times, and under which circumstances.

To do all this, we need a language. That's what Python is for. Python is the language we will use together throughout this book to instruct the computer to do something for us.

Now, enough of this theoretical stuff—let's get started.

Enter the Python

Python is the marvelous creation of Guido Van Rossum, a Dutch computer scientist and mathematician who decided to gift the world with a project he was playing around with over Christmas 1989. The language appeared to the public somewhere around 1991, and since then has evolved to be one of the leading programming languages used worldwide today.

We started programming when we were both very young. Fabrizio started at the age of 7, on a Commodore VIC-20, which was later replaced by its bigger brother, the Commodore 64. The language it used was BASIC. Heinrich started when he learned Pascal in high school. Between us, we've programmed in Pascal, Assembly, C, C++, Java, JavaScript, Visual Basic, PHP, ASP, ASP .NET, C#, and plenty of others we can't even remember; only when we landed on Python did we finally have that feeling that you have when you find the right couch in the shop. When all of your body is yelling: Buy this one! This one is perfect!

It took us about a day to get used to it. Its syntax is a bit different from what we were used to, but after getting past that initial feeling of discomfort (like having new shoes), we both just fell in love with it. Deeply. Let's see why.

About Python

Before we get into the gory details, let's get a sense of why someone would want to use Python (we recommend you read the Python page on Wikipedia to get a more detailed introduction).

In our opinion, Python epitomizes the following qualities.

Portability

Python runs everywhere, and porting a program from Linux to Windows or Mac is usually just a matter of fixing paths and settings. Python is designed for portability and it takes care of specific operating system (OS) quirks behind interfaces that shield you from the pain of having to write code tailored to a specific platform.

Coherence

Python is extremely logical and coherent. You can see it was designed by a brilliant computer scientist. Most of the time you can just guess how a method is called if you don't know it.

You may not realize how important this is right now, especially if you aren't that experienced as a programmer, but this is a major feature. It means less cluttering in your head, as well as less skimming through the documentation, and less need for mappings in your brain when you code.

Developer productivity

According to Mark Lutz (Learning Python, 5th Edition, O'Reilly Media), a Python program is typically one-fifth to one-third the size of equivalent Java or C++ code. This means the job gets done faster. And faster is good. Faster means being able to respond more quickly to the market. Less code not only means less code to write, but also less code to read (and professional coders read much more than they write), maintain, debug, and refactor.

Another important aspect is that Python runs without the need for lengthy and time-consuming compilation and linkage steps, so there is no need to wait to see the results of your work.

An extensive library

Python has an incredibly extensive standard library (it is said to come with batteries included). If that wasn't enough, the Python international community maintains a body of third-party libraries, tailored to specific needs, which you can access freely at the Python Package Index (PyPI). When you code Python and realize that a certain feature is required, in most cases, there is at least one library where that feature has already been implemented.

Software quality

Python is heavily focused on readability, coherence, and quality. The language's uniformity allows for high readability, and this is crucial nowadays, as coding is more of a collective effort than a solo endeavor. Another important aspect of Python is its intrinsic multiparadigm nature. You can use it as a scripting language, but you can also exploit object-oriented, imperative, and functional programming styles—it is extremely versatile.

Software integration

Another important aspect is that Python can be extended and integrated with many other languages, which means that even when a company is using a different language as their mainstream tool, Python can come in and act as a gluing agent between complex applications that need to talk to each other in some way. This is more of an advanced topic, but in the real world, this feature is important.

Satisfaction and enjoyment

Last, but by no means least, there is the fun of it! Working with Python is fun; we can code for eight hours and leave the office happy and satisfied, unaffected by the struggle other coders have to endure because they use languages that don't provide them with the same amount of well-designed data structures and constructs. Python makes coding fun, no doubt about it. And fun promotes motivation and productivity.

These are the major aspects of why we would recommend Python to everyone. Of course, there are many other technical and advanced features that we could have mentioned, but they don't really pertain to an introductory section like this one. They will come up naturally, chapter after chapter, as we learn about Python in greater detail.

What are the drawbacks?

Probably, the only drawback that one could find in Python, which is not due to personal preferences, is its execution speed. Typically, Python is slower than its compiled siblings. The standard implementation of Python produces, when you run an application, a compiled version of the source code called byte code (with the extension .pyc), which is then run by the Python interpreter. The advantage of this approach is portability, which we pay for with increased runtimes due to the fact that Python is not compiled down to the machine level, as other languages are.

Despite this, Python speed is rarely a problem today, hence its wide use regardless of this aspect. What happens is that, in real life, hardware cost is no longer a problem, and usually it's easy enough to gain speed by parallelizing tasks. Moreover, many programs spend a great proportion of the time waiting for I/O operations to complete; therefore, the raw execution speed is often a secondary factor to the overall performance.

In situations where speed really is crucial, one can switch to faster Python implementations, such as PyPy, which provides, on average, just over a four-fold speedup by implementing advanced compilation techniques (check https://pypy.org/ for reference). It is also possible to write performance-critical parts of your code in faster languages, such as C or C++, and integrate that with your Python code. Libraries such as pandas and NumPy (which are commonly used for doing data science in Python) use such techniques.

There are a number of different implementations of the Python language. In this book, we will use the reference implementation, known as CPython. You can find a list of other implementations at: https://www.python.org/download/alternatives/

If that isn't convincing enough, you can always consider that Python has been used to drive the backend of services such as Spotify and Instagram, where performance is a concern. From this, it can be seen that Python has does its job perfectly well.

Who is using Python today?

Still not convinced? Let's take a very brief look at the companies using Python today: Google, YouTube, Dropbox, Zope Corporation, Industrial Light & Magic, Walt Disney Feature Animation, Blender 3D, Pixar, NASA, the NSA, Red Hat, Nokia, IBM, Netflix, Yelp, Intel, Cisco, HP, Qualcomm, JPMorgan Chase, and Spotify—to name just a few. Even games such as Battlefield 2, Civilization IV, and The Sims 4 are implemented using Python.

Python is used in many different contexts, such as system programming, web and API programming, GUI applications, gaming and robotics, rapid prototyping, system integration, data science, database applications, real-time communication, and much more. Several prestigious universities have also adopted Python as their main language in computer science courses.

Setting up the environment

Before talking about the installation of Python on your system, let us tell you about the Python version you will be using in this book.

Python 2 versus Python 3

Python comes in two main versions: Python 2, which is the older version, and Python 3, which is the most recent rendition. The two versions, though similar, are incompatible in some respects.

In the real world, Python 2 is now only running legacy software. Python 3 has been out since 2008, and the lengthy transition phase from Version 2 has mostly come to an end. Python 2 was widely used in the industry, and it took a long time and sometimes a huge effort to make the transition. Some Python 2 software will never be updated to Python 3, simply because the cost and effort involved is not considered worth it. Some companies, therefore, prefer to keep their old legacy systems running just as they are, rather than updating them just for the sake of it.

At the time of writing, Python 2 has been deprecated and all of the most widely used libraries have been ported to Python 3. It is strongly recommended to start new projects in Python 3.

During the transition phase, many libraries were rewritten to be compatible with both versions, mostly harnessing the power of the six library (the name comes from the multiplication 2 x 3, due to the porting from Version 2 to 3), which helps you to introspect and adapt the behavior according to the version used. Now that Python 2 has reached its end of life (EOL), some libraries have started to reverse that trend and are dropping support for Python 2.

According to PEP 373 (https://legacy.python.org/dev/peps/pep-0373/), theEOL of Python 2.7 was set to 2020. The last version is 2.7.18; there will not be a Python 2.8.

On Fabrizio's machine (MacBook Pro), this is the latest Python version:

>>>import sys >>> print(sys.version) 3.9.2 (default, Mar 1 2021, 23:29:21) [Clang 12.0.0 (clang-1200.0.32.29)]

So, you can see that the version is 3.9.2, which was out on the 1st of March 2021. The preceding text is a little bit of Python code that was typed into a console. We'll talk about this in a moment.

All the examples in this book will be run using Python 3.9. If you wish to follow the examples and download the source code for this book, please make sure you are using the same version.

Installing Python

We never really understood the point of having a setup section in a book, regardless of what it is that you have to set up. Most of the time, between the time the author writes the instructions and the time you actually try them out, months have passed—if you're lucky. One version change, and things may not work in the way they are described in the book. Luckily, we have the web now, so in order to help you get up and running, we will just give you pointers and objectives.

We are conscious that the majority of readers would probably have preferred to have guidelines in the book. We doubt it would have made their life easier, as we believe that if you want to get started with Python you have to put in that initial effort in order to get familiar with the ecosystem. It is very important, and it will boost your confidence to face the material in the chapters ahead. If you get stuck, remember that Google is your friend—when it comes to setting up, everything related to this can be found online.

Setting up the Python interpreter

First of all, let's talk about your OS. Python is fully integrated and, most likely, already installed in almost every Linux distribution. If you have a Mac, it's likely that Python is already there as well (although possibly only Python 2.7); if you're using Windows, however, you probably need to install it.

Getting Python and the libraries you need up and running requires a bit of handiwork. Linux and macOS seem to be the most user-friendly for Python programmers; Windows, on the other hand, may require a bit more effort.

The place you want to start is the official Python website: https://www.python.org. This website hosts the official Python documentation and many other resources that you will find very useful. Take the time to explore it.

Another excellent resource on Python and its ecosystem is https://docs.python-guide.org. You can find instructions there to set up Python on different operating systems, using different methods.

Find the appropriate "download" section and choose the installer for your OS. If you are on Windows, make sure that when you run the installer, you check the option install pip (actually, we would suggest making a complete installation, just to be safe, of all the components the installer holds). If you need more guidance on how to install Python on Windows, please check out this page on the official documentation: https://docs.python.org/3/using/windows.html.

Now that Python is installed on your system, the objective is to be able to open a console and run the Python interactive shell by typing python.

Please note that we usually refer to the Python interactive shell simply as the Python console.

To open the console in Windows, go to the Start menu, choose Run, and type cmd. If you encounter anything that looks like a permission problem while working on the examples in this book, please make sure you are running the console with administrator rights.

On macOS, start a Terminal by going to Applications>Utilities > Terminal.

If you are on Linux, chances are that you know all that there is to know about the console!

We will use the term console interchangeably to indicate the Linux console, the Windows Command Prompt, and the Macintosh Terminal. We will also indicate the command-line prompt with the Linux default format, like this:

$ sudo apt-get update

If you're not familiar with that, please take some time to learn the basics of how a console works. In a nutshell, after the $ sign, you will normally find an instruction that you have to type. Pay attention to capitalization and spaces, as they are very important.

Whatever console you open, type python at the prompt, and make sure the Python interactive shell shows up. Type exit() to quit. Keep in mind that you may have to specify python3 if your OS comes with Python 2 preinstalled.

This is roughly what you should see when you run Python (it will change in some details according to the version and OS):

fab $ python3 Python 3.9.2 (default, Mar 1 2021, 23:29:21) [Clang 12.0.0 (clang-1200.0.32.29)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>>

Now that Python is set up and you can run it, it is time to make sure you have the other tool that will be indispensable to follow the examples in the book: a virtual environment.

About virtual environments

When working with Python, it is very common to use virtual environments. Let's see what they are and why we need them by means of a simple example.

You install Python on your system and you start working on a website for Client X. You create a project folder and start coding. Along the way, you also install some libraries; for example, the Django framework, which we'll explore in Chapter 14, Introduction to API Development. Let's say the Django version you install for Project X is 2.2.

Now, your website is so good that you get another client, Y. She wants you to build another website, so you start Project Y and, along the way, you need to install Django again. The only issue is that now the Django version is 3.0 and you cannot install it on your system because this would replace the version you installed for Project X. You don't want to risk introducing incompatibility issues, so you have two choices: either you stick with the version you have currently on your machine, or you upgrade it and make sure the first project is still fully working correctly with the new version.

Let's be honest, neither of these options is very appealing, right? Definitely not. But there's a solution: virtual environments!

Virtual environments are isolated Python environments, each of which is a folder that contains all the necessary executables to use the packages that a Python project would need (think of packages as libraries for the time being).

So, you create a virtual environment for Project X, install all the dependencies, and then you create a virtual environment for Project Y, installing all its dependencies without the slightest worry because every library you install ends up within the boundaries of the appropriate virtual environment. In our example, Project X will hold Django 2.2, while Project Y will hold Django 3.0.

It is of vital importance that you never install libraries directly at the system level. Linux, for example, relies on Python for many different tasks and operations, and if you fiddle with the system installation of Python, you risk compromising the integrity of the whole system. So, take this as a rule, such as brushing your teeth before going to bed: always create a virtual environment when you start a new project.

When it comes to creating a virtual environment on your system, there are a few different methods to carry this out. As of Python 3.5, the suggested way to create a virtual environment is to use the venv module. You can look it up on the official documentation page (https://docs.python.org/3/library/venv.html) for further information.

If you're using a Debian-based distribution of Linux, for example, you will need to install the venv module before you can use it:

$ sudo apt-get install python3.9-venv

Another common way of creating virtual environments is to use the virtualenv third-party Python package. You can find it on its official website: https://virtualenv.pypa.io.

In this book, we will use the recommended technique, which leverages the venv module from the Python standard library.

Your first virtual environment

It is very easy to create a virtual environment, but according to how your system is configured and which Python version you want the virtual environment to run, you need to run the command properly. Another thing you will need to do, when you want to work with it, is to activate it. Activating virtual environments basically produces some path juggling behind the scenes so that when you call the Python interpreter from that shell, you're actually calling the active virtual environment one, instead of the system one. We will show you a full example on both Windows and Ubuntu (on a Mac, it's very similar to Ubuntu). We will:

Open a terminal and change into the folder (directory) we use as root for our projects (our folder is srv). We are going to create a new folder called my-project and change into it.Create a virtual environment called lpp3ed.After creating the virtual environment, we will activate it. The methods are slightly different between Linux, macOS, and Windows.Then, we will make sure that we are running the desired Python version (3.9.X) by running the Python interactive shell.Deactivate the virtual environment.

Some developers prefer to call all virtual environments the same name (for example, .venv). This way they can configure tools and run scripts against any virtual environment by just knowing the name of the project they are located in. The dot in .venv is there because in Linux/macOS, prepending a name with a dot makes that file or folder invisible.

These steps are all you need to start a project.

We are going to start with an example on Windows (note that you might get a slightly different result, according to your OS, Python version, and so on). In this listing, lines that start with a hash, #, are comments, spaces have been introduced for readability, and an arrow, →, indicates where the line has wrapped around due to lack of space:

C:\Users\Fab\srv>mkdir my-project # step 1 C:\Users\Fab\srv>cd my-project C:\Users\Fab\srv\my-project>where python # check system python C:\Users\Fab\AppData\Local\Programs\Python\Python39\python.exe C:\Users\Fab\AppData\Local\Microsoft\WindowsApps\python.exe C:\Users\Fab\srv\my-project>python -m venv lpp3ed # step 2 C:\Users\Fab\srv\my-project>lpp3ed\Scripts\activate # step 3 # check python again, now virtual env python is listed first (lpp3ed) C:\Users\Fab\srv\my-project>where python C:\Users\Fab\srv\my-project\lpp3ed\Scripts\python.exe C:\Users\Fab\AppData\Local\Programs\Python\Python39\python.exe C:\Users\Fab\AppData\Local\Microsoft\WindowsApps\python.exe (lpp3ed) C:\Users\Fab\srv\my-project>python # step 4 Python 3.9.2 (tags/v3.9.2:1a79785, Feb 19 2021, 13:44:55) → [MSC v.1928 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> exit() (lpp3ed) C:\Users\Fab\srv\my-project>deactivate # step 5 C:\Users\Fab\srv\my-project>

Each step has been marked with a comment, so you should be able to follow along quite easily.

On a Linux machine, the steps are the same, but the commands are structured slightly differently. Moreover, you might have to execute some additional setup steps to be able to use the venv module to create a virtual environment. It is impossible to give instructions for all the Linux distributions out there, so please have a look online to find what is appropriate for your distribution.

Once you are set up, these are the instructions necessary to create a virtual environment:

fab@fvm:~/srv$ mkdir my-project # step 1 fab@fvm:~/srv$ cd my-project fab@fvm:~/srv/my-project$ which python3.9 # check system python /usr/bin/python3.9 # <-- system python3.9 fab@fvm:~/srv/my-project$ python3.9 -m venv lpp3ed # step 2 fab@fvm:~/srv/my-project$ source ./lpp3ed/bin/activate # step 3 # check python again: now using the virtual environment's one (lpp3ed) fab@fvm:~/srv/my-project$ which python /home/fab/srv/my-project/lpp3ed/bin/python (lpp3ed) fab@fvm:~/srv/my-project$ python # step 4 Python 3.9.2 (default, Feb 20 2021, 20:56:08) [GCC 9.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> exit() (lpp3ed) fab@fvm:~/srv/my-project$ deactivate # step 5 fab@fvm:~/srv/my-project$

Something to notice here is that in order to activate the virtual environment, we need to run the lpp3ed/bin/activate script, which needs to be sourced. When a script is sourced, it means that it is executed in the current shell, and therefore its effects last after the execution. This is very important. Also notice how the prompt changes after we activate the virtual environment, showing its name on the left (and how it disappears when we deactivate it).

At this point, you should be able to create and activate a virtual environment. Please try and create another one without us guiding you. Get acquainted with this procedure—it is something that you will always be doing: we never work system-wide with Python, remember? Virtual environments are extremely important.

The source code for the book contains a dedicated folder for each chapter. When the code shown in the chapter requires third-party libraries to be installed, we will include a requirements.txt file (or an equivalent requirements folder with more than one text file inside) that you can use to install the libraries required to run that code. We suggest that when experimenting with the code for a chapter, you create a dedicated virtual environment for that chapter. This way, you will be able to get some practice in the creation of virtual environments, and the installation of third-party libraries.

Installing third-party libraries

In order to install third-party libraries, we need to use the Python Package Installer, known as pip. Chances are that it is already available to you within your virtual environment, but if not, you can learn all about it on its documentation page: https://pip.pypa.io.

The following example shows how to create a virtual environment and install a couple of third-party libraries taken from a requirements file.

mpro:srv fab$ mkdir my-project mpro:srv fab$ cd my-project/ mpro:my-project fab$ python3.9 -m venv lpp3ed mpro:my-project fab$ source ./lpp3ed/bin/activate (lpp3ed) mpro:my-project fab$ cat requirements.txt Django==3.1.7 requests==2.25.1 # the following instruction shows how to use pip to install # requirements from a file (lpp3ed) mpro:my-project fab$ pip install -r requirements.txt Collecting Django==3.1.7 Using cached Django-3.1.7-py3-none-any.whl (7.8 MB) ... much more collection here ... Collecting requests==2.25.1 Using cached requests-2.25.1-py2.py3-none-any.whl (61 kB) Installing collected packages: ..., Django, requests, ... Successfully installed Django-3.1.7 ... requests-2.25.1 ... (lpp3ed) mpro:my-project fab$

As can be seen at the bottom of the listing, pip has installed both libraries that are in the requirements file, plus a few more. This happened because both django and requests have their own list of third-party libraries that they depend upon, and therefore pip will install them automatically for us.

Now, with the scaffolding out of the way, we are ready to talk a bit more about Python and how it can be used. Before we do that though, allow us to say a few words about the console.

Your friend, the console

In this, the era of GUIs and touchscreen devices, it seems a little ridiculous to have to resort to a tool such as the console, when everything is just about one click away.

But the truth is every time you remove your right hand from the keyboard (or the left one, if you're a lefty) to grab your mouse and move the cursor over to the spot you want to click on, you're losing time. Getting things done with the console, counter-intuitive though it may at first seem, results in higher productivity and speed. Believe us, we know—you will have to trust us on this.

Speed and productivity are important, and even though we have nothing against the mouse, being fluent with the console is very good for another reason: when you develop code that ends up on some server, the console might be the only available tool to access the code on that server. If you make friends with it, you will never get lost when it is of utmost importance that you don't (typically, when the website is down and you have to investigate very quickly what has happened).

If you're still not convinced, please us the benefit of the doubt and give it a try. It's easier than you think, and you won't regret it. There is nothing more pitiful than a good developer who gets lost within an SSH connection to a server because they are used to their own custom set of tools, and only to that.

Now, let's get back to Python.

How to run a Python program

There are a few different ways in which you can run a Python program.

Running Python scripts

Python can be used as a scripting language; in fact, it always proves itself very useful. Scripts are files (usually of small dimensions) that you normally execute to do something like a task. Many developers end up having their own arsenal of tools that they fire when they need to perform a task. For example, you can have scripts to parse data in a format and render it into another one; or you can use a script to work with files and folders; you can create or modify configuration files—technically, there is not much that cannot be done in a script.

It is rather common to have scripts running at a precise time on a server. For example, if your website database needs cleaning every 24 hours (for example, the table that stores the user sessions, which expire pretty quickly but aren't cleaned automatically), you could set up a Cron job that fires your script at 3:00 A.M. every day.

According to Wikipedia, the software utility Cron is a time-based job scheduler in Unix-like computer operating systems. People who set up and maintain software environments use Cron (or a similar technology) to schedule jobs (commands or shell scripts) to run periodically at fixed times, dates, or intervals.

We have Python scripts to do all the menial tasks that would take us minutes or more to do manually, and at some point, we decided to automate. We'll devote half of Chapter 12, GUIs and Scripting, to scripting with Python.

Running the Python interactive shell

Another way of running Python is by calling the interactive shell. This is something we already saw when we typed python on the command line of our console.

So, open up a console, activate your virtual environment (which by now should be second nature to you, right?), and type python