Functional Python Programming, 3rd edition E-Book

Steven F. Lott

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 680

Veröffentlichungsjahr: 2022

Beschreibung

Not enough developers understand the benefits of functional programming, or even what it is. Author Steven Lott demystifies the approach, teaching you how to improve the way you code in Python and make gains in memory use and performance. If you’re a leetcoder preparing for coding interviews, this book is for you.
Starting from the fundamentals, this book shows you how to apply functional thinking and techniques in a range of scenarios, with Python 3.10+ examples focused on mathematical and statistical algorithms, data cleaning, and exploratory data analysis. You'll learn how to use generator expressions, list comprehensions, and decorators to your advantage. You don't have to abandon object-oriented design completely, though – you'll also see how Python's native object orientation is used in conjunction with functional programming techniques.
By the end of this book, you'll be well-versed in the essential functional programming features of Python and understand why and when functional thinking helps. You'll also have all the tools you need to pursue any additional functional topics that are not part of the Python language.

Leseprobe

Functional Python Programming

Third Edition

Use a functional approach to write succinct, expressive, and efficient Python code

Steven F. Lott

BIRMINGHAM—MUMBAI

”Python” and the Python logo are trademarks of the Python Software Foundation.

Functional Python Programming

Third Edition

Copyright © 2022 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrievalsystem, or transmitted in any form or by any means, without the prior writtenpermission of the publisher, except in the case of brief quotations embedded incritical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracyof the information presented. However, the information contained in this book issold without warranty, either express or implied. Neither the author, nor PacktPublishing or its dealers and distributors, will be held liable for any damages causedor alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of thecompanies and products mentioned in this book by the appropriate use of capitals.However, Packt Publishing cannot guarantee the accuracy of this information.

Senior Publishing Product Manager: Denim Pinto

Acquisition Editor – Peer Reviews: Gaurav Gavas

Project Editor: Parvathy Nair

Development Editor: Lucy Wan

Copy Editor: Safis Editing

Technical Editor: Karan Sonawane

Indexer: Hemangini Bari

Proofreader: Safis Editing

Presentation Designer: Sandip Tadge

First published: January 2015

Second edition: April 2018

Third edition: December 2022

Production reference: 1231222

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80323-257-7

www.packt.com

Foreword

Python is an incredibly versatile language that offers a lot of perks for just about every group. For the object-oriented programming fans, it has classes and inheritance. When we talk about functional programming, it has functions as a first-class type, higher-order functions such as map and reduce, and a handy syntax for comprehensions and generators. Perhaps best of all, it doesn’t force any of those on the user – it’s still totally OK to write a script in Python without a single class or function and not feel guilty about it.

Thinking in terms of functional programming, having in mind the goals of minimizing state and side effects, writing pure functions, reducing intermediary data, and what depends on what else will also allow you to see your code under a new light. It’ll also allow you to write more compact, performant, testable, and maintainable code, where instead of writing a program to solve your problem, you “write the language up”, adding new functions to it until expressing the solution you designed is simple and straightforward. This is an extremely powerful mind shift – and an exercise worth doing. It’s a bit like learning a new language, such as Lisp or Forth (or German, or Irish), but without having to leave the comfort of your Python environment.

Not being a pure functional language has its costs, however. Python lacks many features functional languages can use to provide better memory efficiency and speed. Python’s strongest point remains its accessibility – you can fire up your Python interpreter and start playing with the examples in this book right away. This interactive approach allows exploratory programming, where you test ideas easily, and only later need to incorporate them into a more complex program (or not – like I said, it’s totally OK to write a simple script).

This book is intended for people already familiar with Python. You don’t need to know much about functional programming – the book will guide you through many common approaches, techniques, and patterns used in functional programming and how they can be best expressed in Python. Think of this book as an introduction – it’ll give you the basic tools to see, think, and express your ideas in functional terms using Python.

Ricardo Bánffy

Software Engineer, Architect, Evangelist, and Passionate Pythonista

Contributors

About the author

Steven F. Lott has been programming since computers were large, expensive, and rare. Working for decades in high tech has given him exposure to a lot of ideas and techniques; some are bad, but most are useful and helpful to others.

Steven has been working with Python since the ‘90s, building a variety of tools and applications. He’s written a number of titles for Packt Publishing, include Mastering Object-Oriented Python, Modern Python Cookbook, and FunctionalPython Programming.

He’s a technomad, and lives on a boat that’s usually located on the east coast of the US. He tries to live by the words, ”Don’t come home until you have a story.”

About the reviewers

Alex Martelli is a Fellow of the Python Software Foundation, a winner of the Frank Willison Memorial Award for contributions to the Python community, and a top-page reputation hog on Stack Overflow. He spent 8 years with IBM Research, then 13 years at Think3 Inc., followed by 4 years as a consultant, and lately 17 years at Google. He has taught programming languages, development methods, and numerical computing at Ferrara University and other venues.

Books he has authored or co-authored include two editions of Python Cookbook, four editions of Python in a Nutshell, and a chapter in Beautiful Teams. Dozens of his interviews and tech talks at conferences are available on YouTube. Alex’s proudest achievement are the articles that appeared in Bridge World (January and February 2000), which were hailed as giant steps towards solving issues that had haunted contract bridge theoreticians for decades, and still get quoted in current bridge-theoretical literature.

Tiago Antao has a BEng in Informatics and a PhD in Life Sciences. He works in the Big Data space, analyzing very large datasets and implementing complex data processing algorithms. He leverages Python with all its libraries to carry out scientific computing and data engineering tasks. He also uses low-level programming languages like C, C++, and Rust to optimize critical parts of algorithms. Tiago develops on an infrastructure based on AWS, but has used on-premises computing and scientific clusters for most of his career.

While he currently works in industry, he also has exposure to the academic side of scientific computing, with two data analysis postdocs at the universities of Cambridge and Oxford, and a research scientist position at the University of Montana, where he set up, from scratch, the scientific computing infrastructure for the analysis of biological data.

He is one of the co-authors of Biopython, a major bioinformatics package written in Python. He wrote Bioinformatics with Python Cookbook, which is on its third edition. He has also authored and co-authored many high-impact scientific articles in the field of bioinformatics.

Join our community Discord space

Join our Python Discord workspace to discuss and know more about the book: https://packt.link/dHrHU

Functional Python Programming - Third Edition

Preface

Chapter 1: Understanding Functional Programming

1.1

The functional style of programming

1.2

Comparing and contrasting procedural and functional styles

1.3

A classic example of functional programming

1.4

Exploratory data analysis

1.5

Summary

1.6

Exercises

Join our community Discord space

Chapter 2: Introducing Essential Functional Concepts

2.1

Functions as first-class objects

2.2

Immutable data

2.3

Strict and non-strict evaluation

2.4

Lazy and eager evaluation

2.5

Recursion instead of an explicit loop state

2.6

Functional type systems

2.7

Familiar territory

2.8

Learning some advanced concepts

2.9

Summary

2.10

Exercises

Join our community Discord space

Chapter 3: Functions, Iterators, and Generators

3.1

Writing pure functions

3.2

Functions as first-class objects

3.3

Using strings

3.4

Using tuples and named tuples

3.5

Using generator expressions

3.6

Cleaning raw data with generator functions

3.7

Applying generators to built-in collections

3.8

Summary

3.9

Exercises

Join our community Discord space

Chapter 4: Working with Collections

4.1

An overview of function varieties

4.2

Working with iterables

4.3

Using any() and all() as reductions

4.4

Using len() and sum() on collections

4.5

Using zip() to structure and flatten sequences

4.6

Using sorted() and reversed() to change the order

4.7

Using enumerate() to include a sequence number

4.8

Summary

4.9

Exercises

Join our community Discord space

Chapter 5: Higher-Order Functions

5.1

Using max() and min() to find extrema

5.2

Using the map() function to apply a function to a collection

5.3

Using the filter() function to pass or reject data

5.4

The iter() function with a sentinel value

5.5

Using sorted() to put data in order

5.6

Overview of writing higher-order functions

5.7

Writing higher-order mappings and filters

5.8

Building higher-order functions with callables

5.9

Review of some design patterns

5.10

Summary

5.11

Exercises

Join our community Discord space

Chapter 6: Recursions and Reductions

6.1

Simple numerical recursions

6.2

Reductions and folding a collection from many items to one

6.3

Group-by reduction from many items to fewer

6.4

Summary

6.5

Exercises

Join our community Discord space

Chapter 7: Complex Stateless Objects

7.1

Using tuples to collect data

7.2

Using NamedTuple to collect data

7.3

Using frozen dataclasses to collect data

7.4

Complicated object initialization and property computations

7.5

Using pyrsistent to collect data

7.6

Avoiding stateful classes by using families of tuples

7.7

Polymorphism and type pattern matching

7.8

Summary

7.9

Exercises

Join our community Discord space

Chapter 8: The Itertools Module

8.1

Working with the infinite iterators

8.2

Using the finite iterators

8.3

Cloning iterators with tee()

8.4

The itertools recipes

8.5

Summary

8.6

Exercises

Join our community Discord space

Chapter 9: Itertools for Combinatorics – Permutations and Combinations

9.1

Enumerating the Cartesian product

9.2

Reducing a product

9.3

Performance improvements

9.4

Permuting a collection of values

9.5

Generating all combinations

9.6

Recipes

9.7

Summary

9.8

Exercises

Join our community Discord space

Chapter 10: The Functools Module

10.1

Function tools

10.2

Memoizing previous results with cache

10.3

Defining classes with total ordering

10.4

Applying partial arguments with partial()

10.5

Reducing sets of data with the reduce() function

10.6

Handling multiple types with singledispatch

10.7

Summary

10.8

Exercises

Join our community Discord space

Chapter 11: The Toolz Package

11.1

The itertools star map function

11.2

Reducing with operator module functions

11.3

Using the toolz package

11.4

Summary

11.5

Exercises

Chapter 12: Decorator Design Techniques

12.1

Decorators as higher-order functions

12.2

Cross-cutting concerns

12.3

Composite design

12.4

Adding a parameter to a decorator

12.5

Implementing more complex decorators

12.6

Complicated design considerations

12.7

Summary

12.8

Exercises

Chapter 13: The PyMonad Library

13.1

Downloading and installing

13.2

Functional composition and currying

13.3

Functors – making everything a function

13.4

Monad bind() function

13.5

Implementing simulation with monads

13.6

Additional PyMonad features

13.7

Summary

13.8

Exercises

Join our community Discord space

Chapter 14: The Multiprocessing, Threading, and Concurrent.Futures Modules

14.1

Functional programming and concurrency

14.2

What concurrency really means

14.3

Using multiprocessing pools and tasks

14.4

Using a multiprocessing pool for concurrent processing

14.5

Summary

14.6

Exercises

Chapter 15: A Functional Approach to Web Services

15.1

The HTTP request-response model

15.2

The WSGI standard

15.3

Defining web services as functions

15.4

Tracking usage

15.5

Summary

15.6

Exercises

Join our community Discord space

Other Books You Might Enjoy

Index

Landmarks

Table of Contents

Cover

Preface

Functional programming offers a variety of techniques for creating succinct and expressive software. While Python is not a purely functional programming language, we can do a great deal of functional programming in Python.

Python has a core set of functional programming features. This lets us borrow many design patterns and techniques from other functional languages. These borrowed concepts can lead us to create elegant programs. Python’s generator expressions, in particular, negate the need to create large in-memory data structures, leading to programs that may execute more quickly because they use fewer resources.

We can’t easily create purely functional programs in Python. Python lacks a number of features that would be required for this. We don’t have unlimited recursion, for example, we don’t have lazy evaluation of all expressions, and we don’t have an optimizing compiler.

There are several key features of functional programming languages that are available in Python. One of the most important ones is the idea of functions being first-class objects. Python also offers a number of higher-order functions. The built-in map(), filter(), and functools.reduce() functions are widely used in this role, and less obvious are functions such as sorted(), min(), and max().

In some cases, a functional approach to a problem will also lead to extremely high-performance algorithms. Python makes it too easy to create large intermediate data structures, tying up memory (and processor time). With functional programming design patterns, we can often replace large lists with generator expressions that are equally expressive but take up much less memory and run much more quickly.

We’ll look at the core features of functional programming from a Python point of view. Our objective is to borrow good ideas from functional programming languages and use those ideas to create expressive and succinct applications in Python.

Who this book is for

This book is for more experienced programmers who want to create succinct, expressive Python programs by borrowing techniques and design patterns from functional programming languages. Some algorithms can be expressed elegantly in a functional style; we can—and should—adapt this to make Python programs more readable and maintainable.

This is not intended as a tutorial on Python. This book assumes some familiarity with the language and the standard library. For a foundational introduction to Python, consider Learn Python Programming, Third Edition: https://www.packtpub.com/product/learn-python-programming-third-edition/9781801815093.

While we cover the foundations of functional programming, this is not a complete review of the various kinds of functional programming techniques. Having an exposure to functional programming in another language can be helpful.

What this book covers

We can decompose this book into two general kinds of topics:

Essentials of functional programming in Python. This is the content of Chapters 1 through 7.

Library modules to help create functional programs. This is the subject of the remaining chapters of the book. Chapter 12 includes both fundamental language and library topics.

Chapter 1, Understanding Functional Programming, introduces some of the techniques that characterize functional programming. We’ll identify some of the ways to map those features to Python. We’ll also address some ways that the benefits of functional programming accrue when we use these design patterns to build Python applications.

Chapter 2, Introducing Essential Functional Concepts, delves into central features of the functional programming paradigm. We’ll look at each in some detail to see how they’re implemented in Python. We’ll also point out some features of functional languages that don’t apply well to Python. In particular, many functional languages have complex type-matching rules required to support compiling and optimizing.

Chapter 3, Functions, Iterators, and Generators, will show how to leverage immutable Python objects, and how generator expressions adapt functional programming concepts to the Python language. We’ll look at some of the built-in Python collections and how we can leverage them without departing too far from functional programming concepts.

Chapter 4, Working with Collections, shows how you can use a number of built-in Python functions to operate on collections of data. This chapter will focus on a number of relatively simple functions, such as any() and all(), which reduce a collection of values to a single result.

Chapter 5, Higher-Order Functions, examines the commonly used higher-order functions such as map() and filter(). It also shows a number of other higher-order functions as well as how we can create our own functions that work with functions or return functions.

Chapter 6, Recursions and Reductions, teaches how to design an algorithm using recursion and then optimize it into a high-performance for statement. We’ll also look at some other reductions that are widely used, including collections.Counter().

Chapter 7, Complex Stateless Objects, showcases a number of ways that we can use immutable tuples, typing.NamedTuple, and the frozen @dataclass instead of stateful objects. We’ll also look at the pyrsistent module as a way to create immutable objects. Immutable objects have a simpler interface than stateful objects: we never have to worry about abusing an attribute and setting an object into some inconsistent or invalid state.

Chapter 8, The Itertools Module, examines a number of functions in the itertools standard library module. This collection of functions simplifies writing programs that deal with collections or generator functions.

Chapter 9, Itertools for Combinatorics – Permutations and Combinations, covers the combinatoric functions in the itertools module. These functions are more specialized than those in the previous chapter. This chapter includes some examples that illustrate ill-considered use of these functions and the consequences of combinatoric explosion.

Chapter 10, The Functools Module, focuses on how to use some of the functions in the functools module for functional programming. A few functions in this module are more appropriate for building decorators, and they are left for Chapter 12, Decorator Design Techniques.

Chapter 11, The Toolz Package, covers the toolz package, a number of closely related modules that help us write functional programs in Python. The toolz modules parallel the built-in itertools and functools modules, providing alternatives that are often more sophisticated and make better use of curried functions.

Chapter 12, Decorator Design Techniques, covers how we can look at a decorator as a way to build a composite function. While there is considerable flexibility here, there are also some conceptual limitations: we’ll look at ways that overly complex decorators can become confusing rather than helpful.

Chapter 13, The PyMonad Library, examines some of the features of the PyMonad library. This provides some additional functional programming features. It also provides a way to learn more about monads. In some functional languages, monads are an important way to force a particular order for operations that might get optimized into an undesirable order. Since Python already has strict ordering of expressions and statements, the monad feature is more instructive than practical.

Chapter 14, The Multiprocessing, Threading, and Concurrent.FuturesModules, points out an important consequence of good functional design: we can distribute the processing workload. Using immutable objects means that we can’t corrupt an object because of poorly synchronized write operations.

Chapter 15, A Functional Approach to Web Services, shows how we can think of web services as a nested collection of functions that transform a request into a reply. We’ll see ways to leverage functional programming concepts for building responsive, dynamic web content.

Chapter 16, A Chi-Squared Case Study, is a bonus, online-only case study applying a number of functional programming techniques to a specific exploratory data analysis problem. We will apply a χ2 statistical test to some complex data to see if the results show ordinary variability, or if they are an indication of something that requires deeper analysis. You can find the case study here: https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition/blob/main/Bonus_Content/Chapter_16.pdf.

To get the most out of this book

This book presumes some familiarity with Python 3 and general concepts of application development. We won’t look deeply at subtle or complex features of Python; we’ll avoid much consideration of the internals of the language.

Some of the examples use exploratory data analysis (EDA) as a problem domain to show the value of functional programming. Some familiarity with basic probability and statistics will help with this. There are only a few examples that move into more serious data science.

Python 3.10 is required. The examples have also been tested with Python 3.11, and work correctly. For data science purposes, it’s often helpful to start with the conda tool to create and manage virtual environments. It’s not required, however, and readers should be able to use any available Python.

Additional packages are generally installed with pip. The command looks like this:

  % python -m pip install toolz pymonad pyrsistent beautifulsoup4

Complete the exercises

Each chapter includes a number of exercises that help the reader apply the concepts in the chapter to real code. Most of the exercises are based on code available from the book’s repository on GitHub: https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition.

In some cases, the reader will notice that the code provided on GitHub includes partial solutions to some of the exercises. These serve as hints, allowing the reader to explore alternative solutions.

In many cases, exercises will need unit test cases to confirm they actually solve the problem. These are often identical to the unit test cases already provided in the GitHub repository. The reader should replace the book’s example function name with their own solution to confirm that it works.

In some cases, the exercises suggest writing a response document to compare and contrast multiple solutions. It helps to find a mentor or expert who can help the reader by reviewing these small documents for clarity and completeness. A good comparison between design approaches will include performance measurements using the timeit module to show the performance advantages of one design over another.

Download the example code files

The code bundle for the book is hosted on GitHub at https://github.com/PacktPublishing/Functional-Python-Programming-3rd-Edition. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Download the color images

We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://packt.link/OV1CB.

Conventions used

There are a number of text conventions used throughout this book.

CodeInText: Indicates code words in the text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example: ”Python has other statements, such as global or nonlocal, which modify the rules for variables in a particular namespace.”

Bold: Indicates a new term, an important word, or words you see on the screen, such as in menus or dialog boxes. For example: ”The base case states that the sum of a zero-length sequence is 0. The recursive case states that the sum of a sequence is the first value plus the sum of the rest of the sequence.”

A block of code is set as follows:

print("Hello, World!")

Any command-line input or output is written as follows:

  % conda create -n functional3 python=3.10

Warnings or important notes appear like this.

Tips and tricks appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: Email [email protected], and mention the book’s title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book we would be grateful if you would report this to us. Please visit https://subscription.packtpub.com/help, click on the Submit Errata button, search for your book, and enter the details.

Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit http://authors.packtpub.com.

Share your thoughts

Once you’ve read Functional Python Programming, Third Edition, we’d love to hear your thoughts! Scan the QR code below to go straight to the Amazon review page for this book and share your feedback.

https://packt.link/r/1803232579

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere? Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

Scan the QR code or visit the link below

https://packt.link/free-ebook/9781803232577

Submit your proof of purchase

That’s it! We’ll send your free PDF and other benefits to your email directly