Python Object-Oriented Programming E-Book

Steven F. Lott

• Herausgeber: Packt Publishing
• Kategorie: Wissenschaft und neue Technologien
• Sprache: Englisch

Seitenzahl: 1052

Veröffentlichungsjahr: 2021

Beschreibung

Object-oriented programming (OOP) is a popular design paradigm in which data and behaviors are encapsulated in such a way that they can be manipulated together. Python Object-Oriented Programming, Fourth Edition dives deep into the various aspects of OOP, Python as an OOP language, common and advanced design patterns, and hands-on data manipulation and testing of more complex OOP systems. These concepts are consolidated by open-ended exercises, as well as a real-world case study at the end of every chapter, newly written for this edition. All example code is now compatible with Python 3.9+ syntax and has been updated with type hints for ease of learning.

Steven and Dusty provide a comprehensive, illustrative tour of important OOP concepts, such as inheritance, composition, and polymorphism, and explain how they work together with Python’s classes and data structures to facilitate good design. In addition, the book also features an in-depth look at Python’s exception handling and how functional programming intersects with OOP. Two very powerful automated testing systems, unittest and pytest, are introduced. The final chapter provides a detailed discussion of Python's concurrent programming ecosystem.

By the end of the book, you will have a thorough understanding of how to think about and apply object-oriented principles using Python syntax and be able to confidently create robust and reliable programs.

Leseprobe

Python Object-Oriented Programming

Fourth Edition

Build robust and maintainable object-oriented Python applications and libraries

Steven F. Lott

Dusty Phillips

BIRMINGHAM—MUMBAI

"Python" and the Python Logo are trademarks of the Python Software Foundation.

Python Object-Oriented Programming

Fourth Edition

Copyright © 2021 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

Producer: Dr. Shailesh Jain

Acquisition Editor – Peer Reviews: Saby D'silva

Project Editor: Parvathy Nair

Content Development Editor: Lucy Wan

Copy Editor: Safis Editor

Technical Editor: Aditya Sawant

Proofreader: Safis Editor

Indexer: Tejal Daruwale Soni

Presentation Designer: Pranit Padwal

First published: July 2010

Second edition: August 2015

Third edition: October 2018

Fourth edition: June 2021

Production reference: 2281221

Published by Packt Publishing Ltd.

Livery Place

35 Livery Street

Birmingham

B3 2PB, UK.

ISBN 978-1-80107-726-2

www.packt.com

Contributors

About the authors

Steven Lott has been programming since computers were large, expensive, and rare. Working for decades in high tech has given him exposure to a lot of ideas and techniques—some bad, but most are useful and helpful to others.

Steven has been working with Python since the '90s, building a variety of tools and applications. He's written a number of titles for Packt Publishing, including Mastering Object-Oriented, Modern Python Cookbook, and Functional Python Programming.

He's a technomad, and lives on a boat that's usually located on the east coast of the US. He tries to live by the words "Don't come home until you have a story."

Dusty Phillips is a Canadian author and software developer. His storied career has included roles with the world's biggest government, the world's biggest social network, a two person startup, and everything in between. In addition to Python Object-Oriented Programming, Dusty wrote Creating Apps In Kivy (O'Reilly) and is now focused on writing fiction.

Thank you to Steven Lott, for finishing what I started, to all my readers for appreciating what I write, and to my wife, Jen Phillips, for everything else.

About the reviewer

Bernát Gábor, originally from Transylvania, works as a senior software engineer at Bloomberg in London, UK. During his day job, he primarily focuses on improving the quality of the data ingestion pipeline at Bloomberg with predominant use of the Python programming language and paradigms. He's been working with Python for over ten years and is a major open-source contributor in this domain, with a particular focus on the packaging area. He's also the author and maintainer of high-profile projects such as virtualenv, build, andtox. For more information, see https://bernat.tech/about.

I would like to express my very great appreciation to Lisa, my fiancée, for her invaluable support on a daily basis. Love you!

Contents

Preface

Who this book is for

What this book covers

To get the most out of this book

Get in touch

Object-Oriented Design

Introducing object-oriented

Objects and classes

Specifying attributes and behaviors

Data describes object state

Behaviors are actions

Hiding details and creating the public interface

Composition

Inheritance

Inheritance provides abstraction

Multiple inheritance

Case study

Introduction and problem overview

Context view

Logical view

Process view

Development view

Physical view

Conclusion

Recall

Exercises

Summary

Objects in Python

Introducing type hints

Type checking

Creating Python classes

Adding attributes

Making it do something

Talking to yourself

More arguments

Initializing the object

Type hints and defaults

Explaining yourself with docstrings

Modules and packages

Organizing modules

Absolute imports

Relative imports

Packages as a whole

Organizing our code in modules

Who can access my data?

Third-party libraries

Case study

Logical view

Samples and their states

Sample state transitions

Class responsibilities

The TrainingData class

Recall

Exercises

Summary

When Objects Are Alike

Basic inheritance

Extending built-ins

Overriding and super

Multiple inheritance

The diamond problem

Different sets of arguments

Polymorphism

Case study

Logical view

Another distance

Recall

Exercises

Summary

Expecting the Unexpected

Raising exceptions

Raising an exception

The effects of an exception

Handling exceptions

The exception hierarchy

Defining our own exceptions

Exceptions aren't exceptional

Case study

Context view

Processing view

What can go wrong?

Bad behavior

Creating samples from CSV files

Validating enumerated values

Reading CSV files

Don't repeat yourself

Recall

Exercises

Summary

When to Use Object-Oriented Programming

Treat objects as objects

Adding behaviors to class data with properties

Properties in detail

Decorators – another way to create properties

Deciding when to use properties

Manager objects

Removing duplicate code

In practice

Case study

Input validation

Input partitioning

The sample class hierarchy

The purpose enumeration

Property setters

Repeated if statements

Recall

Exercises

Summary

Abstract Base Classes and Operator Overloading

Creating an abstract base class

The ABCs of collections

Abstract base classes and type hints

The collections.abc module

Creating your own abstract base class

Demystifying the magic

Operator overloading

Extending built-ins

Metaclasses

Case study

Extending the list class with two sublists

A shuffling strategy for partitioning

An incremental strategy for partitioning

Recall

Exercises

Summary

Python Data Structures

Empty objects

Tuples and named tuples

Named tuples via typing.NamedTuple

Dataclasses

Dictionaries

Dictionary use cases

Using defaultdict

Counter

Lists

Sorting lists

Sets

Three types of queues

Case study

Logical model

Frozen dataclasses

NamedTuple classes

Conclusion

Recall

Exercises

Summary

The Intersection of Object-Oriented and Functional Programming

Python built-in functions

The len() function

The reversed() function

The enumerate() function

An alternative to method overloading

Default values for parameters

Additional details on defaults

Variable argument lists

Unpacking arguments

Functions are objects, too

Function objects and callbacks

Using functions to patch a class

Callable objects

File I/O

Placing it in context

Case study

Processing overview

Splitting the data

Rethinking classification

The partition() function

One-pass partitioning

Recall

Exercises

Summary

Strings, Serialization, and File Paths

Strings

String manipulation

String formatting

Escaping braces

f-strings can contain Python code

Making it look right

Custom formatters

The format() method

Strings are Unicode

Decoding bytes to text

Encoding text to bytes

Mutable byte strings

Regular expressions

Matching patterns

Matching a selection of characters

Escaping characters

Repeating patterns of characters

Grouping patterns together

Parsing information with regular expressions

Other features of the re module

Making regular expressions efficient

Filesystem paths

Serializing objects

Customizing pickles

Serializing objects using JSON

Case study

CSV format designs

CSV dictionary reader

CSV list reader

JSON serialization

Newline-delimited JSON

JSON validation

Recall

Exercises

Summary

The Iterator Pattern

Design patterns in brief

Iterators

The iterator protocol

Comprehensions

List comprehensions

Set and dictionary comprehensions

Generator expressions

Generator functions

Yield items from another iterable

Generator stacks

Case study

The Set Builder background

Multiple partitions

Testing

The essential k-NN algorithm

k-NN using the bisect module

k-NN using the heapq module

Conclusion

Recall

Exercises

Summary

Common Design Patterns

The Decorator pattern

A Decorator example

Decorators in Python

The Observer pattern

An Observer example

The Strategy pattern

A Strategy example

Strategy in Python

The Command pattern

A Command example

The State pattern

A State example

State versus Strategy

The Singleton pattern

Singleton implementation

Case study

Recall

Exercises

Summary

Advanced Design Patterns

The Adapter pattern

An Adapter example

The Façade pattern

A Façade example

The Flyweight pattern

A Flyweight example in Python

Multiple messages in a buffer

Memory optimization via Python's __slots__

The Abstract Factory pattern

An Abstract Factory example

Abstract Factories in Python

The Composite pattern

A Composite example

The Template pattern

A Template example

Case study

Recall

Exercises

Summary

Testing Object-Oriented Programs

Why test?

Test-driven development

Testing objectives

Testing patterns

Unit testing with unittest

Unit testing with pytest

pytest's setup and teardown functions

pytest fixtures for setup and teardown

More sophisticated fixtures

Skipping tests with pytest

Imitating objects using Mocks

Additional patching techniques

The sentinel object

How much testing is enough?

Testing and development

Case study

Unit testing the distance classes

Unit testing the Hyperparameter class

Recall

Exercises

Summary

Concurrency

Background on concurrent processing

Threads

The many problems with threads

Shared memory

The global interpreter lock

Thread overhead

Multiprocessing

Multiprocessing pools

Queues

The problems with multiprocessing

Futures

AsyncIO

AsyncIO in action

Reading an AsyncIO future

AsyncIO for networking

Design considerations

A log writing demonstration

AsyncIO clients

The dining philosophers benchmark

Case study

Recall

Exercises

Summary

Other Books You May Enjoy

Index

Landmarks

Cover

Index

Share Your Thoughts

Once you've read Python Object-Oriented Programming, Fourth Edition, we'd love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we're delivering excellent quality content.

1

Object-Oriented Design

In software development, design is often considered as the step that's done before programming. This isn't true; in reality, analysis, programming, and design tend to overlap, combine, and interweave. Throughout this book, we'll be covering a mixture of design and programming issues without trying to parse them into separate buckets. One of the advantages of a language like Python is the ability to express the design clearly.

In this chapter, we will talk a little about how we can move from a good idea toward writing software. We'll create some design artifacts – like diagrams – that can help clarify our thinking before we start writing code. We'll cover the following topics:

What object-oriented meansThe difference between object-oriented design and object-oriented programmingThe basic principles of object-oriented designBasic Unified Modeling Language (UML) and when it isn't evil

We will also introduce this book's object-oriented design case study, using the "4+1" architectural view model. We'll touch on a number of topics here:

An overview of a classic machine learning application, the famous Iris classification problemThe general processing context for this classifierSketching out two views of the class hierarchy that look like they'll be adequate to solve the problem

Introducing object-oriented

Everyone knows what an object is: a tangible thing that we can sense, feel, and manipulate. The earliest objects we interact with are typically baby toys. Wooden blocks, plastic shapes, and over-sized puzzle pieces are common first objects. Babies learn quickly that certain objects do certain things: bells ring, buttons are pressed, and levers are pulled.

The definition of an object in software development is not terribly different. Software objects may not be tangible things that you can pick up, sense, or feel, but they are models of something that can do certain things and have certain things done to them. Formally, an object is a collection of data and associated behaviors.

Considering what an object is, what does it mean to be object-oriented? In the dictionary, oriented means directed toward. Object-oriented programming means writing code directed toward modeling objects. This is one of many techniques used for describing the actions of complex systems. It is defined by describing a collection of interacting objects via their data and behavior.

If you've read any hype, you've probably come across the terms object-oriented analysis, object-oriented design, object-oriented analysis and design, and object-oriented programming. These are all related concepts under the general object-oriented umbrella.

In fact, analysis, design, and programming are all stages of software development. Calling them object-oriented simply specifies what kind of software development is being pursued.

Object-oriented analysis (OOA) is the process of looking at a problem, system, or task (that somebody wants to turn into a working software application) and identifying the objects and interactions between those objects. The analysis stage is all about what needs to be done.

The output of the analysis stage is a description of the system, often in the form of requirements. If we were to complete the analysis stage in one step, we would have turned a task, such as As a botanist, I need a website to help users classify plants so I can help with correct identification, into a set of required features. As an example, here are some requirements as to what a website visitor might need to do. Each item is an action bound to an object; we've written them with italics to highlight the actions, and bold to highlight the objects:

BrowsePrevious UploadsUpload new Known ExamplesTest for QualityBrowseProductsSeeRecommendations

In some ways, the term analysis is a misnomer. The baby we discussed earlier doesn't analyze the blocks and puzzle pieces. Instead, she explores her environment, manipulates shapes, and sees where they might fit. A better turn of phrase might be object-oriented exploration. In software development, the initial stages of analysis include interviewing customers, studying their processes, and eliminating possibilities.

Object-oriented design (OOD) is the process of converting such requirements into an implementation specification. The designer must name the objects, define the behaviors, and formally specify which objects can activate specific behaviors on other objects. The design stage is all about transforming what should be done into how it should be done.

The output of the design stage is an implementation specification. If we were to complete the design stage in a single step, we would have turned the requirements defined during object-oriented analysis into a set of classes and interfaces that could be implemented in (ideally) any object-oriented programming language.

Object-oriented programming (OOP) is the process of converting a design into a working program that does what the product owner originally requested.

Yeah, right! It would be lovely if the world met this ideal and we could follow these stages one by one, in perfect order, like all the old textbooks told us to. As usual, the real world is much murkier. No matter how hard we try to separate these stages, we'll always find things that need further analysis while we're designing. When we're programming, we find features that need clarification in the design.

Most 21st century development recognizes that this cascade (or waterfall) of stages doesn't work out well. What seems to be better is an iterative development model. In iterative development, a small part of the task is modeled, designed, and programmed, and then the product is reviewed and expanded to improve each feature and include new features in a series of short development cycles.

The rest of this book is about object-oriented programming, but in this chapter, we will cover the basic object-oriented principles in the context of design. This allows us to understand concepts without having to argue with software syntax or Python tracebacks.

Objects and classes

An object is a collection of data with associated behaviors. How do we differentiate between types of objects? Apples and oranges are both objects, but it is a common adage that they cannot be compared. Apples and oranges aren't modeled very often in computer programming, but let's pretend we're doing an inventory application for a fruit farm. To facilitate this example, we can assume that apples go in barrels and oranges go in baskets.

The problem domain we've uncovered so far has four kinds of objects: apples, oranges, baskets, and barrels. In object-oriented modeling, the term used for a kind of object is class. So, in technical terms, we now have four classes of objects.

It's important to understand the difference between an object and a class. Classes describe related objects. They are like blueprints for creating an object. You might have three oranges sitting on the table in front of you. Each orange is a distinct object, but all three have the attributes and behaviors associated with one class: the general class of oranges.

The relationship between the four classes of objects in our inventory system can be described using a Unified Modeling Language (invariably referred to as UML, because three-letter acronyms never go out of style) class diagram. Here is our first class diagram:

Figure 1.1: Class diagram

This diagram shows that instances of the Orange class (usually called "oranges") are somehow associated with a Basket and that instances of the Apple class ("apples") are also somehow associated with a Barrel. Association is the most basic way for instances of two classes to be related.

The syntax of a UML diagram is generally pretty obvious; you don't have to read a tutorial to (mostly) understand what is going on when you see one. UML is also fairly easy to draw, and quite intuitive. After all, many people, when describing classes and their relationships, will naturally draw boxes with lines between them. Having a standard based on these intuitive diagrams makes it easy for programmers to communicate with designers, managers, and each other.

Note that the UML diagram generally depicts the class definitions, but we're describing attributes of the objects. The diagram shows the class of Apple and the class of Barrel, telling us that a given apple is in a specific barrel. While we can use UML to depict individual objects, that's rarely necessary. Showing these classes tells us enough about the objects that are members of each class.

Some programmers disparage UML as a waste of time. Citing iterative development, they will argue that formal specifications done up in fancy UML diagrams are going to be redundant before they're implemented, and that maintaining these formal diagrams will only waste time and not benefit anyone.

Every programming team consisting of more than one person will occasionally have to sit down and hash out the details of the components being built. UML is extremely useful for ensuring quick, easy, and consistent communication. Even those organizations that scoff at formal class diagrams tend to use some informal version of UML in their design meetings or team discussions.

Furthermore, the most important person you will ever have to communicate with is your future self. We all think we can remember the design decisions we've made, but there will always be Why did I do that? moments hiding in our future. If we keep the scraps of papers we did our initial diagramming on when we started a design, we'll eventually find them to be a useful reference.

This chapter, however, is not meant to be a tutorial on UML. There are many of those available on the internet, as well as numerous books on the topic. UML covers far more than class and object diagrams; it also has a syntax for use cases, deployment, state changes, and activities. We'll be dealing with some common class diagram syntax in this discussion of object-oriented design. You can pick up the structure by example, and then you'll subconsciously choose the UML-inspired syntax in your own team or personal design notes.

Our initial diagram, while correct, does not remind us that apples go in barrels or how many barrels a single apple can go in. It only tells us that apples are somehow associated with barrels. The association between classes is often obvious and needs no further explanation, but we have the option to add further clarification as needed.

The beauty of UML is that most things are optional. We only need to specify as much information in a diagram as makes sense for the current situation. In a quick whiteboard session, we might just draw simple lines between boxes. In a formal document, we might go into more detail.

In the case of apples and barrels, we can be fairly confident that the association is many apples go in one barrel, but just to make sure nobody confuses it with one apple spoils one barrel, we can enhance the diagram, as shown here:

Figure 1.2: Class diagram with more detail

This diagram tells us that oranges go in baskets, with a little arrow showing what goes in what. It also tells us the number of that object that can be used in the association on both sides of the relationship. One Basket can hold many (represented by a *) Orange objects. Any one Orange can go in exactly one Basket. This number is referred to as the multiplicity of the object. You may also hear it described as the cardinality; it can help to think of cardinality as a specific number or range, and what we're using here, multiplicity, as a generalized "more-than-one instance".

We may sometimes forget which end of the relationship line is supposed to have which multiplicity number. The multiplicity nearest to a class is the number of objects of that class that can be associated with any one object at the other end of the association. For the apple goes in barrel association, reading from left to right, many instances of the Apple class (that is, many Apple objects) can go in any one Barrel. Reading from right to left, exactly one Barrel can be associated with any one Apple.

We've seen the basics of classes, and how they specify relationships among objects. Now, we need to talk about the attributes that define an object's state, and the behaviors of an object that may involve state change or interaction with other objects.

Specifying attributes and behaviors

We now have a grasp of some basic object-oriented terminology. Objects are instances of classes that can be associated with each other. A class instance is a specific object with its own set of data and behaviors; a specific orange on the table in front of us is said to be an instance of the general class of oranges.

The orange has a state, for example, ripe or raw; we implement the state of an object via the values of specific attributes. An orange also has behaviors. By themselves, oranges are generally passive. State changes are imposed on them. Let's dive into the meaning of those two words, state and behaviors.

Data describes object state

Let's start with data. Data represents the individual characteristics of a certain object; its current state. A class can define specific sets of characteristics that are part of all objects that are members of that class. Any specific object can have different data values for the given characteristics. For example, the three oranges on our table (if we haven't eaten any) could each weigh a different amount. The orange class could have a weight attribute to represent that datum. All instances of the orange class have a weight attribute, but each orange has a different value for this attribute. Attributes don't have to be unique, though; any two oranges may weigh the same amount.

Attributes are frequently referred to as members or properties. Some authors suggest that the terms have different meanings, usually that attributes are settable, while properties are read-only. A Python property can be defined as read-only, but the value will be based on attribute values that are – ultimately – writable, making the concept of read-only rather pointless; throughout this book, we'll see the two terms used interchangeably. In addition, as we'll discuss in Chapter 5, When to Use Object-Oriented Programming, the property keyword has a special meaning in Python for a particular kind of attribute.

In Python, we can also call an attribute an instance variable. This can help clarify the way attributes work. They are variables with unique values for each instance of a class. Python has other kinds of attributes, but we'll focus on the most common kind to get started.

In our fruit inventory application, the fruit farmer may want to know what orchard the orange came from, when it was picked, and how much it weighs. They might also want to keep track of where each Basket is stored. Apples might have a color attribute, and barrels might come in different sizes.

Some of these properties may also belong to multiple classes (we may want to know when apples are picked, too), but for this first example, let's just add a few different attributes to our class diagram:

Figure 1.3: Class diagram with attributes

Depending on how detailed our design needs to be, we can also specify the type for each attribute's value. In UML, attribute types are often generic names common to many programming languages, such as integer, floating-point number, string, byte, or Boolean. However, they can also represent generic collections such as lists, trees, or graphs, or most notably, other, non-generic, application-specific classes. This is one area where the design stage can overlap with the programming stage. The various primitives and built-in collections available in one programming language may be different from what is available in another.

Here's a version with (mostly) Python-specific type hints:

Figure 1.4: Class diagram with attributes and their types

Usually, we don't need to be overly concerned with data types at the design stage, as implementation-specific details are chosen during the programming stage. Generic names are normally sufficient for design; that's why we included date as a placeholder for a Python type like datetime.datetime. If our design calls for a list container type, Java programmers can choose to use a LinkedList or an ArrayList when implementing it, while Python programmers (that's us!) might specify List[Apple] as a type hint, and use the list type for the implementation.

In our fruit-farming example so far, our attributes are all basic primitives. However, there are some implicit attributes that we can make explicit – the associations. For a given orange, we have an attribute referring to the basket that holds that orange, the basket attribute, with a type hint of Basket.

Behaviors are actions

Now that we know how data defines the object's state, the last undefined term we need to look at is behaviors. Behaviors are actions that can occur on an object. The behaviors that can be performed on a specific class of object are expressed as the methods of the class. At the programming level, methods are like functions in structured programming, but they have access to the attributes – in particular, the instance variables with the data associated with this object. Like functions, methods can also accept parameters and return values.

A method's parameters are provided to it as a collection of objects that need to be passed into that method. The actual object instances that are passed into a method during a specific invocation are usually referred to as arguments. These objects are bound to parameter variables in the method body. They are used by the method to perform whatever behavior or task it is meant to do. Returned values are the results of that task. Internal state changes are another possible effect of evaluating a method.

We've stretched our comparing apples and oranges example into a basic (if far-fetched) inventory application. Let's stretch it a little further and see whether it breaks. One action that can be associated with oranges is the pick action. If you think about implementation, pick would need to do two things:

Place the orange in a basket by updating the Basket attribute of the orange.Add the orange to the Orange list on the given Basket.

So, pick needs to know what basket it is dealing with. We do this by giving the pick method a Basket parameter. Since our fruit farmer also sells juice, we can add a squeeze method to the Orange class. When called, the squeeze method might return the amount of juice retrieved, while also removing the Orange from the Basket it was in.

The class Basket can have a sell action. When a basket is sold, our inventory system might update some data on as-yet unspecified objects for accounting and profit calculations. Alternatively, our basket of oranges might go bad before we can sell them, so we add a discard method. Let's add these methods to our diagram:

Figure 1.5: Class diagram with attributes and methods

Adding attributes and methods to individual objects allows us to create a system of interacting objects. Each object in the system is a member of a certain class. These classes specify what types of data the object can hold and what methods can be invoked on it. The data in each object can be in a different state from other instances of the same class; each object may react to method calls differently because of the differences in state.

Object-oriented analysis and design is all about figuring out what those objects are and how they should interact. Each class has responsibilities and collaborations. The next section describes principles that can be used to make those interactions as simple and intuitive as possible.

Note that selling a basket is not unconditionally a feature of the Basket class. It may be that some other class (not shown) cares about the various Baskets and where they are. We often have boundaries around our design. We will also have questions about responsibilities allocated to various classes. The responsibility allocation problem doesn't always have a tidy technical solution, forcing us to draw (and redraw) our UML diagrams more than once to examine alternative designs.

Hiding details and creating the public interface

The key purpose of modeling an object in object-oriented design is to determine what the public interface of that object will be. The interface is the collection of attributes and methods that other objects can access to interact with that object. Other objects do not need, and in some languages are not allowed, to access the internal workings of the object.

A common real-world example is the television. Our interface to the television is the remote control. Each button on the remote control represents a method that can be called on the television object. When we, as the calling object, access these methods, we do not know or care if the television is getting its signal from a cable connection, a satellite dish, or an internet-enabled device. We don't care what electronic signals are being sent to adjust the volume, or whether the sound is destined for speakers or headphones. If we open the television to access its internal workings, for example, to split the output signal to both external speakers and a set of headphones, we may void the warranty.

This process of hiding the implementation of an object is suitably called information hiding. It is also sometimes referred to as encapsulation, but encapsulation is actually a more encompassing term. Encapsulated data is not necessarily hidden. Encapsulation is, literally, creating a capsule (or wrapper) on the attributes. The TV's external case encapsulates the state and behavior of the television. We have access to the external screen, the speakers, and the remote. We don't have direct access to the wiring of the amplifiers or receivers within the TV's case.

When we buy a component entertainment system, we change the level of encapsulation, exposing more of the interfaces between components. If we're an Internet of Things maker, we may decompose this even further, opening cases and breaking the information hiding attempted by the manufacturer.

The distinction between encapsulation and information hiding is largely irrelevant, especially at the design level. Many practical references use these terms interchangeably. As Python programmers, we don't actually have or need information hiding via completely private, inaccessible variables (we'll discuss the reasons for this in Chapter 2, Objects in Python), so the more encompassing definition for encapsulation is suitable.

The public interface, however, is very important. It needs to be carefully designed as it can be difficult to change when other classes depend on it. Changing an interface can break any client objects that depend on it. We can change the internals all we like, for example, to make it more efficient, or to access data over the network as well as locally, and the client objects will still be able to talk to it, unmodified, using the public interface. On the other hand, if we alter the interface by changing publicly accessed attribute names or the order or types of arguments that a method can accept, all client classes will also have to be modified. When designing public interfaces, keep it simple. Always design the interface of an object based on how easy it is to use, not how hard it is to code (this advice applies to user interfaces as well). For this reason, you'll sometimes see Python variables with a leading _ in their name as a warning that these aren't part of the public interface.

Remember, program objects may represent real objects, but that does not make them real objects. They are models. One of the greatest gifts of modeling is the ability to ignore irrelevant details. The model car one of the authors built as a child looked like a real 1956 Thunderbird on the outside, but it obviously didn't run. When they were too young to drive, these details were overly complex and irrelevant. The model is an abstraction of a real concept.

Abstraction is another object-oriented term related to encapsulation and information hiding. Abstraction means dealing with the level of detail that is most appropriate to a given task. It is the process of extracting a public interface from the inner details. A car's driver needs to interact with the steering, accelerator, and brakes. The workings of the motor, drive train, and brake subsystem don't matter to the driver. A mechanic, on the other hand, works at a different level of abstraction, tuning the engine and bleeding the brakes. Here's an example of two abstraction levels for a car:

Figure 1.6: Abstraction levels for a car

Now, we have several new terms that refer to similar concepts. Let's summarize all this jargon in a couple of sentences: abstraction is the process of encapsulating information with a separate public interface. Any private elements can be subject to information hiding. In UML diagrams, we might use a leading – instead of a leading + to suggest it's not part of a public interface.

The important lesson to take away from all these definitions is to make our models understandable to other objects that have to interact with them. This means paying careful attention to small details.

Ensure methods and properties have sensible names. When analyzing a system, objects typically represent nouns in the original problem, while methods are normally verbs. Attributes may show up as adjectives or more nouns. Name your classes, attributes, and methods accordingly.

When designing the interface, imagine you are the object; you want clear definitions of your responsibility and you have a very strong preference for privacy to meet those responsibilities. Don't let other objects have access to data about you unless you feel it is in your best interest for them to have it. Don't give them an interface to force you to perform a specific task unless you are certain it's your responsibility to do that.

Composition

So far, we have learned to design systems as a group of interacting objects, where each interaction involves viewing objects at an appropriate level of abstraction. But we don't yet know how to create these levels of abstraction. There are a variety of ways to do this; we'll discuss some advanced design patterns in Chapters 10, 11, and 12. But even most design patterns rely on two basic object-oriented principles known as composition and inheritance. Composition is simpler, so let's start with that.

Composition is the act of collecting several objects together to create a new one. Composition is usually a good choice when one object is part of another object. We've already seen a first hint of composition when talking about cars. A fossil-fueled car is composed of an engine, transmission, starter, headlights, and windshield, among numerous other parts. The engine, in turn, is composed of pistons, a crank shaft, and valves. In this example, composition is a good way to provide levels of abstraction. The Car object can provide the interface required by a driver, while also giving access to its component parts, which offers the deeper level of abstraction suitable for a mechanic. Those component parts can, of course, be further decomposed into details if the mechanic needs more information to diagnose a problem or tune the engine.

A car is a common introductory example of composition, but it's not overly useful when it comes to designing computer systems. Physical objects are easy to break into component objects. People have been doing this at least since the ancient Greeks originally postulated that atoms were the smallest units of matter (they, of course, didn't have access to particle accelerators). Because computer systems involve a lot of peculiar concepts, identifying the component objects does not happen as naturally as with real-world valves and pistons.

The objects in an object-oriented system occasionally represent physical objects such as people, books, or telephones. More often, however, they represent concepts. People have names, books have titles, and telephones are used to make calls. Calls, titles, accounts, names, appointments, and payments are not usually considered objects in the physical world, but they are all frequently-modeled components in computer systems.

Let's try modeling a more computer-oriented example to see composition in action. We'll be looking at the design of a computerized chess game. This was a very popular pastime in the 80s and 90s. People were predicting that computers would one day be able to defeat a human chess master. When this happened in 1997 (IBM's Deep Blue defeated world chess champion, Gary Kasparov), interest in the problem of chess waned. Nowadays, the descendants of Deep Blue always win.

A game of chess is played between two players, using a chess set featuring a board containing 64 positions in an 8×8 grid. The board can have two sets of 16 pieces that can be moved, in alternating turns by the two players in different ways. Each piece can take other pieces. The board will be required to draw itself on the computer screen after each turn.

I've identified some of the possible objects in the description using italics, and a few key methods using bold. This is a common first step in turning an object-oriented analysis into a design. At this point, to emphasize composition, we'll focus on the board, without worrying too much about the players or the different types of pieces.

Let's start at the highest level of abstraction possible. We have two players interacting with a Chess Set by taking turns making moves:

Figure 1.7: Object/instance diagram for a chess game

This doesn't quite look like our earlier class diagrams, which is a good thing since it isn't one! This is an object diagram, also called an instance diagram. It describes the system at a specific state in time, and is describing specific instances of objects, not the interaction between classes. Remember, both players are members of the same class, so the class diagram looks a little different:

Figure 1.8: Class diagram for a chess game

This diagram shows that exactly two players can interact with one chess set. This also indicates that any one player can be playing with only one Chess Set at a time.

However, we're discussing composition, not UML, so let's think about what the Chess Set is composed of. We don't care what the player is composed of at this time. We can assume that the player has a heart and brain, among other organs, but these are irrelevant to our model. Indeed, there is nothing stopping said player from being Deep Blue itself, which has neither a heart nor a brain.

The chess set, then, is composed of a board and 32 pieces. The board further comprises 64 positions. You could argue that these pieces are not part of the chess set, because you could replace the pieces of a chess set with a different set of pieces. While this is unlikely or impossible in a computerized version of chess, it introduces us to aggregation.

Aggregation is almost exactly like composition. The difference is that aggregate objects can exist independently. It would be impossible for a position to be associated with a different chess board, so we say the board is composed of positions. But the pieces, which might exist independently of the chess set, are said to be in an aggregate relationship with that set.

Another way to differentiate between aggregation and composition is to think about the lifespan of the object:

If the composite (outside) object controls when the related (inside) objects are created and destroyed, composition is most suitable. If the related object is created independently of the composite object, or can outlast that object, an aggregate relationship makes more sense.

Also, keep in mind that composition is aggregation; aggregation is simply a more general form of composition. Any composite relationship is also an aggregate relationship, but not vice versa.

Let's describe our current Chess Set composition and add some attributes to the objects to hold the composite relationships:

Figure 1.9: Class diagram for a chess game

The composition relationship is represented in UML as a solid diamond. The hollow diamond represents the aggregate relationship. You'll notice that the board and pieces are stored as part of the Chess Set in exactly the same way a reference to them is stored as an attribute on the chess set. This shows that, once again, in practice, the distinction between aggregation and composition is often irrelevant once you get past the design stage. When implemented, they behave in much the same way.

This distinction can help you differentiate between the two when your team is discussing how the different objects interact. You'll often need to distinguish between them when talking about how long related objects exist. In many cases, deleting a composite object (like the board) deletes all the locations. The aggregated objects, however, are not deleted automatically.

Inheritance

We discussed three types of relationships between objects: association, composition, and aggregation. However, we have not fully specified our chess set, and these tools don't seem to give us all the power we need. We discussed the possibility that a player might be a human or it might be a piece of software featuring artificial intelligence. It doesn't seem right to say that a player is associated with a human, or that the artificial intelligence implementation is part of the player object. What we really need is the ability to say that Deep Blue is a player, or that Gary Kasparov is a player.

The is a relationship is formed by inheritance. Inheritance is the most famous, well-known, and overused relationship in object-oriented programming. Inheritance is sort of like a family tree. Dusty Phillips is one of this book's authors.

His grandfather's last name was Phillips, and his father inherited that name. Dusty inherited it from him. In object-oriented programming, instead of inheriting features and behaviors from a person, one class can inherit attributes and methods from another class.

For example, there are 32 chess pieces in our chess set, but there are only six different types of pieces (pawns, rooks, bishops, knights, king, and queen), each of which behaves differently when it is moved. All of these classes of piece have properties, such as color and the chess set they are part of, but they also have unique shapes when drawn on the chess board, and make different moves. Let's see how the six types of pieces can inherit from a Piece class:

Figure 1.10: How chess pieces inherit from the Piece class

The hollow arrows indicate that the individual classes of pieces inherit from the Piece class. All the child classes automatically have a chess_set and color attribute inherited from the base class. Each piece provides a different shape property (to be drawn on the screen when rendering the board), and a different move method to move the piece to a new position on the board at each turn.

We actually know that all subclasses of the Piece class need to have a move method; otherwise, when the board tries to move the piece, it will get confused. It is possible that we would want to create a new version of the game of chess that has one additional piece (the wizard). Our current design will allow us to design this piece without giving it a move method. The board would then choke when it asked the piece to move itself.

We can fix this by creating a dummy move method on the Piece class. The subclasses can then override this method with a more specific implementation. The default implementation might, for example, pop up an error message that says That piece cannot be moved.

Overriding methods in subclasses allows very powerful object-oriented systems to be developed. For example, if we wanted to implement a Player class with artificial intelligence, we might provide a calculate_move method that takes a Board object and decides which piece to move where. A very basic class might randomly choose a piece and direction and move it accordingly. We could then override this method in a subclass with the Deep Blue implementation. The first class would be suitable for play against a raw beginner; the latter would challenge a grand master. The important thing is that other methods in the class, such as the ones that inform the board as to which move was chosen, need not be changed; this implementation can be shared between the two classes.

In the case of chess pieces, it doesn't really make sense to provide a default implementation of the move method. All we need to do is specify that the move method is required in any subclasses. This can be done by making Piece an abstract class with the move method declared as abstract. Abstract methods basically say this:

"We demand this method exist in any non-abstract subclass, but we are declining to specify an implementation in this class."

Indeed, it is possible to make an abstraction that does not implement any methods at all. Such a class would simply tell us what the class should do, but provides absolutely no advice on how to do it. In some languages, these purely abstract classes are called interfaces. It's possible to define a class with only abstract method placeholders in Python, but it's very rare.

Inheritance provides abstraction

Let's explore the longest word in object-oriented argot. Polymorphism is the ability to treat a class differently, depending on which subclass is implemented. We've already seen it in action with the pieces system we've described. If we took the design a bit further, we'd probably see that the Board object can accept a move from the player and call the move function on the piece. The board need not ever know what type of piece it is dealing with. All it has to do is call the move method, and the proper subclass will take care of moving it as a Knight or a Pawn.

Polymorphism is pretty cool, but it is a word that is rarely used in Python programming. Python goes an extra step past allowing a subclass of an object to be treated like a parent class. A board implemented in Python could take any object that has a move method, whether it is a bishop piece, a car, or a duck. When move is called, the Bishop will move diagonally on the board, the car will drive someplace, and the duck will swim or fly, depending on its mood.

This sort of polymorphism in Python is typically referred to as duck typing: if it walks like a duck or swims like a duck, we call it a duck. We don't care if it really is a duck (is a being a cornerstone of inheritance), only that it swims or walks. Geese and swans might easily be able to provide the duck-like behavior we are looking for. This allows future designers to create new types of birds without actually specifying a formal inheritance hierarchy for all possible kinds of aquatic birds. The chess examples, above, use formal inheritance to cover all possible pieces in the chess set. Duck typing also allows a programmer to extend a design, creating completely different drop-in behaviors the original designers never planned for. For example, future designers might be able to make a walking, swimming penguin that works with the same interface without ever suggesting that penguins have a common superclass with ducks.

Multiple inheritance

When we think of inheritance in our own family tree, we can see that we inherit features from more than just one parent. When strangers tell a proud mother that her son has his father's eyes, she will typically respond along the lines of, yes, but he got my nose.

Object-oriented design can also feature such multiple inheritance, which allows a subclass to inherit functionality from multiple parent classes. In practice, multiple inheritance can be a tricky business, and some programming languages (most famously, Java) strictly prohibit it. However, multiple inheritance can have its uses. Most often, it can be used to create objects that have two distinct sets of behaviors. For example, an object designed to connect to a scanner to make an image and send a fax of the scanned image might be created by inheriting from two separate scanner and faxer objects.