46,44 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
Explore various verticals in software engineering through high-end systems using Python
Key Features
- Master the tools and techniques used in software engineering
- Evaluates available database options and selects one for the final Central Office system-components
- Experience the iterations software go through and craft enterprise-grade systems
Book Description
Software Engineering is about more than just writing code—it includes a host of soft skills that apply to almost any development effort, no matter what the language, development methodology, or scope of the project. Being a senior developer all but requires awareness of how those skills, along with their expected technical counterparts, mesh together through a project's life cycle. This book walks you through that discovery by going over the entire life cycle of a multi-tier system and its related software projects. You'll see what happens before any development takes place, and what impact the decisions and designs made at each step have on the development process. The development of the entire project, over the course of several iterations based on real-world Agile iterations, will be executed, sometimes starting from nothing, in one of the fastest growing languages in the world—Python. Application of practices in Python will be laid out, along with a number of Python-specific capabilities that are often overlooked. Finally, the book will implement a high-performance computing solution, from first principles through complete foundation.
What you will learn
- Understand what happens over the course of a system's life (SDLC)
- Establish what to expect from the pre-development life cycle steps
- Find out how the development-specific phases of the SDLC affect development
- Uncover what a real-world development process might be like, in an Agile way
- Find out how to do more than just write the code
- Identify the existence of project-independent best practices and how to use them
- Find out how to design and implement a high-performance computing process
Who this book is for
Hands-On Software Engineering with Python is for you if you are a developer having basic understanding of programming and its paradigms and want to skill up as a senior programmer. It is assumed that you have basic Python knowledge.
Das E-Book können Sie in Legimi-Apps oder einer beliebigen App lesen, die das folgende Format unterstützen:
Seitenzahl: 879
Veröffentlichungsjahr: 2018
Ähnliche
Hands-On Software Engineering with Python
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
Commissioning Editor: Merint MathewAcquisition Editor: Sandeep MishraContent Development Editor: Anugraha ArunagiriTechnical Editor: Ashi SinghCopy Editor: Safis EditingProject Coordinator: Ulhas KambaliProofreader: Safis EditingIndexer: Tejal Daruwale SoniGraphics: Tania DuttaProduction Coordinator: Shantanu Zagade
First published: October 2018
Production reference: 1241018
Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK.
ISBN 978-1-78862-201-1
www.packt.com
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors
About the author
Brian Allbee has been writing programs since the mid-1970s, and started a career in software just as the World Wide Web was starting to take off. He has worked in areas as varied as organization membership management, content/asset management, and process and workflow automation in industries as varied as advertising, consumer health advisement, technical publication, and cloud-computing automation. He has focused exclusively on Python solutions for the best part of a decade.
About the reviewers
Chad Greer's focus lies in helping others find excellence. He works to replace typical "systems thinking" with talent-based approaches, breaking the mold of traditional business processes. Embracing the principles of agility, he works to respond to changing market and societal needs in order to ensure that the best solutions are created and delivered. He has worked in many different industries, including real estate, accounting, construction, local government, law enforcement, martial arts, music, healthcare, and several others. He draws on his breadth of experience to help others around him develop and prosper.
Nimesh Kiran Verma has a dual degree in maths and computing from IIT Delhi and has worked with companies such as LinkedIn, Paytm, and ICICI for about 5 years in software development and data science. He co-founded a micro-lending company, Upwards Fintech, and presently serves as its CTO. He loves coding and has mastered Python and its popular frameworks, Django and Flask. He extensively leverages Amazon Web Services, design patterns, and SQL and NoSQL databases to build reliable, scalable, and low latency architectures.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and apply today. We have worked with thousands of developers and tech professionals, just like you, to help them share their insight with the global tech community. You can make a general application, apply for a specific hot topic that we are recruiting an author for, or submit your own idea.
Table of Contents
Title Page
Copyright and Credits
Hands-On Software Engineering with Python
Packt Upsell
Why subscribe?
Packt.com
Contributors
About the author
About the reviewers
Packt is searching for authors like you
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
Programming versus Software Engineering
The bigger picture
Asking questions
Summary
The Software Development Life Cycle
Pre-development phases of the SDLC
Initial concept/vision
Concept development
Project management planning
Development – specific phases of the SDLC
Requirements analysis and definition
System architecture and design
Development and quality assurance
System integration, testing, and acceptance
Post-development phases of the SDLC
Summary
System Modeling
Architecture, both logical and physical
Logical architecture
Physical architecture
Use cases (business processes and rules)
Data structure and flow
Interprocess communication
System scope and scale
Summary
Methodologies, Paradigms, and Practices
Process methodologies
Waterfall
Agile (in general)
Scrum
Scrum and the phases of the SDLC model
Kanban
Kanban and the phases of the SDLC model
Other Agile methodologies
Extreme programming
Feature-driven development
Test-driven design
Development paradigms
Object-oriented programming
Functional programming
Development practices
Continuous integration
Continuous delivery or deployment
Summary
The hms_sys System Project
Goals for the system
What's known/designed before development starts
What the iteration chapters will look like
Iteration goals and stories
Writing and testing the code
Post-development considerations and impact
Summary
Development Tools and Best Practices
Development tools
Integrated Development Environment (IDE) options
IDLE
Geany
Eclipse variations + PyDev
Others
Source Code Management
Typical SCM activities
Git
Subversion
Basic workflows for Git and SVN compared
Other SCM options
Best practices
Standards for code
PEP-8
Internal standards
Code organization in modules
Structure and standards for classes
Function and method annotation (hinting)
Process standards
Unit testing
Repeatable build processes
Integrating unit tests and build processes
Defining package structures for Python code
Packages in a project's context
Using Python virtual environments
Summary
Setting Up Projects and Processes
Iteration goals
Assembly of stories and tasks
Setting Up SCM
Stubbing out component projects
Component project analysis
Component project setup
Packaging and build process
Python virtual environments
Basic unit testing
Identifying missing test case classes
Identifying missing test methods
Creating reusable module code coverage tests
The property and method testing decorators
Creating unit test template files
Integrating tests with the build process
Summary
Creating Business Objects
Iteration goals
Assembly of stories and tasks
A quick review of classes
Implementing the basic business objects in hms_sys
Address
BaseArtisan
OO principles – composition over inheritance
Implementing BaseArtisan's properties
Implementing BaseArtisan's methods
BaseCustomer
BaseOrder
BaseProduct
Dealing with duplicated code – HasProducts
Summary
Testing Business Objects
Starting the unit testing process
Unit testing the Address class
Unit testing HasProducts
Unit testing BaseProduct
Unit testing BaseOrder
Unit-testing BaseCustomer
Unit testing BaseArtisan
Unit testing patterns established so far
Distribution and installation considerations
Quality assurance and acceptance
Operation/use, maintenance, and decommissioning considerations
Summary
Thinking About Business Object Data Persistence
Iterations are (somewhat) flexible
Data storage options
Relational databases
Advantages and drawbacks
MySQL/MariaDB
MS-SQL
PostgresQL
NoSQL databases
Advantages and drawbacks
MongoDB
Other NoSQL options
Other data storage options
Selecting a data storage option
Polymorphism (and programming to an interface)
Data access design strategies
Data access decisions
Why start from scratch?
Summary
Data Persistence and BaseDataObject
The BaseDataObject ABC
Unit testing BaseDataObject
Summary
Persisting Object Data to Files
Setting up the hms_artisan project
Creating a local file system data store
Implementing JSONFileDataObject
The concrete business objects of hms_artisan
Dealing with is_dirty and properties
hms_artisan.Artisan
hms_artisan.Product
hms_artisan.Order
Summary
Persisting Data to a Database
The Artisan Gateway and Central Office application objects
Picking out a backend datastore engine
The data access strategy for the Central Office projects
Supporting objects for data persistence
RDBMS implementations
The concrete business objects of the Central Office projects
hms_core.co_objects.Artisan
hms_core.co_objects.Product
Other hms_core.co_objects classes
Accounting for the other CRUD operations
Summary
Testing Data Persistence
Writing the unit tests
Testing hms_artisan.data_storage
Testing hms_artisan.artisan_objects
Testing the new hms_core Classes
Unit testing hms_core.data_storage.py
Unit testing hms_core.co_objects.py
Unit tests and trust
Building/distribution, demonstration, and acceptance
Operations/use, maintenance, and decommissioning considerations
Summary
Anatomy of a Service
What is a service?
Service structure
Configuration
Windows-style .ini files
JSON files
YAML files
Logging service activities
Handling requests and generating responses
Filesystem – based
HTTP- or web-based
Message- queue-based
Other request types
Request and response formats
A generic service design
The BaseDaemon ABC
The BaseRequestHandler and BaseResponseFormatter ABCs
Integrating a service with the OS
Running a service using systemctl (Linux)
Running a service using NSSM (Windows)
macOS, launchd, and launchctl
Managing services on other systems
Summary
The Artisan Gateway Service
Overview and goal
Iteration stories
Messages
Deciding on a message-transmission mechanism
Message-queue implementation with RabbitMQ
Handling messages
Queues and related Artisan properties
Requirements for a web-service-based daemon
Traffic to and from the service
Impacts on testing and deployment
Summary
Handling Service Transactions
Remaining stories
A bit of reorganization
Preparation for object transactions
Product object transactions
Artisan – creating a product
Central Office – approving/listing a product
Central Office – altering product data
Artisan – updating product data
Artisan – deleting a product
Artisan object transactions
Central Office – creating an artisan
Central Office – updating artisan data
Central Office – deleting an artisan
Artisan – updating Artisan data
Order object transactions
Customer – relaying order items to artisans
Customer – canceling an order
Artisan – fulfilling an item in an order
When do messages get sent?
Summary
Testing and Deploying Services
The challenges of testing services
The overall testing strategy
Unit testing variations of note
Testing Artisan transactions
Demonstrating the service
Packaging and deploying the service
Common considerations across all operating systems
Linux (systemd) execution
Windows (NSSM) execution
Where hms_sys development could go from here
Code review, refactoring, and cleanup
Developing a UI
Order fulfilment and shipping APIs
Summary
Multiprocessing and HPC in Python
Common factors to consider
A simple but expensive algorithm
Some testing setup
Local parallel processing
Threads
Parallelizing across multiple machines
Common functionality
The Worker nodes
The Orchestrator
The Dispatcher
Integrating Python with large-scale, cluster computing frameworks
Python, Hadoop, and Spark
Summary 
Other Books You May Enjoy
Leave a review - let other readers know what you think
Preface
Ultimately, the purpose of this book is to illustrate pragmatic software engineering principles and how they can be applied to Python development. To that end, most of this book is dedicated to exploring and implementing what I'd consider to be a realistically scoped, but probably unrealistic project: a distributed product-management and order-fulfillment system. In many cases, the functionality is developed from scratch, and from first principles—the fundamental concepts and assumptions that lie at the foundation of the system. In a real-world scenario, the odds are good that ready-made solutions would be available to deal with many of the implementation details, but exposing the underlying theories and requirements is, I think, essential for understanding why things work the way they do. That, I believe, is an essential part of the difference between programming and software engineering, no matter what languages are in play.
Python is a rare beast in many respects—it's a dynamic language that is nevertheless strongly typed. It's an object-oriented language too. These, taken together, make for an amazingly flexible and sometimes surprisingly powerful language. Though it can be taken as my opinion, I strongly believe that you'd be hard-pressed to find another language that is as generally capable as Python that is also as easy to write and maintain code in. It doesn't surprise me in the least that Python has racked up the kinds of success stories that are listed on the language's official site (https://www.python.org/about/success/). It also doesn't surprise me that Python is one of the core supported languages for at least two of the big name public cloud providers—Amazon and Google. Even so, it's often still thought of as only a scripting language, and I sincerely hope that this book can also show that view to be wrong.
Who this book is for
This book is aimed at developers with some Python experience looking to expand their repertoire from "just writing code" to a more "software engineering" focus. Knowledge of Python basics—functions, modules, and packages, and their relationship to files in a project's structure, as well as how to import functionality from other packages—is assumed.
What this book covers
Chapter 1, Programming versus Software Engineering, discusses the differences between programming (merely writing code), and software engineering—the discipline, mindset, and ramifications of them.
Chapter 2, The Software Development Life Cycle, examines a detailed software development life cycle, with particular attention to the inputs, needs, and outcomes that relate to software engineering.
Chapter 3, System Modeling, explores different ways of modeling and diagramming functional, data-flow, and interprocess-communication aspects of systems and their components, and what information those provide with respect to software engineering.
Chapter 4, Methodologies, Paradigms, and Practices, delves into current process methodologies, including a few Agile process variants, looking at the advantages and drawbacks to each, before reviewing object-oriented programming (OOP) and functional programming paradigms.
Chapter 5, Thehms_sys System Project, introduces the concepts behind the example project used through the book to exercise software engineering design and development mindsets.
Chapter 6, Development Tools and Best Practices, investigates some of the more common (or at least readily available) development tools—both for writing code and for managing it in ways that reduce ongoing development efforts and risks.
Chapter 7, Setting up Projects and Processes, walks through an example structure that could be used for any Python project or system, and the thought processes behind establishing a common starting-point that is compatible with source control management, automated testing, and repeatable build and deployment processes.
Chapter 8, Creating the Business Objects, starts the first iteration of the hms_sys project, defining core library business-object data structures and capabilities.
Chapter 9, Testing the Business Objects, closes the first iteration of the hms_sys project after designing, defining, and executing repeatable automated testing of the business object code defined during the iteration.
Chapter 10, Thinking about Business Object Data Persistence, examines the common need for data persistence in applications, some of the more common mechanisms, and criteria for selecting a "best match" data-storage solution for a variety of implementation requirements.
Chapter 11, Data Persistence and BaseDataObject, starts the second iteration of the hms_sys project with the design and implementation of a common, abstract data-access strategy that can be re-used across any of the project's components.
Chapter 12, Persisting Object Data to Files, continues the second iteration's efforts with a concrete implementation of the abstract Data Access Layer (DAL), which persists business-object data into local files.
Chapter 13, Persisting Data to a Database, implements a concrete DAL that stores and retrieves data from a commonly used NoSQL database—MongoDB—and compares that approach with the requirements of an equivalent SQL-based DAL.
Chapter 14, Testing Data Persistence, concludes the second iteration of the hms_sys project by implementing automated tests against the varied implementations of both DAL strategies built during the iteration.
Chapter 15, Anatomy of a Service, analyzes the common functional requirements for free-standing services, and works through the construction of abstract service/daemon classes, which are reusable for creating a variety of concrete service implementations.
Chapter 16, The Artisan Gateway Service, starts the third iteration of the hms_sys project with an analysis of the communication needs of the system components, several options for implementing those communications, securing them, and finally working them into the concrete implementation of the core service for the project.
Chapter 17, Handling Service Transactions, considers all of the necessary business-object communications between hms_sys components, extracts some common functionality for all of them, and walks through the processes required to implement them.
Chapter 18, Testing and Deploying Services, wraps up the hms_sys development in the book, and investigates and resolves some common automated-testing concerns for service/daemon applications.
Chapter 19, Multi-Processing and HPC in Python, walks through the theory and basic practices involved in writing Python code that can scale to multiple processors on a single machine, or to multiple machines in a clustered-computing environment, and provides starting-point code-structure variations for executing Python code on common high-performance computing systems.
To get the most out of this book
You should know, specifically, about the following:
How to download and install Python (3.6.x was used while writing this book, but the code here is expected to work in 3.7.x with little or no modification)
How to write Python functions
How to write basic Python classes
How to install Python modules with pip, and how to import modules into your code
Download the example code files
You can download the example code files for this book from your account at www.packt.com. If you purchased this book elsewhere, you can visit www.packt.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register at
www.packt.com
Select the
SUPPORT
tab
Click on
Code Downloads & Errata
Enter the name of the book in the
Search
box and follow the onscreen instructions
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR/7-Zip for Windows
Zipeg/iZip/UnRarX for Mac
7-Zip/PeaZip for Linux
The code bundle for the book is also hosted on GitHub athttps://github.com/PacktPublishing/Hands-On-Software-Engineering-with-Python. We also have other code bundles from our rich catalog of books and videos available athttps://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://www.packtpub.com/sites/default/files/downloads/9781788622011_ColorImages.pdf.
Get in touch
Feedback from our readers is always welcome.
General feedback: Email [email protected] and mention the book title in the subject of your message. If you have questions about any aspect of this book, please email us at [email protected].
Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packt.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details.
Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material.
If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you!
For more information about Packt, please visit packtpub.com.
Programming versus Software Engineering
Development shopsoften have specific levels, grades, or ranks that their developers fall into, indicating the levels of experience, expertise, and industry wisdom expected of staff at each level. These may vary (perhaps wildly) from location to location, but a typical structure looks something like the following:
Junior developers:
A junior developer is typically someone that doesn't have much programming experience
. They
probably
know the basics of writing code, but they are not expected to know much beyond that.
Developers:
Mid-level developers (referred to by whatever formal title might apply) usually have enough experience that they can be relied on to write reasonably solid code, with little to no supervision. They probably have enough experience to determine implementation details and strategies, and they will often have some understanding of how different chunks of code can (and do) interact with each other, and what approaches will minimize difficulties in those interactions.
Senior developers:
Senior developers have enough experience - even if it's focused on a set of specific products/projects - to firmly grasp all of the technical
skills involved in typical development efforts. At this point in their careers, they will almost always have a solid handle on a lot of the
non-technical
(or semi-technical) skills that are involved, as well—especially policies and procedures, and strategies and tactics that encourage or enforce
business values
such as stability and the predictability of development efforts. They may not be
experts
in those areas, but they will know when to call out risks,
and they will often have several options to suggest for mitigating those risks.
The dividing line between programming and software engineering falls somewhere within the differences between developers and senior developers,as far as technical capabilities and expertise are concerned.At a junior level, and sometimes at a developer level, efforts are often centered around nothing more than writing code to meet whatever requirements apply,and conforming to whatever standards are in play. Software engineering, at a senior developer level, has a big-picture view of the same end results. The bigger picture involves awareness of, and attention paid to, the following things:
Standards, both technical/developmental and otherwise, including best practices
The goals that code is written to accomplish, including the business values that are attached to them
The shape and scope of the entire system that the code is a part of
The bigger picture
So, what does this bigger picture look like? There are three easily-identifiable areas of focus, with a fourth (call it user interaction) that either weaves through the other three or is broken down into its own groups.
Software engineering must pay heed to standards, especially non-technical (business) ones, and also best practices. These may or may not be followed but, since they are standards or best practices for a reason, not following them is something that should always be a conscious (and defensible) decision.It's not unusual for business-process standards and practices to span multiple software components, which can make them difficult to track if a certain degree of discipline and planning isn't factored into the development processto make them more visible.On the purely development-related side, standards and best practices can drastically impact the creation and upkeep of code, its ongoing usefulness, and even just the ability to find a given chunk of code, when necessary.
It's rare for code to be written simply for the sake of writing code. There's almost always some other value associated with it, especially if there's business valueor actual revenueassociated with a product that the code is a part of.In those cases, understandably, the people that are paying for the developmental effort will be very interested in ensuring that everything works as expected (code-quality) and can be deployed when expected (process-predictability).
Code-quality concerns will be addressed during the development of thehms_sysprojecta few chapters from now, and process-predictabilityis mostly impacted by the developmental methodologies discussed in Chapter 5, The hms_sys System-Project.
The remaining policy-and-procedure related concerns aregenerallymanaged by setting up and following various standards,processes,and best practices during the startup of a project (or perhaps a development team). Those items - things such as setting up source control, having standard coding conventions, and planning for repeatable, automated testing - will be examined in some detail during the set upchapter for thehms_sysproject. Ideally, oncethese kinds of developmental process are in place, the ongoing activities that keep them running and reliable will just become habits, a part of the day-to-day process, almost fading into the background.
Finally, with more of a focus on the code side, software engineering must, by necessity, pay heed to entire systems,keeping a universal view of the system in mind. Software is composed of a lot of elements that might be classified as atomic; they are indivisible units in and of themselves, under normal circumstances. Just like their real-world counterparts, when they start to interact, things get interesting, and hopefully useful.Unfortunately, that's also when unexpected (or even dangerous) behaviors—bugs—usually start to appear.
This awareness is, perhaps, one of the more difficult items to cultivate. It relies on knowledge that may not be obvious, documented, or readily available.In large or complex systems, it may not even be obvious where to start looking, or what kinds of question to ask to try to find the information needed to acquire that knowledge.
Summary
There's more to software engineering than just writing code. Experience; attention to detail; and asking questions about how the code functions, interacts with the rest of a system, and so on; are important aspects of evolving from a programming to a software engineering mindset. The time required to acquire experience can be shortened, perhaps significantly, by simply asking the right questions.
There are also factors completely outside the realm of creating and managing code that require examination and questioning. They mainly focus on what can, or should, be expected from the pre-development planning around a developmental effort, and that starts with understanding a typical software development life cycle.
The Software Development Life Cycle
All software development, Python or otherwise, above a certain level of complexity follows repeatable patterns, or has a life cycle. A Software(or System)Development Life-Cycle (SDLC)might be used as its own distinct development methodology, providing a set of tasks and activities that apply to the development process. That is, even if there is noformal process wrapped around an SDLC,any or all of the activities that go on through one may still take place, and any or all of the artifacts that come out of them may be available during the development of a project.
From the perspective of the actual development, not all of the artifacts resulting from an SDLC, formal or otherwise, may be significantly useful, either, particularly those coming out of thefirst fewphases of the life cycle's process.Even so, the more knowledge that is available during the development process, the less likely it is that development efforts will go in directions that run contrary to the intentions of the system on a longer-term basis.
In order to fully explore what an SDLC might provide, we'll use one of the more detailed ones that can be found on the internet. It breaks the life cycle down into ten phases, which would be executed in the following order, barring process alterations from a development methodology:
Initial concept/vision
Concept development
Project management planning
Requirements analysis and definition
System architecture and design
Development (writing code) and quality assurance
System integration, testing, and acceptance
Implementation/installation/distribution
Operations/use and maintenance
Decommissioning
The first three phases may all occur before any code is written, defining the high-level concepts and goals, and planning for how to accomplish those goals. The last three generally happen after code is complete, though as new features are thought of, or as bugs surface, code development may restart to address those items. The balance, phases 4 through 7, are loosely classifiable as during development, though, except for the actual writing of code in phase 6, that classification may depend on what development processes or methodologies are in play, something that is likely decided during phase 3 if it isn't already determined by external policies or forces.
Pre-development phases of the SDLC
Before the first line of code is written, there is the potential for a fair amount of thought and work going into a project. Not all of the work is going to be visible by the time development starts, and, realistically, not all of what could be produced pre-development will be, in many cases. Even those artifacts that are created may not have any formal structure or documentation around them, or may not be as complete or detailed as might be desired. Despite all of that, knowing what might be available that is of use or interest during development can at least help answer questions that can arise during the actual writing-of-code portion of a system/project.
Initial concept/vision
The very first thing that happens in a project's or system's life is its conception. Behind the scenes, that usually involves the recognition of some unfulfilled need, or something that isn't working the way it should, though other variations might occur as well. As part of that realization, there will frequently be a collection of capabilities that the conceived system will provide, benefits or functionality that will drive the system's development, and determine when that development is complete. At this initial, very high-level overview, there may not be much in the way of detail—we need a better way to manage inventory, maybe for the entire vision, for example—but it's possible that more detail will enter the picture, too.
The concept and the benefits might come from anyone with a stake in the system: business staff who are looking for a better way of doing things, developers who perhaps recognize that an existing system isn't as effective as it could be, or maybe that it's difficult to maintain. System administrators might have concerns about how easily managed an in-place system is and want a newer, better approach taken, or the initial vision might be for something completely new, at least in the context of the business setting—we need a way to keep track of fuel efficiency across our delivery truck fleet, maybe. What about if our customers could order our products online?
Hopefully, if off-the-shelf solutions or products are available that meet parts of these needs, those options will have been investigated in some detail—maybe even to the point where the vision owner would be able to point to some feature set(s) of those products and say, "We want something like this." Having examples of functionality that's close to what's actually wanted can be a significant time-saver during pre-development design and development alike, and it's almost always worth asking if there are examples of what's wanted as the design and development processes move along. If that sort of investigation was undertaken and no options were found that were even close, that, too, has useful information embedded in it—what was missing? What did product X do that wasn't meeting the needs in the concept? If no investigation was undertaken, or if nothing came out of an investigation, it's quite possible that the initial concept would be no more than a sentence or two. That's alright, though, since more detail will be extracted later on as the concept development gets underway.
In more formal processes,other analyses may also take place, looking for things such as the following:
Specific user needs
: What users must be able to do within the system, and probably what they should be able to do. There may also be a collection of nice-to-have features
—
things that users would like to be able to do, but that are not a functional necessity.
Specific functional needs
: What problems the system needs to solve, or at least mitigate in a significant fashion.
Risks
: Usually business-process-related risks, but those may also serve to guide design and development in later phases.
Costs
: Both in money and resources. Odds are that this information won't yield much use from a development process perspective, but it's not impossible for an occasional significant nugget of information to come out of this either.
Operational feasibility
: Examining how well the conceptual system addresses the needs it's been thought up to address. Like with cost analysis, the odds are good that there won't be much that comes out of this that's directly useful for development purposes, but it might identify operational or design areas where there is doubt about feasibility, and those doubts, in turn, may well shape design and/or implementation by the time the system is in development.
At best, then, given either a formal process, or sufficient attention to detail in an informal one, the initial concept might produce information or documentation about the following:
Benefits or functionality expected from the system (usually at a high level, at least to start with):
A collection of specific, high-level functional needs
A collection of specific user needs
Specific features or functionality that were not provided by an off-the-shelf system (thus justifying custom development effort)
Specific risks to mitigate against
Specific functional or feasibility concerns to address
All of these have at least some value once development is underway and will hopefully make their way into design or requirements, and from there into development.
Concept development
Concept development is concerned mostly with fleshing out some of the high-level details that come out of the initial concept, providing details and direction for efforts later in the life cycle. One of the more significant aspects of this step is the generation of various System Modeling artifacts—and there's enough involved in those efforts that they'll be covered in a separate chapter. The balance of the development-related information that comes out of this phase is probably focused more on marrying business processes and system functionality, and providing some detail around system goals. There is also room here for a definition of at least a basic user experience and/or user interface, especially as they connect to the process/functionality.
Defining the business processes embedded in a system includes identifying the business objects that the system keeps track of, the actions that can be taken with respect to those objects, and the outcomes of those actions, at a minimum. Applying of the sort of questioning described earlier in Chapter 1,Programming versus Software Engineering, can yield a fair bit of that information, if more detail is needed.
By way of example, consider a system whose concept begins with the knowledge that they need a way to keep track of fuel efficiency across their delivery truck fleet. Working out the business objects and activities from there could answer some very basic questions, such as the following:
What is the system keeping track of?
:
The individual trucks in the fleet, the mileage on the odometers of those trucks at irregular intervals, and the refueling of those trucks, at a minimum.
What does a refueling look like?
: A fuel quantity and the odometer reading at the time of refueling, to start with. Those two data points would allow for the calculation of fuel efficiency, which is calculated in whatever units each uses (gallons or liters for fuel, miles or kilometers for the odometer). Fuel efficiency becomes a calculation of any given refueling for any given truck, and the current odometer reading for any given truck can be retrieved from the odometer reading at its last refueling.
How many refuelings should be kept for any given truck?
:
If one of the goals of the system is to detect when a truck's fuel efficiency has dropped, in order to flag it for maintenance, perhaps, or to trigger a review of the delivery scheduling associated with it, then there is an obvious need to keep track of more than one such refueling—maybe
all of them
.
Who will be using the system, how, and where?
:
There would need to be at least two types of physical access point: one from mobile devices (when fueling a truck), and one from in-office computers (for reporting purposes, if nothing else). That set of use cases tells us that we're looking at either a web application, or some sort of dedicated phone and computer application set, with access to some common data stores, possibly through a service layer.
There may be other questions that could be asked, but these four alone probably give enough information to make the most of major concept design decisions, though the latter may require a bit more exploration before they can be finalized. Similar questioning, asking things such as What can (a specific type of user) do with the system until there aren't any more users and activities, can also yield more specific system goals:
Various users can log refuelings, providing the current odometer reading, and the quantity of fuel involved:
Delivery drivers (at local fuel stations)
Fleet maintenance staff (at the main office, where there is a company fuel station)
Fleet maintenance staff will be alerted when a truck's calculated fuel efficiency drops to lower than 90% of its average, so that the truck can be scheduled for an examination
Office staff will also be alerted when a truck's calculated fuel efficiency drops to lower than 90% of its average, so that the truck's delivery rounds can be examined
The question of how and where users will interact with the system may well spark some discussion and design decisions around user experience and interface design as well. In this case, perhaps after discussion about whether the system is a web application or dedicated phone and desktop application, the decision is made to make it a web application and to use the Clarity Design System for the UI, because the primary stakeholder in the system's vision likes the way it handles on-screen cards:
Project management planning
This phase of the life cycle is where all of the conceptual items come together, hopefully in a form or fashion that's ready for the actual creation of code to start. If there is a formal PMP document as a result, its outline might look something like this:
Business purpose
Objectives
Goals
What's included
What's excluded
Key assumptions
Project organization:
Roles and responsibilities
Stakeholders
Communication
Risks, issues, and dependencies
Preliminary schedule of deliverables
Change management
Risk and issue management
Developers won't need all of these items, but knowing where to look for various bits and pieces of the information they will need (or, in some cases, who to contact for information) is advantageous, so:
TheBusiness purpose,Objectives, andGoalssections should, ideally, collect all of the original vision information(from the initial concept/vision phase) with whatever details have been added or changes made after the concept design was complete.These will, in all probability, include the starting points for theRequirements analysis and definitionefforts that go on during the development-specific phases of the life cycle. In addition, theWhat's included,What's excluded, andKey assumptionssections, between them, should expose what the actual scope of development looks like, as well as providing high-level design decisions and any relevant high-level system modeling information.Risks, issues, and dependenciesmay provide specific items of concern or other interests that will help shape the development efforts. Finally,Change managementwill set expectations (at a high level, at least) for what processes are expected or planned for as changes to the system are made.
People in a position to answer questions or make decisions about the system's implementation that fall outside the scope of pure development will probably be listed in the Roles and responsibilitiesand/orStakeholderssections, though there may be specific established processes for raising those questions in theCommunicationsection.
Even without formal documentation around project management expectations, much of the information noted previously should still be made known to development staff—the less time spent having to track down who can answer a question, the more time can be devoted to actually writing code, after all.
Development – specific phases of the SDLC
Since the advent of Agile methodologies, and the widespread adoption of many of them, the specific shapes of the development-specific phases of an SDLC can vary substantially. Different methodologies make different decisions about what to prioritize or emphasize, and those differences can, in turn, yield significantly different processes and artifactsto accomplish the goals of formal SDLC phases that focus directly on developer needs and activities. Whole books have been written about several of the Agile processes, so a complete discussion of them is well beyond the scope of this book, but all of them address the following activities.
Requirements analysis and definition
Requirements analysis and definition are concerned with discovering and detailing the specific requirements of a system—what the system needs to allow users to do with it. Users obviously includes end users, ranging from office workers using the system to conduct day-to-day business, to external end users such as customers. Less obviously, users should also include system administrators, staff who receive data from the system through some reporting processes, and perhaps any number of other people who interact with the system in any fashion, or who are acted upon by it—including the developers themselves.
System architecture and design
If requirements analysis and definition are about what a system provides, system architecture and design are primarily about how those capabilities work. The differences in how various development methodologies deal with architecture and design is less about that how and more about when they are defined. Essentially, given a set of requirements (the intentions behind the system, or the why), the implementation details (the how) will almost certainly be determined more by those requirements and the specifics of how best to implement them in the programming language than by when they are identified, consolidated, or formalized.
