E-Book
77,99 €

Software Project Estimation E-Book

Alain Abran

0,0

77,99 €

Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.
Mehr erfahren.

Herausgeber: John Wiley & Sons
Kategorie: Wissenschaft und neue Technologien
Sprache: Englisch

Beschreibung

This book introduces theoretical concepts to explain the fundamentals of the design and evaluation of software estimation models. It provides software professionals with vital information on the best software management software out there. * End-of-chapter exercises * Over 100 figures illustrating the concepts presented throughout the book * Examples incorporated with industry data

Details

Sie lesen das E-Book in den Legimi-Apps auf:

Android

iOS

von Legimi
zertifizierten E-Readern

Seitenzahl: 391

Veröffentlichungsjahr: 2015

Bewertungen

0,0

Rezensionen(0 Rezensionen)

Leseprobe

Cover

Title Page

Foreword

Summary

Overview

Structure and organization

Acknowledgments

About the Author

Part I: Understanding the Estimation Process

Chapter 1: The Estimation Process: Phases and Roles

1.1 Introduction

1.2 Generic Approaches in Estimation Models: Judgment or Engineering?

1.3 Overview of Software Project Estimation and Current Practices

1.4 Levels of Uncertainty in an Estimation Process

1.5 Productivity Models

1.6 The Estimation Process

1.7 Budgeting and Estimating: Roles and Responsibilities

1.8 Pricing Strategies

1.9 Summary – Estimating Process, Roles, and Responsibilities

Exercises

Term Assignments

Chapter 2: Engineering and Economics Concepts for Understanding Software Process Performance

2.1 Introduction: The Production (Development) Process

2.2 The Engineering (and Management) Perspective on a Production Process

2.3 Simple Quantitative Process Models

2.4 Quantitative Models and Economics Concepts

2.5 Software Engineering Datasets and Their Distribution

2.6 Productivity Models: Explicit and Implicit Variables

2.7 A Single and Universal Catch-All Multidimensional Model or Multiple Simpler Models?

Exercises

Term Assignments

Chapter 3: Project Scenarios, Budgeting, and Contingency Planning*

3.1 Introduction

3.2 Project Scenarios for Estimation Purposes

3.3 Probability of Underestimation and Contingency Funds

3.4 A Contingency Example for a Single Project

3.5 Managing Contingency Funds at the Portfolio Level

3.6 Managerial Prerogatives: An Example in the AGILE Context

3.7 Summary

Further Reading: A Simulation for Budgeting at the Portfolio Level

Exercises

Term Assignments

Part II: Estimation Process: What Must be Verified?

Chapter 4: What Must be Verified in an Estimation Process: An Overview

4.1 Introduction

4.2 Verification of the Direct Inputs to an Estimation Process

4.3 Verification of the Productivity Model

4.4 Verification of the Adjustment Phase

4.5 Verification of the Budgeting Phase

4.6 Re-Estimation and Continuous Improvement to The Full Estimation Process

Further Reading: The Estimation Verification Report

Exercises

Term Assignments

Chapter 5: Verification of the Dataset Used to Build the Models

5.1 Introduction

5.2 Verification of

Direct

Inputs

5.3 Graphical Analysis – One-Dimensional

5.4 Analysis of the Distribution of the Input Variables

5.5 Graphical Analysis – Two-Dimensional

5.6 Size Inputs Derived from a Conversion Formula

5.7 Summary

Summary

Further Reading: Measurement and Quantification

Exercises

Term Assignments

Exercises – Further Reading Section

Term Assignments – Further Reading Section

Chapter 6: Verification of Productivity Models

6.1 Introduction

6.2 Criteria Describing the Relationships Across Variables

6.3 Verification of the Assumptions of the Models

6.4 Evaluation of Models by Their Own Builders

6.5 Models Already Built–Should You Trust Them?

6.6 Lessons Learned: Distinct Models by Size Range

6.7 Summary

Exercises

Term Assignments

Chapter 7: Verification of the Adjustment Phase

7.1 Introduction

7.2 Adjustment Phase in the Estimation Process

7.3 The Bundled Approach in Current Practices

7.4 Cost Drivers as Estimation Submodels!

7.5 Uncertainty and Error Propagation

Pages

xiii

xiv

xvii

xviii

xix

xxi

xxii

xxiii

xxiv

xxv

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

257

258

259

260

261

Guide

Cover

Table of Contents

Foreword

Part I: Understanding the Estimation Process

Begin Reading

List of Illustrations

Figure 1.1

Figure 1.2

Figure 1.3

Figure 1.4

Figure 1.5

Figure 1.6

Figure 1.7

Figure 1.8

Figure 1.9

Figure 1.10

Figure 1.11

Figure 1.12

Figure 1.13

Figure 1.14

Figure 1.15

Figure 1.16

Figure 2.1

Figure 2.2

Figure 2.3

Figure 2.4

Figure 2.5

Figure 2.6

Figure 2.7

Figure 2.8

Figure 2.9

Figure 2.10

Figure 2.11

Figure 2.12

Figure 2.13

Figure 2.14

Figure 2.15

Figure 2.16

Figure 2.17

Figure 2.18

Figure 2.19

Figure 3.1

Figure 3.2

Figure 3.3

Figure 3.4

Figure 3.5

Figure 3.6

Figure 3.7

Figure 3.8

Figure 3.9

Figure 3.10

Figure 4.1

Figure 4.2

Figure 4.3

Figure 4.4

Figure 4.5

Figure 5.1

Figure 5.2

Figure 5.3

Figure 5.4

Figure 5.5

Figure 5.6

Figure 5.7

Figure 5.8

Figure 5.9

Figure 5.10

Figure 5.11

Figure 6.1

Figure 6.2

Figure 6.3

Figure 6.4

Figure 6.5

Figure 6.6

Figure 7.1

Figure 7.2

Figure 7.3

Figure 7.4

Figure 7.5

Figure 7.6

Figure 7.7

Figure 7.8

Figure 7.9

Figure 8.1

Figure 8.2

Figure 8.3

Figure 8.4

Figure 9.1

Figure 9.2

Figure 9.3

Figure 9.4

Figure 10.1

Figure 10.2

Figure 10.3

Figure 10.4

Figure 10.5

Figure 11.1

Figure 11.2

Figure 12.1

Figure 12.2

Figure 12.3

Figure 12.4

Figure 12.5

Figure 12.6

Figure 12.7

Figure 12.8

Figure 12.9

Figure 13.1

Figure 13.2

Figure 13.3

List of Tables

Table 3.1

Table 5.1

Table 5.2

Table 5.3

Table 6.1

Table 6.2

Table 7.1

Table 7.2

Table 7.3

Table 8.1

Table 8.2

Table 8.3

Table 8.4

Table 8.5

Table 8.6

Table 9.1

Table 9.2

Table 9.3

Table 9.4

Table 10.1

Table 10.2

Table 11.1

Table 11.2

Table 11.3

Table 11.6

Table 12.1

Table 12.2

Table 12.3

Table 12.4

Software Project Estimation

The Fundamentals for Providing High Quality Information to Decision Makers

Alain Abran

Published by John Wiley & Sons, Inc., Hoboken, New Jersey.

Published simultaneously in Canada.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4470, or on the web at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permission.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.

For general information on our other products and services or for technical support, please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic formats. For more information about Wiley products, visit our web site at www.wiley.com.

Library of Congress Cataloging-in-Publication Data:

Abran, Alain, 1949-

Software project estimation : the fundamentals for providing high quality information to decision makers / Alain Abran.

pages cm

ISBN 978-1-118-95408-9 (pbk.)

1. Computer software–Development–Estimates. I. Title.

QA76.76.D47A245 2015

005.1–dc23

2014033319

Foreword

Software project estimation is a challenge in most software organizations – and it is also a challenge to their customers, who endure software projects significantly over budget, with significant delays, less functionality than promised, and with unknown levels of quality.

Is industry better today at estimating software projects than it was 40 years ago, and do today's estimation models perform better?

What has not changed much in software estimation over that period of time?

Software managers (and their development staff) across the world are still expected to meet budget targets and deadlines typically determined based on imprecise requirements.

Researchers continue to develop increasingly complex estimation models and techniques in pursuit of ‘accurate’ estimates.

Estimation tools are still offered (at a cost by estimation tool vendors or free of charge on the Web and in books), for which there is little or no documented evidence on how these tools have performed on projects completed.

Books and tools on software estimation have been around for four decades now, and a number of solutions (estimation tools, models and techniques) have been proposed to address the challenge of software estimation.

But how good are these solutions?

What knowledge is available to assess the estimation tools available?

What do managers typically know about the quality of their estimation process, or about the performance of estimation tools available in the marketplace? Usually not much! Still, management takes a lot of decisions based on the numbers that these estimation tools yield.

In estimation:

The role of the software estimator is not to promise miracles, but to provide the best and most complete technical

information

context

, and

insights

; that is, it is the software estimator's role to provide his manager with

information to support decision making

;

The role of the manager is to look at all this information, select and allocate a project budget, and then manage the risks associated with it: it is the manager's role to take risks and manage those risks along the way.

When an organization has collected its own data and developed its own set of capabilities for analyzing their data and documenting the quality of their estimation models, then it has developed:

a key competitive advantage in market-oriented organizations, and

a key credibility advantage in organizations in non competitive contexts.

When an organization has not measured its own productivity on past projects, it is mostly in the dark about:

how the organization is performing,

how much a manager's performance differs from anyone else's, and

how much the assumptions made in a manager's estimation model differ from those made in someone else's!

In this context, which is typical of many software organizations, using estimation models originating in environments with different productivity performance ratios does not provide real value. This is all the more true when little is known about:

the quality of the data in these external repositories, or

the quality of the estimation models in the environments in which they have been built.

Those who might feel good about such models, with all their fancy features and cost drivers, deceive themselves about the numbers coming from these ‘black-boxes’.

This book teaches the way to develop estimationinformation (that is, numbers + context) to be used by managers in making budgeting decisions in contexts of uncertainty.

This book is not about:

black-box estimation claiming to handle all cost drivers at once;

a cookbook with estimation recipes;

a compendium of estimation models, techniques and cost drivers;

a compendium of hints to handle the detailed planning for each project phase.

This book is about some of the best engineering practices in software project estimation, including:

the right concepts to measure software project productivity – i.e. functional size measurement;

how to use productivity findings to develop estimation models;

how to verify the quality of the various components of an estimation process;

how to provide value (i.e. the right information) to support decision making in software project management (budgeting and control).

…and no, there is no engineering, even in software estimation, without a sound statistical foundation!

Summary

This book is not geared to readers looking for quick one-shot solutions. It is geared to those interested in building a long term and sustainable competitive advantage in software estimation by learning about the best practices and what is needed to implement them (including the necessary effort for data collection and data analysis using simple and sound statistical methods). prior to exploring much more complex statistical approaches, including for instance machine learning techniques or fuzzy logic.

Overview

In this book we share years of experience in the design of credible software estimation processes as decision support tools for managers.

We introduce the basic statistics and economics concepts needed to understand the fundamentals of the design and evaluation of software estimation models, and improvements to them.

Because quantitative data and quantitative models constitute a fundamental concept in engineering, science, and management, this book will be useful to software organizations of all sizes, and managers will find in it effective strategies for improving the quantitative aspects of software project estimation, along with numerous examples.

The book is intended for IT practitioners, managers, and auditors involved in software project estimation, and for students registered in software project management courses.

Structure and organization

The book is organized into three parts and thirteen chapters.

Part 1

Part 2

Part 3

Understanding the Estimation Process

Estimation Process: What Must be Verified?

Building Estimation Models: Data Collection & Analysis

Chapters 1 to 3

Chapters 4 to 7

Chapters 8 to 13

Part 1 presents various views of software estimation that both estimators and managers must be aware of when designing and using software estimation models for decision making. It explains the structure of an estimation process, including the productivity models embedded in the estimation process, and clarifies the distinct roles and responsibilities that estimators and managers have. Finally, it introduces a number of economics concepts that must be taken into consideration, such as economies/diseconomies of scale and fixed/variable costs.

Part 2 introduces the concepts and techniques necessary for understanding that the quality of the outcomes of an estimation process depends on the quality of its inputs, of the underlying productivity models it uses, and an understanding of the limitations of the factors added as adjustments for estimation purposes.

Part 3 presents a number of issues related to building estimation models, including data collection and the use of international standards for ease of comparison across groups, organizations, and countries. In addition, it looks at how to build models which have more than one independent variable, using quality data as input and based on a number of economics concepts.

Part 1

: Understanding the Estimation Process

Chapter 1

: The Estimation Process: Phases & Roles

The estimation process and its various phases is introduced, along with the distinct roles and responsibilities of software estimators and their managers.

Chapter 2

: Engineering & Economics Concepts for Understanding Software Process Performance

Some key economics concepts are presented which are useful for understanding, and then modeling, the performance of the development process underlying productivity models. In particular, the concepts of economies/diseconomies of scale and of fixed/variable costs in production models are explained, as well as some characteristics of typical and atypical datasets in software engineering, and the presence of explicit and implicit variables in the productivity model.

Chapter 3

: Project Scenarios, Budgeting & Contingency Planning

The impact of the selection of a single budget value among a range of estimates is discussed, including the identification of scenarios and their corresponding probabilities, and the identification and management of contingencies at the project portfolio level.

Part 2

: Estimation Process: What Must be Verified?

Chapter 4

: What Must be Verified in an Estimation Process: An Overview

The various parts of an estimation process that must be understood and verified when building and using productivity models are identified. We look at models from an engineering perspective, not from a ‘craft’ perspective.

Chapter 5

: Verification of the Dataset Used to Build the Models

The criteria required to analyze the quality of the direct inputs to mathematical models are presented, that is, the independent variables used to predict the dependent variable for estimation purposes.

Chapter 6

: Verification of the Models

The criteria required to analyze the quality of mathematical models are presented, along with the outcomes of these models. Then, by way of illustration, these quality criteria are used to evaluate the performance of the models and tools proposed to the industry.

Chapter 7

: Verification of the Adjustment Phase

Uncertainties and errors are inherent in measurements and in models of relationships across factors. Some sources of uncertainty and error are presented, and we show how they can accumulate when additional factors are introduced into the estimation process.

Part 3

: Building Estimation Models: Data Collection & Analysis

Chapter 8

: Data Collection & Industry Standards: The ISBSG Repository

Models for industry usage should be based on well defined and standardized definitions of the parameters included in the estimation process. Some standards for software projects data collection that have been defined by the International Software Benchmarking Standards Group – ISBSG – are introduced. Standardized definitions are, of course, critical for internal and external benchmarking, as well as for building productivity and estimation models.

Chapter 9

: Building & Evaluating Single Variable Models

The way to build a model, one independent variable at a time, is illustrated, starting with the variable recognized as the most significant, that is, the size of the software to be delivered. The way to build models using the ISBSG repository of projects is also shown, including data preparation and the identification of relevant samples to handle additional descriptive variables, such as the development environment.

Chapter 10

: Building Models with Categorical Variables

A case study on building models with industry data on project size as the key factor is presented, along with a few additional categorical variables, and the way to analyze and understand the quality of such models.

Chapter 11

: Contribution of Productivity Extremes in Estimation

An analysis is presented on how projects with the best and worst productivity are identified, and we show how to take advantage of the lessons learned in analyzing their performance and use this information for estimation purposes.

Chapter 12

: Multiple Models from a Single Dataset

An analysis is presented on how to identify multiple models from a single dataset by exploring concepts of economies and diseconomies of scale, as well as process performance capabilities and the impact of constraints on productivity.

Chapter 13

: Re-Estimation: A Recovery Effort Model

Throughout a software project, a number of factors influence productivity, functions are added and/or modified, risks materialize, and so on. As a result, projects often have to be re-estimated across life cycle phases. An approach to building re-estimation models is presented.

Additional material to complement the information contained in this book can be found at http://profs.etsmtl.ca/aabran/English/Autres/index.html

The table below provides a reading guide for software managers.

Chapters to read

Why?

What to do with the information

Part 1

Chapter 1

: full reading

Estimation is a multi phases process and estimators and managers have distinct and complementary responsibilities.

Verify that your estimation process covers all the phases described in this chapter & corresponding responsibilities well understood.

Chapter 2

: full reading

Economics concepts are useful for estimation purposes: they help explain fundamental issues in the software project cost structure, such as fixed/variable costs and economies/diseconomies of scale in software development.

Ask your software engineers:1- What are my fixed-variable efforts in software projects?2- Do we have economies or diseconomies of scale in our software development process?

Chapter 3

: full reading

Estimators should provide scenarios and plausible ranges of estimates. From these, business executives should allocate a project budget price, as well as contingency funds at the level of a portfolio of projects.

The manager should be the one to select the project budget (from a range of estimates) and set aside contingency funds to manage the inherent estimation risks.

Part 2

Chapters 4 to 7: quick reading

Estimation models should produce ‘credible numbers’: the quality of the estimation models must be verified and documented – if not, estimation is reduced to ‘garbage in, garbage out’.

Ask your estimators to document the quality controls implemented in your estimation process.Ask for an audit of your estimation process.

Part 3

Chapters: 8 to 13: quick reading

Standardized definitions for the data collected allow performance comparison within your organization and across the industry.Engineering techniques should be used for data analysis and building estimation models.

Verify that your organization is using the best industry standards for data collection.Ask your estimators to implement the best practices recommended in these chapters.

When project budgets go off-track, regular estimation models no longer work: re-estimation models are needed.

Ask your estimator to develop a re-estimation model.

The table below provides a reading guide for IT practitioners, IT auditors and undergraduate or graduate students interested in:

developing specialized expertise in software estimation,

verifying the quality of existing software estimation models and processes, or

designing new software estimation models and processes.

Chapters to read

Why?

What to do with the information

Part 1

Chapters 1 to 3: full reading

Estimation models must be based on a good understanding of the performance of an organization: fixed/variable costs, as well as economies/diseconomies of scale in software development.

When preparing project estimates, use your organization's historical data on fixed/variable costs as the foundation for your estimation process.Clarify the boundaries of the responsibilities between estimator and manager.

Part 2

Chapters 4 to 7: full reading

Estimation models should produce ‘information’, not only numbers. These 4 chapters illustrate what must be verified and which criteria must be used to document the quality of the outcomes of the productivity models used or to be implemented in your organization.This chapter also illustrates that adding more factors does not automatically increase certainty.

For decision making, you must provide information: i.e. numbers and context.This includes documenting the quality of the inputs to your productivity models, as well as the probable range of estimates.

Part 3

Chapters: 8 to 13: full reading

To design a credible estimation process, you need:

–

standards for data collection,

At estimation time, use the proposed techniques for building sound estimation models based on relevant data sets.

–

identification of statistical outliers,

–

select relevant samples for data analysis,

–

built single and multiple variable models,

–

take into account non quantitative variables.

–

And at re-estimation time, a number of additional constraints have to be taken into account.

At re-estimation time, include the relevant additional productivity factors.

Acknowledgments

A number of colleagues in industry and at universities across the world, as well as former graduate students, have helped clarify many of the concepts presented in this book over the years, in particular:

Chapter

Contributors

2 – Engineering and Economics Concepts for understanding Software Process Performance

Juan Cuadrado-Gallego, University of Alcala (Spain)

3 – Project Scenarios, Budgeting and Contingency Planning

Eduardo Miranda, Carnegie Mellon University (USA)

7 – Verification of the Adjustment Phase

Luca Santillo, Agile Metrics (Italy)

8 – Data collection and industry standards: the ISBSG repository

David Déry (Canada)

Laila Cheikhi, ENSIAS (Morocco)

9 – Building and evaluating single variable models

Pierre Bourque, ETS – U. Québec (Canada)

Iphigénie Ndiaye (Canada)

10 – Building models with categorical variables

Ilionar Silva and Laura Primera (Canada)

11 – Contribution of productivity extremes in estimation

Dominic Paré (Canada)

12 – Multiple models from a single dataset

Jean-Marc Desharnais, ETS – U. Québec (Canada)

Mohammad Zarour, Prince Sultan University

Onur Demırörs, Middle East Technical University (Turkey)

13 – Re-estimation: a recovery effort model

Eduardo Miranda, Carnegie Mellon University (USA)

Special thanks to:

Professor Cuauhtémoc Lopez Martin of the University of Guadalajara and Charles Symons, who have provided very thoughtful comments on draft versions of this book,

Mr. Maurice Day for improvements to the graphics included in this book.

Above all, this book is dedicated to:

those who, over the years, have provided me with feedback and insights on software estimation, and who are continuously contributing, each in their own way, to the improvement of software estimation as a foundation for sound, quantitatively based decision-making;

my PhD students, many with years of industry practice, who have explored various specialized views to develop a more in-depth understanding of software estimation models.

About the Author

Dr. Alain Abran is a professor of software engineering at the École de Technologie Supérieure (ETS) – Université du Québec, Montréal, Canada.

Dr. Abran has more than 20 years of industry experience in information systems development and software engineering, and 20 years of university teaching. He holds a PhD in electrical and computer engineering (1994) from École Polytechnique de Montréal (Canada) and Master's degrees in Management Sciences (1974) and Electrical Engineering (1975) from the University of Ottawa (Canada).

He is the chairman of the Common Software Measurement International Consortium (COSMIC) – www.cosmicon.com. He published Software Metrics and Software Metrology in 2010, Management of Software Maintenance1 in 2008, both at Wiley & IEEE CS Press, and co-edited the 2004 version of the Guide to the Software Engineering Body of Knowledge (www.swebok.org).

His research interests include software productivity and estimation models, software quality, software measurement, functional size measurement methods, software risk management, and software maintenance management.

Most of his publications can be downloaded from: http://www.researchgate.net

Dr. Abran can be contacted at: [email protected]

Co-author: Alain April

Part I

Understanding the Estimation Process

Estimation is not at all about coming up with a magic number to which everyone must commit at the peril of their professional career (which leads to staff members spending lots of overtime attempting to meet unrealistic deadlines.)

Part 1 of this book consists of three chapters, in which some of the key concepts of an estimation process are introduced.

Chapter 1 introduces the estimation process, including:

the collection of data to be input to the estimation process,

their usage with a productivity model,

the adjustment phase to handle project assumptions, uncertainties, and risks,

the budgeting phase,

the estimator role: to provide information on a range of estimates,

the manager role: to select a specific budget from the range of estimates identified by the estimator.

Chapter 2 explains the relationship between the software development life process and the classic model of a process. A number of economics concepts are introduced and illustrated in the context of software projects, such as:

economies and diseconomies of scale, and

fixed and variable costs.

Chapter 3 discusses the impact of the selection of a single budget value from a range of estimates, including the identification of scenarios and their corresponding probabilities, and the identification and management of contingencies at the project portfolio level.

Chapter 1The Estimation Process: Phases and Roles

Objectives

This chapter covers

–

Two generic approaches to estimation: judgment-based and engineering based

–

An overview of the process for estimating software projects

–

The foundation: The productivity model

–

The phases of the estimation process

–

Roles and responsibilities in estimating and budgeting

1.1 Introduction

When an organization has not measured its own productivity on past projects, it is mostly in the dark about:

how the organization is performing,

how much a manager's performance differs from someone else's, and

how much the assumptions made in a manager's estimation judgment differ from those made in someone else's!

In this context, which is typical in many software organizations, using productivity models originating in environments with different productivity performance ratios does not provide real value. This is all the more true when little is known about

the quality of the data in these external repositories and

the quality of the productivity models within the environments in which they have been built.

When an organization has collected its own data and developed its own set of capabilities for analyzing those data and documenting the quality of their productivity models, then it has developed

a key competitive advantage in market-oriented organizations and

a key credibility advantage in organizations in noncompetitive contexts.

This chapter presents an overview of the phases of an estimation process and explains the differences between a productivity model and its use in an estimation process. It is organized as follows:

Section 1.2

introduces two generic approaches to estimation: judgment and engineering.

Section 1.3

provides an overview of some common practices and expectations involved in estimating software projects.

Section 1.4

discusses the levels of uncertainty in an estimation process.

Section 1.5

presents the key concepts of a productivity model.

Section 1.6

explains the use of a productivity model in an estimation process.

Section 1.7

discusses the estimation responsibilities in a business context.

Section 1.8

explains the differences between budgeting and pricing.

Section 1.9

provides a summary of the chapter.

1.2 Generic Approaches in Estimation Models: Judgment or Engineering?

1.2.1 Practitioner's Approach: Judgment and Craftsmanship

In contrast to estimation with mathematical models, where explicit cost drivers are included in the models as either quantitative or categorical parameters, which are manipulated with well-described mathematical equations, the estimation technique often used in practice in industry (also referred to as the expert judgment estimation approach) would not typically document which parameters are taken into account, or how they are explicitly combined.

The overall estimation process in the expert judgment approach is similar to the estimation process described later in this chapter, but is much less transparent, of course, and there is no possibility of tracing back to historical data on how the expert judgment models were built. In addition, it is not feasible to gauge the performance of the expert judgment models when there are no objectively quantified and standardized data on key project variables, such as software size:

A project might appear to be successful if it has respected the “official” budget; however, without the ability to verify that all the promised functions have been delivered, it is a mistake to claim that the estimates were correct: when only a portion of the required functions are delivered, then the expected benefits cannot all be harvested, which destroys the cost-benefit analysis that justified the launching of the project in the first place.

We can conclude from this that analyzing the performance of an expert-based estimate without a corresponding analysis of the functionality delivered is of very limited value.

Of course, the expert judgment approach is highly dependent on the specific expertise of the people participating in the estimation process, and will vary from project to project, making performance assessment challenging.

This dependency of expert judgment on expertise gives the estimation process many of the characteristics of a craft, which is mostly dependent on the abilities of the craftsmen, rather than on a thoroughly tested and well-defined engineering technique.

The decision as to which cost drivers to include, as well as the determination of the values of each interval within a cost driver for particular impact, is most often based entirely on the judgment of a group of estimation tool builders, or even a single tool builder.

Practitioners will also typically attempt to improve conventional software estimation models using a similar approach:

The addition, modification, and/or deletion of cost drivers is based on a value judgment (also referred to as

expert judgment

subject matter expertise

An impact factor is also assigned on this basis.

What this means is that the improvement process is typically subjective and most often undertaken without statistical analysis to support proposed changes.

1.2.2 Engineering Approach: Modest – One Variable at a Time

Building software models from an engineering perspective is based on

Observation of past projects and quantitative data collection

including identification of the scale types of the variables, and taking them into account to ensure adequate use of those variables in productivity models.

Analysis of the impact of individual variables, one at a time.

Selection of relevant samples, and of samples of sufficient size from a statistical viewpoint.

Documentation and analysis of the demographics of the dataset used.

Very careful extrapolation to contexts other than those from which the data were collected.

The engineering approach consists of investigating the factors involved and studying them one at a time before combining them.

In this approach, it is not taken for granted that it is feasible for a single model to handle all sets of conditions:

Instead, a search is conducted for models that are reasonably good within a well-identified and understood set of constraints.

This is the approach taken in this book for building the basis for productivity models:

Look at the work–effort relationship, one variable at a time, to gain insights for each variable.

Taking this approach means initially obtaining a number of productivity models for each variable, and admitting that

no one model will be perfect (i.e., it will not take into account the other variables

directly

) and that

each model will teach us something about the effect of that single variable on the dependent variable, which is effort.

1.3 Overview of Software Project Estimation and Current Practices

Here we present an overview of the estimation process, followed by some current practices and expectations.

1.3.1 Overview of an Estimation Process

A high-level view of a software estimation approach is depicted in Figure 1.1:

On the left are the inputs to the software estimation process. These inputs typically consist of

Product requirements:

the functional requirements requested by the users and allocated to the software.

the nonfunctional requirements, some of which will be allocated to software, and others to other parts of the system (hardware, procedures manual, etc.).

Software development process: typically, a specific life cycle is selected (agile, iterative, etc.), along with its various components, including the development platform, the programming languages, and the project team.

Project constraints: these are the constraints externally imposed on the project (predefined deadlines, maximum available budget, etc.).

In the center is a representation of the productivity model used as the foundation of the estimation process, and includes

the “implicit” models of each of the experts participating in the estimation process (typically, the productivity model of the experts is not documented).

mathematical models: regressions, case-based reasoning, neural networks, and so on.

On the right is the estimation output normally expected, which constitutes

an estimate of the amount of effort (or cost, or project duration) required to deliver software that will meet the requirements specified in input at the specified level of quality.

Figure 1.1 One Perception of an Estimation Process

1.3.2 Poor Estimation Practices

In the literature, there is a large body of knowledge on project estimation in general, and on software project estimation in particular; however, in practice, the software industry is plagued by a number of poor estimation practices, such as those illustrated in Figure 1.2:

The estimation inputs:

There is only a very brief description by the customer of the software system expected, usually at a very high level (i.e., poorly defined) and, of course, poorly documented. How many times are software staff asked to provide estimates based on a half-page description of user requirements? This type of estimation input is referred to as a “wish list” in

Figure 1.2

. Such a list will inevitably change over time, and most probably expand at an unpredictable rate.

In the hope of offsetting this lack of description of what is expected by users, a software manager will want to take as many cost drivers as possible into account, expecting in this way to lower his estimation risks.

The estimation model:

A formal or an informal model to mix (in a black-box manner) these ill-defined requirements together through the use of readily available:

local experience: internal or external experience (the expert judgment approach) or

mathematical models described in books or hidden in estimation tools.

The estimation output is made up of

a single estimate, which is made up of the mandated project budget that must be respected, along with the requirement that the expected functionality be produced within a prescribed period of time;

Note: this figure does not take into account unplanned overtime, for which the development team will not be paid!

an overly optimistic attitude, which is very common among software professionals, that the development team will outperform any previous historical performance and overcome all constraints in a timely manner; and

accountability on the part of the software engineer or project manager providing the estimate, in terms of meeting customer expectations and respecting the project budget allocated by senior management.

Figure 1.2 Some of the Poor Estimation Practices Observed in Industry

To summarize, in this worst practice, both customers and senior management expect that their staff (and suppliers) will commit to delivering the expected software functionality on time and on budget, and all this without having themselves worked out the details of what they expect as a well working product and the uncertainties inherent to any new project.

In other words, on the one hand, miracles are expected by customers and senior management, and, on the other, too many software staff, when providing single-point estimates, behave as if they are in the business of continuously delivering miracles!

Some of the Best Estimation Practices in Industry

Mature software organizations consider estimation as a process that gives them a business advantage over their competitors: to acquire this competitive advantage, they have invested in their estimation process to master the key factors, including:

–

investment in gathering project requirements and in understanding their qualities;

–

use of international standards for software measurement;

–

continuous quantitative measurement throughout the project life cycle;

–

quantitative analysis of their past performance: that is, how productive were they in terms of delivering past projects and meeting project objectives;

–

in-depth analysis of their estimation performance (actual vs estimated).

Some of the Worst Estimation Practices in Industry

–

Wishful thinking and single-point estimates.

–

Use of estimation black boxes (expert judgment and/or undocumented mathematical models).

–

Reliance on others' numbers: no investment in their estimation process to develop a sustainable competitive advantage.

1.3.3 Examples of Poor Estimation Practices

The following are some examples of poor estimation practices – see also Figure 1.3.

Inputs to estimation models:

Product requirements = Wish list:

No measurement of the functional requirements themselves, using international standards.

Use of post-project KLOC (thousands of lines of code) without considering the mix of programming languages and their different characteristics.

Size units often considered almost irrelevant.

Guesstimate of KLOC based on poor requirements and a poor understanding of the relationships between requirements and KLOC in various programming languages.

Development process:

Individual descriptive factors transformed into quantitative impact factors without knowledge of their accuracy and variance.

No objective quantitative knowledge of the impact of the project variables in their own development environment.

Total reliance on outside numbers without strong supporting evidence.

Productivity model:

Unknown estimation performance of the so-called experts in the expert-based estimation approach.

No verification that the assumptions necessary for each statistical technique have been met (e.g., The “normality” distribution of the variables for regression models).

Too many variables and not enough data points for sound statistical analysis.

No verification of the size of the software to be delivered in the analysis of the performance of the expert-based approach.

And so on.

Estimation output:

The dream: an accurate estimate.

Limited analysis of the range of candidate values and candidate causes of variations in the estimates.

Limited documentation of the quality of their estimation outcomes.

Figure 1.3 The Dream: An “Accurate” Estimate

1.3.4 The Reality: A Tally of Failures

Software project estimation is a recurrent and important activity in large and small organizations across the world, and a large amount of research has been performed on software project estimation over the past 50 years and a large number of models proposed to industry. The bottom line is, how well is software estimation performing in industry?

The answer is, not very impressively [Jorgensen and Molokken 2006; Jorgensen and Shepperd 2007; Petersen 2011]:

Figure 1.4

, constructed using data from the Standish Group Chaos Reports cited by [Eveleens and Verhoef 2010], shows that, over the 30-year period, barely 30% of software projects have been delivered on time and on budget:

Since the publication of the first Standish Group report in 1995, the software development community has been making some progress in its ability to complete development projects on time and on budget, but almost 70% of software projects still finish late and over budget, or are cancelled.

The 2008 study by El Eman and Koru [2008] puts the average number of challenged and failed projects at 50%.

Figure 1.4 Project Success Trends Based on Standish Group Data

[Adapted from Miranda 2010].