103,99 €
Mehr erfahren.
- Herausgeber: John Wiley & Sons
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
This book introduces Software Quality Assurance (SQA) and provides an overview of standards used to implement SQA. It defines ways to assess the effectiveness of how one approaches software quality across key industry sectors such as telecommunications, transport, defense, and aerospace. * Includes supplementary website with an instructor's guide and solutions * Applies IEEE software standards as well as the Capability Maturity Model Integration for Development (CMMI) * Illustrates the application of software quality assurance practices through the use of practical examples, quotes from experts, and tips from the authors
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 919
Veröffentlichungsjahr: 2017
Ähnliche
IEEE Press Editorial Board
Tariq Samad, Editor in Chief
Giancarlo Fortino
Xiaoou Li
Ray Perez
Dmitry Goldgof
Andreas Molisch
Linda Shafer
Don Heirman
Saeid Nahavandi
Mohammad Shahidehpour
Ekram Hossain
Jeffrey Nanzer
Zidong Wang
About IEEE Computer Society
IEEE Computer Society is the world's leading computing membership organization and the trusted information and career-development source for a global workforce of technology leaders including: professors, researchers, software engineers, IT professionals, employers, and students. The unmatched source for technology information, inspiration, and collaboration, the IEEE Computer Society is the source that computing professionals trust to provide high-quality, state-of-the-art information on an on-demand basis. The Computer Society provides a wide range of forums for top minds to come together, including technical conferences, publications, and a comprehensive digital library, unique training webinars, professional training, and the TechLeader Training Partner Program to help organizations increase their staff's technical knowledge and expertise, as well as the personalized information tool myComputer. To find out more about the community for technology leaders, visit http://www.computer.org.
IEEE/Wiley Partnership
The IEEE Computer Society and Wiley partnership allows the CS Press authored book program to produce a number of exciting new titles in areas of computer science, computing, and networking with a special focus on software engineering. IEEE Computer Society members continue to receive a 15\% discount on these titles when purchased through Wiley or at wiley.com/ieeecs.
To submit questions about the program or send proposals, please contact Mary Hatcher, Editor, Wiley-IEEE Press: Email: [email protected], Telephone: 201-748-6903, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ\break 07030-5774.
Software Quality Assurance
Claude Y. Laporte
Alain April
This edition first published 2018 © 2018 the IEEE Computer Society, Inc.
All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by law. Advice on how to obtain permission to reuse material from this title is available at http://www.wiley.com/go/permissions.
The rights of Claude Y. Laporte and Alain April to be identified as the authors of this work have been asserted in accordance with law.
Translated by Rosalia Falco.
Registered OfficeJohn Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA
Editorial Office111 River Street, Hoboken, NJ 07030, USA
For details of our global editorial offices, customer services, and more information about Wiley products visit us at www.wiley.com.
Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that appears in standard print versions of this book may not be available in other formats.
Limit of Liability/Disclaimer of WarrantyWhile the publisher and authors have used their best efforts in preparing this work, they make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives, written sales materials or promotional statements for this work. The fact that an organization, website, or product is referred to in this work as a citation and/or potential source of further information does not mean that the publisher and authors endorse the information or services the organization, website, or product may provide or recommendations it may make. This work is sold with the understanding that the publisher is not engaged in rendering professional services. The advice and strategies contained herein may not be suitable for your situation. You should consult with a specialist where appropriate. Further, readers should be aware that websites listed in this work may have changed or disappeared between when this work was written and when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages.
Library of Congress Cataloging-in-Publication Data
Names: Laporte, Claude Y., author. | April, Alain, author. Title: Software quality assurance / by Claude Y. Laporte, Alain April. Description: 1 | Hoboken, NJ : Wiley-IEEE Computer Society, Inc., 2018. | Includes bibliographical references and index. | Identifiers: LCCN 2017036440 (print) | LCCN 2017041869 (ebook) | ISBN 9781119312413 (pdf) | ISBN 9781119312420 (epub) | ISBN 9781118501825 (hardback) Subjects: LCSH: Computer software--Quality control. | Computer software--Quality control--Standards. | BISAC: TECHNOLOGY & ENGINEERING / Quality Control. Classification: LCC QA76.76.Q35 (ebook) | LCC QA76.76.Q35 L42 2018 (print) | DDC 005.3028/7--dc23 LC record available at https://lccn.loc.gov/2017036440
Cover image: ©naqiewei/Gettyimages Cover design by Wiley
CONTENTS
Preface
Acknowledgments
Chapter 1 Software Quality Fundamentals
1.1 Introduction
1.2 Defining Software Quality
1.3 Software Errors, Defects, and Failures
1.4 Software Quality
1.5 Software Quality Assurance
1.6 Business Models and the Choice of Software Engineering Practices
1.7 Success Factors
1.8 Further Reading
1.9 Exercises
Chapter 2 Quality Culture
2.1 Introduction
2.2 Cost of Quality
2.3 Quality Culture
2.4 The Five Dimensions of a Software Project
2.5 The Software Engineering Code of Ethics
2.6 Success Factors
2.7 Further Reading
2.8 Exercises
Note
Chapter 3 Software Quality Requirements
3.1 Introduction
3.2 Software Quality Models
3.3 Definition of Software Quality Requirements
3.4 Requirement Traceability During the Software Life Cycle
3.5 Software Quality Requirements and the Software Quality Plan
3.6 Success Factors
3.7 Further Reading
3.8 Exercises
Note
Chapter 4 Software Engineering Standards and Models
4.1 Introduction
4.2 Standards, Cost of Quality, and Business Models
4.3 Main Standards for Quality Management
4.4 ISO/IEC/IEEE 12207 Standard
4.5 ISO/IEC/IEEE 15289 Standard for the Description of Information Elements
4.6 IEEE 730 Standard for SQA Processes
4.7 Other Quality Models, Standards, References, and Processes
4.8 Specific Standards for an Application Domain
4.9 Standards and the SQAP
4.10 Success Factors
4.11 Further Reading
4.12 Exercises
Chapter 5 Reviews
5.1 Introduction
5.2 Personal Review and Desk-Check Review
5.3 Standards and Models
5.4 Walk-Through
5.5 Inspection Review
5.6 Project Launch Reviews and Project Assessments
5.7 Agile Meetings
5.8 Measures
5.9 Selecting the Type of Review
5.10 Reviews and Business models
5.11 Software Quality Assurance Plan
5.12 Success Factors
5.13 Tools
5.14 Further Reading
5.15 Exercises
Chapter 6 Software Audits
6.1 Introduction
6.2 Types of Audits
6.3 Audit and Software Problem Resolution According to ISO/IEC/IEEE 12207
6.4 Audit According to the IEEE 1028 Standard
6.5 Audit Process and the ISO 9001 Standard
6.6 Audit According to the CMMI
6.7 Corrective Actions
6.8 Audits for Very Small Entities
6.9 Audit and the SQA Plan
6.10 Presentation of an Audit Case Study
6.11 Success Factors
6.12 Further Reading
6.13 Exercises
Chapter 7 Verification and Validation
7.1 Introduction
7.2 Benefits and Costs of V&V
7.3 V&V Standards and Process Models
7.4 V&V According to ISO/IEC/IEEE 12207
7.5 V&V According to the CMMI Model
7.6 ISO/IEC 29110 and V&V
7.7 Independent V&V
7.8 Traceability
7.9 Validation Phase of Software Development
7.10 Tests
7.11 Checklists
7.12 V&V Techniques
7.13 V&V Plan
7.14 Limitations OF V&V
7.15 V&V in the SQA Plan
7.16 Success Factors
7.17 Further Reading
7.18 Exercises
Chapter 8 Software Configuration Management
8.1 Introduction
8.2 Software Configuration Management
8.3 Benefits of Good Configuration Management
8.4 SCM Activities
8.5 Baselines
8.6 Software Repository and Its Branches
8.7 Configuration Control
8.8 Configuration Status Accounting
8.9 Software Configuration Audit
8.10 Implementing SCM in Very Small Entities with ISO/IEC 29110
8.11 SCM and the SQAP
8.12 Success Factors
8.13 Further Reading
8.14 Exercises
Chapter 9 Policies, Processes, and Procedures
9.1 Introduction
9.2 Policies
9.3 Processes
9.4 Procedures
9.5 Organizational Standards
9.6 Graphical Representation of Processes and Procedures
9.7 Process Notation of ISO/IEC 29110
9.8 Case Study
9.9 Personal Improvement Process
9.10 Policies, Processes, and Procedures in the SQA Plan
9.11 Success Factors
9.12 Further Reading
9.13 Exercises
Note
Chapter 10 Measurement
10.1 Introduction—the Importance of Measurement
10.2 Software Measurement According to ISO/IEC/IEEE 12207
10.3 Measurement According to ISO 9001
10.4 The Practical Software and Systems Measurement Method
10.5 ISO/IEC/IEEE 15939 Standard
10.6 Measurement According to the CMMI Model
10.7 Measurement in Very Small Entities
10.8 The Survey as a Measurement Tool
10.9 Implementing a Measurement Program
10.10 Practical Considerations
10.11 The Human Side of Measurement
10.12 Measurement and the IEEE 730 SQAP
10.13 Success Factors
10.14 Further Reading
10.15 Exercises
Chapter 11 Risk Management
11.1 Introduction
11.2 Risk Management According to Standards and Models
11.3 Practical Considerations for Risk Management
11.4 Risk Management Roles
11.5 Measurement and Risk Management
11.6 Human Factors and Risk Management
11.7 Success Factors
11.8 Conclusion
11.9 Further Reading
11.10 Exercises
Chapter 12 Supplier Management and Agreements
12.1 Introduction
12.2 Supplier Requirements of ISO 9001
12.3 Agreement Processes of ISO 12207
12.4 Supplier Agreement Management According to the CMMI
12.5 Managing Suppliers
12.6 Software Acquisition Life Cycle
12.7 Software Contract Types
12.8 Software Contract Reviews
12.9 Supplier and Acquirer Relationship and the Sqap
12.10 Success Factors
12.11 Further Reading
12.12 Exercises
Chapter 13 Software Quality Assurance Plan
13.1 Introduction
13.2 SQA Planning
13.3 Executing the SQAP
13.4 Conclusion
13.5 Further Reading
13.6 Exercises
Appendix 1 Software Engineering Code of Ethics and Professional Practice (Version 5.2)
Appendix 2 Incidents and Horror Stories Involving Software
Glossary – Abbreviations – Acronyms
References
Index
EULA
List of Tables
Chapter 1
Table 1.1
Chapter 2
Table 2.1
Table 2.2
Table 2.3
Table 2.4
Table 2.5
Chapter 3
Table 3.1
Table 3.2
Table 3.3
Chapter 4
Table 4.1
Table 4.2
Table 4.3
Table 4.4
Table 4.5
Table 4.6
Table 4.7
Table 4.8
Table 4.9
Table 4.10
Table 4.11
Chapter 5
Table 5.1
Table 5.2
Table 5.3
Table 5.4
Table 5.5
Chapter 6
Table 6.1
Table 6.2
Table 6.3
Chapter 7
Table 7.1
Table 7.2
Table 7.3
Table 7.4
Table 7.5
Table 7.6
Table 7.7
Table 7.8
Table 7.9
Table 7.10
Chapter 8
Table 8.1
Table 8.2
Table 8.3
Table 8.4
Table 8.5
Chapter 9
Table 9.1
Table 9.2
Table 9.3
Table 9.4
Table 9.5
Table 9.6
Table 9.7
Table 9.8
Table 9.9
Table 9.10
Chapter 10
Table 10.1
Table 10.2
Chapter 11
Table 11.1
Table 11.2
Table 11.3
Table 11.4
Table 11.5
Table 11.6
Table 11.7
Table 11.8
Table 11.9
Table 11.10
Table 11.11
Table 11.12
Chapter 12
Table 12.1
Chapter 13
Table 13.1
Table 13.2
Table 13.3
Table 13.4
Table 13.5
Table 13.6
Table 13.7
List of Illustrations
Chapter 1
Figure 1.1
Software Quality in the SWEBOK
®
Guide [SWE 14].
Figure 1.2
Terminology recommended for describing software problems.
Figure 1.3
Errors, defects, and failures in the software life cycle.
Figure 1.4
Context of software requirements elicitation.
Figure 1.5
Reliability curve for hardware as a function of time.
Figure 1.6
Reliability curve of software.
Source
: Adapted from Pressman 2014. [PRE 14].
Chapter 2
Figure 2.1
Balance between the software quality level and the cost of quality [GAL 17].
Figure 2.2
Costs of propagating an error
1
[JON 00].
Figure 2.3
Defect injection distribution during the life cycle [SEL 07].
Figure 2.4
Data on software quality costs [HAL 96].
Figure 2.5
Software engineering culture.
Figure 2.6
Start coding… I'll go and see what the client wants!
Figure 2.7
Dilbert is threatened and must provide an estimate on the fly. DILBERT © 2007 Scott Adams. Used By permission of UNIVERSAL UCLICK. All rights reserved.
Figure 2.8
Dilbert tries to negotiate a change in his project. DILBERT © 2009 Scott Adams. Used By permission of UNIVERSAL UCLICK. All rights reserved.
Figure 2.9
Diagram of the flexibility of an in-house project.
Figure 2.10
Diagram of the flexibility of a critical software project.
Chapter 3
Figure 3.1
The three perspectives and 11 quality factors of McCall et al. (1977) [MCC 77].
Figure 3.2
Quality factors and criteria from McCall et al. (1977) [MCC 77].
Figure 3.3
Framework for measuring software quality as per the IEEE 1061 [IEE 98b].
Figure 3.4
Quality model for an ISO 25010 software product.
Figure 3.5
Context of software requirements elicitation.
Figure 3.6
Steps suggested for defining non-functional requirements.
Chapter 4
Figure 4.1
A few laws of nature used by some engineering disciplines.
Figure 4.2
The development of standards and models.
Figure 4.3
The evolution of standards SC7 [SUR 17].
Figure 4.4
Elements of a process [ISO 15].
Figure 4.5
The four life cycle process groups of ISO 12207 [ISO 17].
Figure 4.6
The links between requirements and the artifacts of a project [IEE 14].
Figure 4.7
Table of contents of a SQAP according to the IEEE 730 [IEE 14].
Figure 4.8
Structure of the staged representation of the CMMI [SEI 10a].
Figure 4.9
The staged representation of the CMMI
®
for Development model.
Figure 4.10
Structure of the S
3m
model [APR 08].
Figure 4.11
The main ITIL guides.
Figure 4.12
Processes of ISO 20000 service management system [ISO 11h].
Figure 4.13
Family of ISO 27000 Standards [ISO 05c].
Figure 4.14
Management and implementation processes for the software Basic profile [LAP 16a].
Figure 4.15
Activities for the software Basic profile management process [ISO 11e].
Figure 4.16
Software implementation activities for the Basic profile [ISO 11e].
Figure 4.17
DPs for the software Basic profile [LAP 08].
Figure 4.18
Processes and activities for the Basic profile for the development of systems [ISO 16f].
Figure 4.19
Independence versus SWSIL [CEN 01].
Chapter 5
Figure 5.1
Types of reviews used during the software development cycle [CEG 90] (© 1990 – ALSTOM Transport SA).
Figure 5.2
Objectives of a review.
Figure 5.3
Process of developing a document.
Figure 5.4
Review process.
Figure 5.5
Error detection during the software development life cycle [RAD 02].
Figure 5.6
Personal review process.
Figure 5.7
Desk-check review.
Figure 5.8
Desk-check review activities.
Figure 5.9
Desk-Check review form.
Figure 5.10
Peer reviews as described in the process area “Verification” of the CMMI-DEV.
Figure 5.11
The walk-through review.
Figure 5.12
The inspection process.
Chapter 6
Figure 6.1
Example of a conformity declaration form [ISO 04a].
Source
: Standards Council of Canada.
Figure 6.2
Audit process activities according to IEEE 1028.
Figure 6.3
Major steps of a software audit or assessment [APR 08].
Figure 6.4
Example of a problem resolution process.
Figure 6.5
Problem report and resolution proposal form.
Figure 6.6
ISO/IEC 29110 certification approach.
Chapter 7
Figure 7.1
V&V activities in the software development life cycle.
Figure 7.2
A V software life cycle describing when system and software validations are executed.
Figure 7.3
Example of software process phases where defects are injected [SEL 07].
Figure 7.4
Percentage of the defects detected by the development process phase [SEL 07].
Figure 7.5
Relationship between IV&V, supplier, and customer [EAS 96].
Figure 7.6
Validation representation of a process using the ETVX process notation [CEG 90] (© 1990 - ALSTOM Transport SA).
Figure 7.7
Typical table of contents of a validation plan [CEG 90].
Figure 7.8
A software requirements checklist [GIL 93].
Chapter 8
Figure 8.1
CM activities according to CMMI [SEI 00].
Figure 8.2
Example of a directory structure to control configuration with the Subversion CM tool.
Figure 8.3
Example of a project life cycle where planned baselines are indicated.
Figure 8.4
Description of the specifications steps using ETVX notation.
Figure 8.5
Choosing a bad branching strategy.
Figure 8.6
The simplest branching strategy of versions by branch.
Figure 8.7
A typical strategy of development and production branches.
Figure 8.8
Impact of a change request on a software [WIE 13].
Figure 8.9
Change request change management workflow.
Figure 8.10
Example of a status report with the Commit Monitor tool [COL 10].
Chapter 9
Figure 9.1
The SWEBOK
®
Guide software engineering process knowledge area [SWE 14].
Figure 9.2
Example of a quality system documentation model.
Figure 9.3
Processes documented for an expert.
Figure 9.4
Control flow graphical notation objects.
Figure 9.5
ETVX notation concept [RAD 85].
Figure 9.6
Illustration of the modified ETVX notation [LAP 97].
Figure 9.7
Template of a textual process description using the ETVX notation.
Figure 9.8
Example of a NASA process graphically represented using the ETVX notation [NAS 04].
Figure 9.9
A NASA template used to explain the ETVX notation [NAS 04].
Figure 9.10
A configuration management process graphically represented using the ETVX notation ETVX [LAP 97].
Figure 9.11
Bombardier Transport software life cycle high level representation [LAP 12].
Figure 9.12
Notation IDEF0 [IEE 98].
Figure 9.13
Three processes of the project planning phase.
Figure 9.14
Planning phase.
Figure 9.15
ETVX representation of step 120.
Figure 9.16
Integration of software engineering to the system engineering process [LAP 97].
Figure 9.17
BPMN list of event types [DEC 08].
Figure 9.18
Example of a provisioning process represented using BPMN notation.
Figure 9.19
PM process diagram of ISO/IEC 29110 [ISO 11e].
Figure 9.20
Four-stage roadmap of ISO/IEC 29110.
Figure 9.21
Effort estimation improvement (adapted from [HUM 00]).
Figure 9.22
Quality improvement (adapted from [HUM 00]).
Figure 9.23
Productivity improvement (adapted from [HUM 00]).
Chapter 10
Figure 10.1
Example of a measure used for decision making [WES 03].
Figure 10.2
Software engineering measurement as proposed by the SWEBOK [SWE 14].
Figure 10.3
Measuring process performance according to ISO 9001 [ISO 15].
Figure 10.4
The three disciplines of quantitative management [PSM 00].
Figure 10.5
Measurement process activities proposed by the PSM [PSM 00].
Figure 10.6
ISO 15939 software measurement process model [ISO 17c].
Figure 10.7
The software measurement process activities and tasks [ISO 17c].
Figure 10.8
Information measurement model [ISO 17c].
Figure 10.9
Example of a measurement construct for productivity [ISO 17c].
Figure 10.10
Examples of information included in a measurement plan [ISO 17c].
Figure 10.11
Components of a software measurement program [DES 95].
Chapter 11
Figure 11.1
Some software project internal and external risk sources.
Figure 11.2
Software projects—many surrounding contexts [
CHA 99
].
Figure 11.3
Typical expense curve of a project [FOR 05].
Figure 11.4
Risk management in the SWEBOK [SWE 14].
Figure 11.5
ISO 16085 recommended risk management process [ISO 06a].
Figure 11.6
Impact of variables as the project progresses [PMI 13].
Figure 11.7
Risk management activities.
Figure 11.8
A risk documentation template [WIE 98].
Figure 11.9
Grid illustrating risk exposure using three categories.
Figure 11.10
Example of three project risks.
Chapter 12
Figure 12.1
Interpretation of the CMMI-DEV for supplier agreement management [KON 00].
Figure 12.2
Example of a process map describing the RFI-RFQ stage for a software acquisition.
Figure 12.3
Graphical representation of risk sharing contract.
Chapter 13
Figure 13.1
The house of quality for software projects.
Figure 13.2
Example of a software project functional organization chart for a large organization.
Figure 13.3
Example of a workflow explaining the acceptance steps of a project.
Figure 13.4
Example of an acceptance process for a software project.
Guide
Cover
Table of Contents
Preface
Pages
ii
iii
iv
xv
xvi
xvii
xviii
xix
xx
xxi
xxiii
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
43
44
45
46
47
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
82
83
84
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
127
128
129
130
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
179
180
181
182
183
184
186
187
188
189
190
191
192
193
194
195
196
197
198
199
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
262
263
265
266
267
268
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
305
306
307
308
309
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
350
351
352
353
354
355
356
357
358
359
362
364
366
367
368
369
370
371
372
373
374
376
377
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
453
454
455
456
457
458
459
460
461
462
463
465
466
468
469
470
471
472
473
474
475
476
477
478
479
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
591
592
593
594
595
596
Preface
This book addresses the global challenge of the improvement of software quality. It seeks to provide an overview of software quality assurance (SQA) practices for customers, managers, auditors, suppliers, and personnel responsible for software projects, development, maintenance, and software services.
In a globally competitive environment, clients and competitors exert a great deal of pressure on organizations. Clients are increasingly demanding and require, among other things, software that is of high quality, low cost, delivered quickly, and with impeccable after-sales support. To meet the demand, quality, and deadlines, the organization must use efficient quality assurance practices for their software activities.
Ensuring software quality is not an easy task. Standards define ways to maximize performance but managers and employees are largely left to themselves to decide how to practically improve the situation. They face several problems:
–
increasing pressure to deliver quality products quickly;
–
increasing size and complexity of software and of systems;
–
increasing requirements to meet national, international, and professional standards;
–
subcontracting and outsourcing;
–
distributed work teams; and
–
ever changing platforms and technologies.
We will focus on the issue of SQA in industry and in public organizations. Industry and public organizations do not have access to a complete and integrated reference (i.e., one book) that can help them with assessing and improving activities specific to SQA. The SQA department must meet service standards for its customers, the technical criteria of the field, and maximize strategic and economic impacts.
The purpose of this book is to enable managers, clients, suppliers, developers, auditors, software maintainers, and SQA personnel to use this information to assess the effectiveness and completeness of their approach to SQA. Some of the issues raised here include:
–
What are the processes, practices, and activities of SQA and software improvement?
–
Can the current standards and models serve as a reference?
–
How do we ensure that managers and their staff understand the value of SQA activities and their implementation?
To answer these questions, we drew upon over 30 years of practical experience in software engineering and SQA in different organizations such as telecom, banking, defense, and transportation. This industry experience has convinced us of the importance of supporting the presentation of concepts and theory with references and practical examples. We have illustrated the correct and effective implementation of numerous quality assurance practices with real case studies throughout the book.
In many organizations, SQA is a synonym for testing. SQA, as presented in this book, covers a large spectrum of proven practices to provide a level of confidence that quality in software development and maintenance activities is independent of the life cycle selected by an organization or a project.
In this book, we will extensively use the term “software quality assurance” and the acronym SQA. As defined in the IEEE Standard for Software Quality Assurance Processes, IEEE 730-2014, a function is a set of resources and activities that achieve a particular purpose [IEE 14]. The SQA function can be executed by a software project team member. It could also be executed by an independent party (e.g., within a quality assurance (QA) department responsible for hardware, software, and supplier quality).
Structure and Organization of this Book
The book is divided into 13 chapters that cover the basic knowledge of SQA as identified, among others, by the IEEE 730 Standard for SQA Processes of the Institute of Electrical and Electronics Engineers (IEEE), the ISO/IEC/IEEE 12207 software life cycle processes standard, the Capability Maturity Model® Integration for Development (CMMI®-DEV) developed by the Software Engineering Institute as well as the ISO Guide to the Software Engineering Body of Knowledge (SWEBOK®). Numerous practical examples are used to illustrate the application of SQA practices.
Chapter 1: Software Quality Fundamentals
This chapter presents an overview of the knowledge required by SQA practitioners. From this overview, the book develops every aspect of the field and cites the important references that deepen each specific topic. We use the concept of business models to explain the significant differences in the selection of SQA practices. In this chapter, we also establish terms and their definitions as well as useful concepts that are used throughout the book.
Chapter 2: Quality Culture
This chapter introduces the concept of cost of quality, followed by practical examples. It also introduces the concept of quality culture and its influence on the SQA practices used. We also present five dimensions of a software project and how these dimensions can be used to identify the degrees of freedom a project manager has to ensure its success. In this chapter, we present an overview of software engineering ethics and the techniques to manage the expectations of managers and customers with respect to software quality.
Chapter 3: Software Quality Requirements
This chapter adds to the concepts and terminology already presented. It deals with software quality models as well as ISO standards on software quality models. These models propose classifications of software quality requirements and steps to define them. Practical examples describe how to use these models to define the quality requirements of a software project. Finally, we introduce the concept of requirements traceability and the importance of quality requirements for the SQA plan.
Chapter 4: Software Engineering Standards and Models
This chapter presents the most important international standards of ISO and models about software quality, such as the CMMI® developed by the Software Engineering Institute. A new ISO standard for very small organizations is also presented. The SQA practitioner and specialist will find proven practices from standards and models. This chapter provides the framework that can be useful for the following major software activities: (1) development, (2) maintenance, and (3) IT services. Finally, a short discussion on the standards specific to certain domains of application is presented, followed by recommendations for a SQA plan.
Chapter 5: Reviews
This chapter presents different types of software reviews: personal review, the “desk check,” the walk-through, and the inspection. We describe the theory about reviews and then provide practical examples. It introduces reviews in an agile context. Subsequently, we describe other reviews specific to a project: the project launch review and lessons learned review. The chapter concludes with a discussion on the selection of one type of review depending on your business domain and how these techniques fit into the SQA plan.
Chapter 6: Software Audits
This chapter describes the audit process and the software problem resolution process. Sooner or later in the career of a software practitioner, audits will be conducted in a software project. Standards and models describing audits are presented followed by a practical case. The chapter concludes with a discussion of the role of audits in the SQA plan.
Chapter 7: Verification and Validation
This chapter describes the concept of software verification and validation (V&V). It describes its benefits as well as the costs of using V&V practices. Then, the standards and models that impose or describe V&V practices for a project are described. Finally, the description of the contents of a V&V plan is presented.
Chapter 8: Software Configuration Management
This chapter describes an important component of software quality: software configuration management (SCM). The chapter begins by presenting the usefulness of SCM and typical SCM activities. It presents repositories and branching techniques involved in source code management, as well as the concepts of software control, software status, and software audits. Finally, this chapter concludes with a proposal for the implementation of SCM in a small organization and ends with a discussion of the role of SCM in the SQA plan.
Chapter 9: Policies, Processes, and Procedures
This chapter explains how to develop, document, and improve policies, processes, and procedures to ensure the effectiveness and efficiency of the software organization. It explains the importance of documentation presenting a few notations, as examples, to document processes and procedures. The chapter ends by presenting the Personal Software Process (PSP) developed by the Software Engineering Institute to ensure individuals have a disciplined and structured approach to software development that enables them to significantly increase the quality of their software products.
Chapter 10: Measurement
This chapter explains the importance of measurement, standards, and models, and presents a methodology to describe the requirements for a measurement process. It presents how measurement can be used by small organizations and small projects. Then, an approach to implement a measurement program, to detect the potential pitfalls, and the potential impact of human factors, when measuring, is discussed. The chapter concludes with a discussion of the role of measurement in a SQA plan.
Chapter 11: Risk Management
This chapter presents the main models and standards that include requirements for the management of risks. It discusses the risks that may affect the quality of software and techniques to identify, prioritize, document, and mitigate them. It also presents the roles of stakeholders in the risk management process and discusses the human factors to consider in the management of software risks. The chapter concludes with a discussion on the critical role of risk in the development of a SQA plan.
Chapter 12: Supplier Management and Agreements
This chapter deals with the important topic of supplier management and agreements. It discusses the major reviews and recommendations of the CMMI®. Subsequently, it lists the different types of software agreements and the benefits of the risk sharing agreement are illustrated using a practical example. This chapter concludes with recommendations for the content of the SQA plan when suppliers are involved.
Chapter 13: Software Quality Assurance Plan
This chapter summarizes the topics presented in the whole book by using the concepts presented in each chapter to assemble a comprehensive SQA plan that conforms to the IEEE 730 recommendation. It ends by presenting additional recommendations and practical examples.
Appendices
Appendix 1 – Software Engineering Code of Ethics and Professional Practice (Version 5.2)
Appendix 2 – Incidents and Horror Stories involving Software
Icons Used in the Book
Different icons are used throughout this book to illustrate a concept with a practical example; to focus on a definition; to present an anecdote, a tool, or checklist; or simply to provide a quote or a website. Consult the table below for the meaning of each icon.
Icon
Meaning
Practical example: An example of the practical application of a theoretical concept
Quote: A quote from an expert
Definition: A definition of an important term
Reference on the Web: An internet site to learn more about a specific topic
Tools: Examples of tools that support the techniques presented
Anecdote: A short story of a little known fact, or a curious point on the subject discussed
Checklist: A list of items to check, or not to be forgotten, during the execution of a presented technique
Tip: A tip from the authors or from another professional
Website
Supplementary material for teaching as well as for use in organizations (e.g., presentation material, solutions, project descriptions, templates, tools, articles, and links) is available on the website: www.sqabook.org.
Given that international standards are updated on a regular basis, the website will also highlight the latest developments that contribute to SQA practices.
Exercises
Each chapter contains exercises.
Notes
Many software engineering standards from ISO and IEEE have been cited in this book. These standards are updated on a regular basis, typically every five years, to reflect evolving software engineering practices. The accompanying website, www.sqabook.org, contains complementary information as well as the latest developments that impact or contribute to SQA practices described in each chapter and will evolve over time.
Since software engineering standards can be cited in an agreement between a customer and a supplier and add additional legal requirements to the agreement, we have not paraphrased the text of standards in our book, we have directly quoted the text from the standards.
Acknowledgments
We would like to thank Professor Normand Séguin of the University of Quebec in Montreal (UQAM), Mr. Jean-Marc Desharnais for allowing us to use an excerpt that describes the implementation process of a measurement program, and many graduate students of the Masters in Software Engineering from the École de technologie supérieure (ÉTS) who reviewed the chapters of this book and contributed through their vast industry experience, analogies, and case studies to enrich the content.
We are also very grateful to Kathy Iberle for letting us use her description of business models and their application in different business domains [IBE 02, IBE 03]. The business models are very helpful in understanding the risks facing a specific business domain as well as the breadth and depth of software engineering practices used to mitigate the risks. Finally, we would like to thank Karl Wiegers and Daniel Galin for allowing us to use figures from their books.
Chapter 1Software Quality Fundamentals
After completing this chapter, you will be able to:
–
use the correct terminology to discuss software quality issues;
–
identify the major categories of software errors;
–
understand the different viewpoints regarding software quality;
–
provide a definition of software quality assurance;
–
understand software business models as well as their respective risks.
1.1 Introduction
Software is developed, maintained, and used by people in a wide variety of situations. Students create software in their classes, enthusiasts become members of open-source development teams, and professionals develop software for diverse business fields from finance to aerospace. All these individual groups will have to address quality problems that arise in the software they are working with. This chapter will provide definitions for terminology and discuss the source of software errors and the choice of different software engineering practices depending on an organization's sector of business.
Every profession has a body of knowledge made up of generally accepted principles. In order to obtain more specific knowledge about a profession, one must either: (a) have completed a recognized curriculum or (b) have experience in the domain. For most software engineers, software quality knowledge and expertise is acquired in a hands-on fashion in various organizations. The Guide to the Software Engineering Body of Knowledge (SWEBOK) [SWE 14] constitutes the first international consensus developed on the fundamental knowledge required by all software engineers. Chapter 10 of SWEBOK is dedicated to software quality (see Figure 1.1) and its first topic is labeled “fundamentals” and introduces the concepts and terminology that form the underlying basis for understanding the role and scope of software quality activities. The second topic refers to the management processes and highlights the importance of software quality across the life cycle of a software project. The third topic presents practical considerations where various factors that influence planning, management, and selection of software quality activities and techniques are discussed. Last, software quality related tools are presented.
Figure 1.1 Software Quality in the SWEBOK® Guide [SWE 14].
1.2 Defining Software Quality
Before explaining the components of software quality assurance (SQA), it is important to consider the basic concepts of software quality. Once you have completed this section, you will be able to:
–
define the terms “software,” “software quality,” and “software quality assurance”;
–
differentiate between a software “error,” a software “defect,” and a software “failure.”
Intuitively, we see software simply as a set of instructions that make up a program. These instructions are also called the software's source code. A set of programs forms an application or a software component of a system with hardware components. An information system is the interaction between the software application and the information technology (IT) infrastructure of the organization. It is the information system or the system (e.g., digital camera) that clients use.
Is ensuring the quality of the source code sufficient for the client to be able to obtain a quality system? Of course not; a system is far more complex than a single program. Therefore, we must identify all components and their interactions to ensure that the information system is one of quality. An initial response to the challenge regarding software quality can be found in the following definition of the term “software.”
Software
All or part of the programs, procedures, rules, and associated documentation of an information processing system.
Computer programs, procedures, and possibly associated documentation and data pertaining to the operation of a computer system.
When we consider this definition, it is clear that the programs are only one part of a set of other products (also called intermediary products or software deliverables) and activities that are part of the software life cycle.
Let us look at each part of this definition of the term “software” in more detail:
–
Programs: the instructions that have been translated into source code, which have been specified, designed, reviewed, unit tested, and accepted by the clients;
–
Procedures: the user procedures and other processes that have been described (before and after automation), studied, and optimized;
–
Rules: the rules, such as business rules or chemical process rules, that had to be understood, described, validated, implemented, and tested;
–
Associated documentation: all types of documentation that is useful to customers, software users, developers, auditors, and maintainers. Documentation enables different members of a team to better communicate, review, test, and maintain software. Documentation is defined and produced throughout the key stages of the software life cycle;
–
Data: information that is inventoried, modeled, standardized, and created in order to operate the computer system.
Software found in embedded systems is sometimes called microcode or firmware. Firmware is present in commercial mass-market products and controls machines and devices used in our daily lives.
Firmware
Combination of a hardware device and computer instructions or computer data that reside as read-only software on the hardware device.
1.3 Software Errors, Defects, and Failures
If you listen closely during various meetings with your colleagues, you will notice that there are many terms that are used to describe problems with a software-driven system. For example:
–
The system
crashed
during production.
–
The designer made an
error.
–
After a review, we found a
defect
in the test plan.
–
I found a
bug
in a program today.
–
The system
broke down.
–
The client complained about a
problem
with a calculation in the payment report.
–
A
failure
was reported in the monitoring subsystem.
Do all of these terms refer to the same concept or to different concepts? It is important to use clear and precise terminology if we want to provide a specific meaning to each of these terms. Figure 1.2 describes how to use these terms correctly.
Figure 1.2 Terminology recommended for describing software problems.
Bug
Since the time of Thomas Edison, engineers have used the word “bug” to refer to failures in the systems that they have developed. This word can describe a multitude of possible problems. The first documented case of a “computer bug” involved a moth trapped in a relay of the Mark II computer at Harvard University in 1947. Grace Hopper, the computer operator, pasted the insect into the laboratory log, specifying it as the “First actual case of a bug being found” (see the page of this log in the photograph below).
In the early 1950s, the terms “bug,” “debug,” and “debugging,” as applied to computers and computer programs, started to appear in the popular press [KID 98].
Photograph from the Smithsonian National Museum of American History.
A failure (synonymous with a crash or breakdown) is the execution (or manifestation) of a fault in the operating environment. A failure is defined as the termination of the ability of a component to fully or partially perform a function that it was designed to carry out. The origin of a failure lies with a defect hidden, that is, not detected by tests or reviews, in the system currently in operation. As long as the system in production does not execute a faulty instruction or process faulty data, it will run normally. Therefore, it is possible that a system contains defects that have not yet been executed. Defects (synonym of faults) are human errors that were not detected during software development, quality assurance (QA), or testing. An error can be found in the documentation, the software source code instructions, the logical execution of the code, or anywhere else in the life cycle of the system.
Error, Defect, and Failure
Error
A human action that produces an incorrect result (ISO 24765) [ISO 17a].
Defect
A problem (synonym of fault) which, if not corrected, could cause an application to either fail or to produce incorrect results. (ISO 24765) [ISO 17a].
An imperfection or deficiency in a software or system component that can result in the component not performing its function, e.g. an incorrect data definition or source code instruction. A defect, if executed, can cause the failure of a software or system component (ISTQB 2011 [IST 11]).
Failure
The termination of the ability of a product to perform a required function or its inability to perform within previously specified limits (ISO 25010 [ISO 11i]).
Figure 1.3 shows the relationship between errors, defects, and failures in the software life cycle. Errors may appear during the initial feasibility and planning stages of new software. These errors become defects when documents have been approved and the errors have gone unnoticed. Defects can be found in both intermediary products (such as requirements specifications and design) and the source code itself. Failures occur when an intermediary product or faulty software is used.
Figure 1.3 Errors, defects, and failures in the software life cycle.
Source: Galin (2017). [GAL 17]. Adapted with permission of Wiley-IEEE Computer Society Press.
Case of Errors, Defects, and Failures
Case 1
: A local pharmacy added a software requirement to its cash register to prevent sales of more than $75 to customers owing more than $200 on their pharmacy credit card. The programmer did not fully understand the specification and created a sales limit of $500 within the program. This defect never caused a failure since no client could purchase more than $500 worth of items given that the pharmacy credit card had a limit of $400.
Case 2
: In 2009, a loyalty program was introduced to the clients of American Signature, a large furniture supplier. The specifications described the following business rules: a customer who makes a monthly purchase that is higher than the average amount of monthly purchases for all customers will be considered a Preferred Customer. The Preferred Customer will be identified when making a purchase, and will be immediately given a gift or major discount once a month. The defect introduced into the system (due to a poor understanding of the algorithm to set up for this requirement) involved only taking into account the average amount of current purchases and not the customer's monthly history. At the time of the software failure, the cash register was identifying far too many Preferred Clients, resulting in a loss for the company.
Case 3
: Peter tested Patrick's program when Patrick was away. He found a defect in the calculation for a retirement savings plan designed to apply the new tax-exemption law for this type of investment. He traced the error back to the project specification and informed the analyst. In this case, the test activity correctly identified the defect and the source of the error.
The three cases above correctly use the terms to describe software quality problems. They also identify issues that are investigated by researchers in the field of software quality in order to discover means to help eliminate these problems:
–
Errors can occur in any of the software development phases throughout the life cycle.
–
Defects must be identified and fixed before they become failures.
–
The cause of failures, defects, and errors must be identified.
Life Cycle
Evolution of a system, product, service, project, or other human-made entity from conception through retirement.
Development Life Cycle
Software life cycle process that contains the activities of requirements analysis, design, coding, integration, testing, installation, and support for acceptance of software products.
ISO 12207 [ISO 17]
During software development, defects are constantly being involuntarily introduced and must be located and corrected as soon as possible. Therefore, it is useful to collect and analyze data on the defects found as well as the estimated number of defects left in the software. By doing so, we can improve the software engineering processes and in turn, minimize the number of defects introduced in new versions of software products in the future.
Methods for classifying defects have been created for this purpose, one of which is explained in the chapter on verification and validation.
Undetected Hole in the Ozone Layer
The hole in the ozone layer over Antarctica went unnoticed for a long period of time because the TOMS data analysis software used by NASA as part of its project to map the ozone layer had been designed to ignore values that deviate significantly from the anticipated measurements.
The project was launched in 1978, but it was only in 1985 that the hole was discovered, and not by NASA. Following data analysis, NASA confirmed this design error.
http://earthobservatory.nasa.gov/Features/RemoteSensingAtmosphere/remote_sensing5.php
Depending on the business model of your organization, you will have to allow for varying degrees of effort in identifying and correcting defects. Unfortunately, there exists today a certain culture of tolerance for software defects. However, there is no question that we all want Airbus, Boeing, Bombardier, and Embraer to have identified and corrected all the defects in the software for their airplanes before we board them!
Many researchers have studied the source of software errors and have published studies classifying software errors by type in order to evaluate the frequency of each type of error. Beizer (1990) [BEI 90] provides a study that has combined the result of several other studies to provide us with an indication of the origin of errors. The following is a summarized list of this study's results [BEI 90].
–
25% structural;
–
22% data;
–
16% functionalities implemented;
–
10% construction/coding;
–
9% integration;
–
8% requirements/functional specifications;
–
3% definition/running tests;
–
2% architecture/design;
–
5% unspecified.
Researchers also try to determine how many errors can be expected in a typical software. McConnell (2004) [MCC 04] suggested that this number varied based on the quality and maturity of the software engineering processes as well as the training and competency of the developers. The more mature the processes are, the fewer errors are introduced into the development life cycle of the software. Humphrey (2008) [HUM 08] also collected data from many developers. He found that a developer involuntarily creates about 100 defects for each 1000 lines of source code written. In addition, he noted large variations for a group of 800 experienced developers, that is, from less than 50 defects to more than 250 defects injected per 1000 lines of code. At Rolls-Royce, the manufacturer of airplane engines, the variation published is from 0.5 to 18 defects per 1000 lines of source code [NOL 15]. The use of proven processes, competent and well-trained developers, and the reuse of already proven software components can considerably reduce the number of errors of a software.
McConnell also referenced other studies that have come to the following conclusions:
–
The scope of most defects is very limited and easy to correct.
–
Many defects occur outside of the coding activity (e.g., requirement, architecture activities).
–
Poor understanding of the design is a recurrent problem in programming error studies.
–
It is a good idea to measure the number and origin of defects in your organization to set targets for improvement.
Therefore, errors are the main cause of poor software quality. It is important to look for the cause of error and identify ways in which to prevent these errors in the future. As we have shown in Figure 1.3, errors can be introduced at each step of the software life cycle: errors in the requirements, code, documentation, data, tests, etc. The causes are almost always human mistakes made by clients, analysts, designers, software engineers, testers, or users. SQA will need to develop a classification of the causes of software error by category which can be used by everyone involved in the software engineering process.
For example, here are eight popular error-cause categories:
problems with defining requirements;
maintaining effective communication between client and developer;
deviations from specifications;
architecture and design errors;
coding errors (including test code);
non-compliance with current processes/procedures;
inadequate reviews and tests;
documentation errors.
Each of the eight categories of error causes listed above is described in more detail in the following sections.
1.3.1 Problems with Defining Requirements
Defining software requirements is now considered a specialty, which means a business analyst or a software engineer specialized in requirements. Requirements definition is the topic of interest groups as well as the subject of professional certification programs (see http://www.iiba.org).
There are a certain number of problems related to the clear, correct, and concise writing of requirements so that they can be converted into specifications that can be directly used by colleagues, such as architects, designers, programmers, and testers.
It must also be understood that there are a certain number of activities that must be mastered when eliciting requirements:
–
identifying the stakeholders (i.e., key players) who must participate in the requirements elicitation;
–
managing meetings;
–
interview techniques that can identify differences between wishes, expectations, and actual needs;
–
clear and concise documentation of functional requirements, performance requirements, obligations, and properties of future systems;
–
applying systematic techniques for requirement elicitation;
–
managing priorities and changes (e.g., changes to requirements).
It is clear that errors can arise when eliciting requirements. It can be difficult to cater to the wishes, expectations, and needs of many different user groups at the same time (see Figure 1.4). Therefore, it is important to pay particular attention to erroneous requirement definitions, the lack of definitions for critical obligations and software characteristics, the addition of unnecessary requirements (e.g., those not requested by the customer), the lack of attention to business priorities, and fuzzy requirements descriptions.
Figure 1.4 Context of software requirements elicitation.
Ordering Soup
Let us say you order soup at a restaurant. Your expressed requirement is “I would like the soup of the day.” But in fact, unexpressed wishes, expectations, and needs include: not too hot, not too cold; soup that is not too salty; utensils, salt, and pepper available on the table; clean washrooms; a well situated table; a quiet environment.
And that was a simple requirement, imagine a complex software!
A requirement is said to be of good quality when it meets the following characteristics:
–
correct;
–
complete;
–
clear for each stakeholder group (e.g., the client, the system architect, testers, and those who will maintain the system);
–
unambiguous, that is, same interpretation of the requirement from all stakeholders;
–
concise (simple, precise);
–
consistent;
–
feasible (realistic, possible);
–
necessary (responds to a client's need);
–
independent of the design;
–
independent of the implementation technique;
–
verifiable and testable;
–
can be traced back to a business need;
–
unique.
We will present techniques to help detect defects in requirements documentation in a later chapter concerning reviews.
We must also ensure that we are not looking for the Holy Grail of the perfect specification, since we do not always have the time or means, or the budget, to achieve this level of perfection.
The article by Ambler [AMB 04] entitled “Examining the Big Requirements Up Front Approach” suggests that it is sometimes ineffective to write detailed requirements early in the life cycle of a software project. He claims that this traditional approach increases the risk of a project's failure. He stipulates that a large percentage of these specifications are not integrated in the final version of the software and that the corresponding documentation is rarely updated during the project. He thus asserts that this way of working is outdated. In his article, he recommends using more recent agile techniques, such as Test-Driven Development, in order to produce a minimum amount of paper documentation.
We have observed that software analysts and designers also often use prototyping, which helps to partially eliminate the traditional requirements document and replace it with a set of user interfaces and test cases that describe the requirements, architecture, and software design to be developed. Prototypes prove useful for pinpointing what the client is envisioning and getting valuable feedback early in the project. In the next section, the development practices adopted by different business sectors will be discussed.
In a system with hardware and software components, requirements are developed at the system level and then allocated to hardware, software, and sometimes to an operator. The following figure illustrates the interactions between system, hardware, and software life cycle processes.
Systems engineering must work in close collaboration with hardware and software engineering during the allocation of system requirements (for example, functionalities and quality requirements such as safety and performance) to hardware and software.
