41,99 €
Mehr erfahren.
- Herausgeber: Packt Publishing
- Kategorie: Wissenschaft und neue Technologien
- Sprache: Englisch
A comprehensive, hands-on guide on unit testing framework for Java programming language
About This Book
- In-depth coverage of Jupiter, the new programming and extension model provided by JUnit 5
- Integration of JUnit 5 with other frameworks such as Mockito, Spring, Selenium, Cucumber, and Docker
- Best practices for writing meaningful Jupiter test cases
Who This Book Is For
This book is for Java software engineers and testers. If you are a Java developer who is keen on improving the quality of your code and building world class applications then this book is for you. Prior experience of the concepts of automated testing will be helpful.
What You Will Learn
- The importance of software testing and its impact on software quality
- The options available for testing Java applications
- The architecture, features and extension model of JUnit 5
- Writing test cases using the Jupiter programming model
- How to use the latest and advanced features of JUnit 5
- Integrating JUnit 5 with existing third-party frameworks
- Best practices for writing meaningful JUnit 5 test cases
- Managing software testing activities in a living software project
In Detail
When building an application it is of utmost importance to have clean code, a productive environment and efficient systems in place. Having automated unit testing in place helps developers to achieve these goals. The JUnit testing framework is a popular choice among Java developers and has recently released a major version update with JUnit 5.
This book shows you how to make use of the power of JUnit 5 to write better software.
The book begins with an introduction to software quality and software testing. After that, you will see an in-depth analysis of all the features of Jupiter, the new programming and extension model provided by JUnit 5. You will learn how to integrate JUnit 5 with other frameworks such as Mockito, Spring, Selenium, Cucumber, and Docker.
After the technical features of JUnit 5, the final part of this book will train you for the daily work of a software tester. You will learn best practices for writing meaningful tests. Finally, you will learn how software testing fits into the overall software development process, and sits alongside continuous integration, defect tracking, and test reporting.
Style and approach
The book offers definitive and comprehensive coverage of all the Unit testing concepts with JUnit and its features using several real world examples so that readers can put their learning to practice almost immediately. This book is structured in three parts:
- Software testing foundations (software quality and Java testing)
- JUnit 5 in depth (programming and extension model of JUnit 5)
- Software testing in practice (how to write and manage JUnit 5 tests)
Sie lesen das E-Book in den Legimi-Apps auf:
Seitenzahl: 318
Veröffentlichungsjahr: 2017
Ähnliche
Boni García
BIRMINGHAM - MUMBAI
Mastering Software Testing with JUnit 5
Copyright © 2017 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.
First published: October 2017
Production reference: 1231017
ISBN 978-1-78728-573-6
www.packtpub.com
Credits
Author
Boni García
Copy Editor
Charlotte Carneiro
Reviewers
Luis Fernández Muñoz
Ashok Kumar S
Project Coordinator
Ritika Manoj
Commissioning Editor
Smeet Thakkar
Proofreader
Safis Editing
Acquisition Editor
Nigel Fernandes
Indexer
Aishwarya Gangawane
ContentDevelopmentEditor
Mohammed Yusuf Imaratwale
Graphics
Jason Monteiro
Technical Editor
Ralph Rosario
Production Coordinator
Shraddha Falebhai
About the Author
Boni García has a PhD degree on information and communications technology from TechnicalUniversity of Madrid (UPM) in Spain since 2011. His doctoral dissertation was focused on test automation for web applications. He is an author of more than 30 research papers, including international conferences, journals, and magazines.
At the time of writing this book, he was working as a researcher at King Juan Carlos University (URJC), and also as assistant professor at Digital Art and Technology University (U-tad) in Spain. He has participated in different research initiatives. For instance, he is member of Kurento project, where he has created a testing framework for WebRTC applications. He also participates in the coordination of ElasTest, a project aimed to create an elastic platform for end-to-end testing of different types of applications.
Boni is an active member on the free open source software community with a big emphasis on software testing and web engineering. Among others, he is the owner and main developer of the WebDriverManager and selenium-jupiterprojects (JUnit 5 extension for Selenium).
About the Reviewers
Luis Fernández Muñoz in the last 25 years he has been serving as a Professor and Researcher at Technical University of Madrid (UPM) and King Juan Carlos University (URJC), both in Spain. he has also collaborate as Consultor and Trainer in different public and private national and international institutions. His main expertise comprises all the software engineering disciplines, from coding and testing, to analysis, design, architecture, and project management, usually around C++/Java and Web/JEE technologies.
As a Consultor in private companies, he has participated in different projects as different roles, from management to development of code and tests. In the different areas he has worked, he highlight cloud computing, combat simulation and physical activity expert systems, among others.
Additional, my entrepreneur spirit took me to become in the one of co-founders of TS Company, an UPM startup with 25 employees in only four years.
He has developed my research endeavor in the Natural Computing Group, in which he made his PhD dissertation focused on Parallel Algorithms for the Application of Evolution Rules in Endomembrane P Systems. After that, he has directed some others PhD dissertations. Thanks to all of this, he has participated in more than 60 publications, in conferences, journals, and several research projects.
This versatile career has given me the opportunity of learn by reading and listening to relevant people in the field, which is one of his main hobbies. Deep reflections, discussions and building the mental structure of conceptual ideas rigorously, but his way, is his motivation challenge. Traveling, chatting, and venturing -without losing his mind- is the way he enjoy's the best in life.
Ashok Kumar S is an Android developer residing in Bangalore, India. A gadget enthusiast, he thrives on innovating and exploring the latest gizmos in the tech space. He has been developing softwares for all Google related technologies, and he also a Google certified Android developer. A strong believer in spirituality, he heavily contributes to the open source community as an e-spiritual ritual to improve his e-karma. He regularly conducts workshops about Android application development in local engineering colleges and schools. He also organizes multiple tech events at his organization and he is a regular attendee of all the conferences that happen on Android and Java related technologies in the silicon valley of India (Bengaluru). He also runs a YouTube channel, called AndroidABCD, where he discusses all aspects of Android development, including quick tutorials. Having 4 years of professional experience. Currently working with Dunst Technologies Pvt Ltd as full-time Sr. Mobile Engineer. He has extensively worked on Android Native applications ranging from Enterprise applications to commerce application. I spend most of my time in exploring brilliant architectures and libraries. I have a strong interest in code quality, testing, and automation, and all three. he is a speaker at Android conferences that happen's in Bengaluru, Apart from all of it, I am also a photographer, A storyteller.
www.PacktPub.com
For support files and downloads related to your book, please visit www.PacktPub.com.
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters and receive exclusive discounts and offers on Packt books and eBooks.
https://www.packtpub.com/mapt
Get the most in-demand software skills with Mapt. Mapt gives you full access to all Packt books and video courses, as well as industry-leading tools to help you plan your personal development and advance your career.
Why subscribe?
Fully searchable across every book published by Packt
Copy and paste, print, and bookmark content
On demand and accessible via a web browser
Customer Feedback
Thanks for purchasing this Packt book. At Packt, quality is at the heart of our editorial process. To help us improve, please leave us an honest review on this book's Amazon page at https://www.amazon.com/dp/1787285731.
If you'd like to join our team of regular reviewers, you can e-mail us at [email protected]. We award our regular reviewers with free eBooks and videos in exchange for their valuable feedback. Help us be relentless in improving our products!
Table of Contents
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Errata
Piracy
Questions
Retrospective On Software Quality And Java Testing
Software quality
Quality engineering
Requirements and specification
Quality Assurance
ISO/IEC-25000
Verification and Validation
Software defects
Static analysis
Software testing
Testing levels
Unit testing
Integration testing
System testing
Testing methods
Black-box testing
White-box testing
Non-functional testing
Testing types
Other testing approaches
Testing frameworks for the JVM
JUnit 3
Standard tests in JUnit 3
Test execution in JUnit 3
JUnit 4
Standard tests in JUnit 4
Test execution in JUnit 4
Advanced features of JUnit 4
JUnit ecosystem
Summary
What’s New In JUnit 5
Road to JUnit 5
JUnit 5 motivation
Modularity
JUnit 4 runners
JUnit 4 rules
JUnit 5 inception
JUnit 5 community
JUnit 5 architecture
Test Engine SPI
Test Launcher API
Running tests in JUnit 5
Jupiter tests with Maven
Jupiter tests with Gradle
Legacy tests with Maven
Legacy tests wih Gradle
The ConsoleLauncher
Jupiter tests in JUnit 4
IntelliJ
Eclipse
The extension model of JUnit 5
Test lifecycle
Conditional extension points
Dependency injection
Third-party extensions
Summary
JUnit 5 Standard Tests
Test lifecycle
Test instance lifecycle
Skipping tests
Display names
Assertions
Jupiter assertions
Group of assertions
Asserting exceptions
Asserting timeouts
Third-party assertion libraries
Tagging and filtering tests
Filtering tests with Maven
Maven regular support
Filtering tests with Gradle
Meta-annotations
Conditional test execution
Assumptions
Nested tests
Repeated tests
Migration from JUnit 4 to JUnit 5
Rule support in Jupiter
Summary
Simplifying Testing With Advanced JUnit Features
Dependency injection
TestInfoParameterResolver
RepetitionInfoParameterResolver
TestReporterParameterResolver
Dynamic tests
Test interfaces
Test templates
Parameterized tests
@ValueSource
@EnumSource
@MethodSource
@CsvSource and @CsvFileSource
@ArgumentsSource
Argument conversion
Implicit conversion
Explicit conversion
Custom names
Java 9
JUnit 5 and Java 9 compatibility
Beyond JUnit 5.0
Summary
Integration Of JUnit 5 With External Frameworks
Mockito
Mockito in a nutshell
JUnit 5 extension for Mockito
Spring
Spring in a nutshell
Spring modules
Introduction to Spring Test
Testing Spring Boot applications
JUnit 5 extension for Spring
Selenium
Selenium in a nutshell
JUnit 5 extension for Selenium
Cucumber
Cucumber in a nutshell
JUnit 5 extension for Cucumber
Docker
Docker in a nutshell
JUnit 5 extension for Docker
Android
Android in a nutshell
Gradle plugin for JUnit 5 in Android projects
REST
REST in a nutshell
Using REST test libraries with Jupiter
Summary
From Requirements To Test Cases
The importance of requirements
Test planning
Test design
Equivalence partitioning
Boundary analysis
Test coverage
Software testing principles
The psychology of testing
Test anti-patterns
Code smells
Summary
Testing Management
Software development processes
Continuous Integration
Jenkins
Travis CI
Test reporting
Maven Surefire Report
Allure
Defect-tracking systems
Static analysis
Putting all pieces together
Features and requirements
Design
Tests
Summary
Preface
Humans are not perfect thinkers. At the time of this writing, software engineers are human beings. Most of them. For that reason, writing high-quality, useful software is a really difficult task. As we will discover in this book, software testing is one of the most important activities carried out by software engineers (that is, developers, programmers, or testers) to warranty a level of quality and confidence in a given piece of software.
JUnit is the most used testing framework for the Java language, and one of the most remarkable in software engineering in general. Nowadays, JUnit is much more than a unit testing framework for Java. As we will discover, it can be used to implement different types of tests (such as unit, integration, end-to-end, or acceptance tests) using different strategies (such as black-box or white-box).
On September 10, 2017, the JUnit team released JUnit 5.0.0. This book is mainly focused on this new major release of JUnit. As we will discover, JUnit 5 has supposed a complete redesign of the JUnit framework, improving important features, such as modularization (JUnit 5 architecture is completely modular), composability (the extension model of JUnit 5 allows to integrate third-party frameworks in the JUnit 5 test lifecycle is an easy way), and compatibility (JUnit 5 supports the execution of JUnit 3 and 4 legacy tests in the brand-new JUnit Platform). All of it, following a modern programming model based on Java 8 and compliant with Java 9.
Software engineering involves a multidisciplinary body of knowledge with a strong impetus for the change. This book provides a comprehensive review of many different aspects related to software testing from, mainly following an open source point of view (JUnit is open source from its inception). In this book, in addition to JUnit, you learn how to use third-party frameworks and technologies in our development process, namely, Spring, Mockito, Selenium, Appium, Cucumber, Docker, Android, REST services, Hamcrest, Allure, Jenkins, Travis CI, Codecov, or SonarCube, among others.
What this book covers
Chapter 1, Retrospective On Software Quality And Java Testing, provides a detailed review of software quality and testing. The objective of this chapter is to clarify the terminology of this domain in an intelligible way. Moreover, this chapter provides a summary the history of JUnit (version 3 and 4) and also some JUnit enhancers (for example, libraries that can be used to extend JUnit).
Chapter 2, What’s New In JUnit 5, first introduces the motivation to create a version 5 of JUnit. Then, this chapter describes the main components of the JUnit 5 architecture, namely, Platform, Jupiter, and Vintage. Next, we discover how to run JUnit tests, for example, using different build tools such as Maven or Gradle. Finally, this chapter is the extension model of JUnit 5, which allows extending the core functionality of JUnit 5 by any third party.
Chapter 3, JUnit 5 Standard Tests, gives a detailed description of the basic features of the new JUnit 5 programming model. This programming model, together with the extension model, is called Jupiter. In this chapter, you learn about the basic test lifecycle, assertions, tagging and filtering tests, conditional test execution, nested and repeated tests, and finally how to migrate from JUnit 4.
Chapter 4, Simplifying Testing With Advanced JUnit Features, provide a detailed description of the JUnit 5 features, such as dependency injection, dynamic tests, test interfaces, test templates, parameterized tests, compatibility with Java 9, and planned features for the for JUnit 5.1 (not released yet at the time of this writing).
Chapter 5, Integration Of JUnit 5 With External Frameworks, talks about the integration of JUnit 5 with existing third-party software. This integration can be done in different ways. Typically, the Jupiter extension model should be used to interact with external frameworks. This is the case of Mockito (a popular mock framework), Spring (a Java framework aimed to created enterprise applications based on dependency injection), Docker (a container platform technology), or Selenium (test framework for web applications). In addition, developers can reuse to Jupiter test lifecycle to interact with other technologies, for example, Android or REST services.
Chapter 6, From Requirements To Test Cases, provides a set of best practices aimed to help a software tester to write meaningful test cases. Considering the requirements as the basis of software testing, this chapter provides a comprehensive guide for coding tests avoiding typical mistakes (anti-patterns and code smell).
Chapter 7, Testing Management, is the final chapter of the book, and its objective is to guide the reader to understand how software testing activities are managed in a living software project. To that aim, this chapter reviews concepts such as Continuous Integration (CI), build servers (Jenkins, Travis), test reporting, or defect tracking systems. To conclude the book, a complete example application together with different types of tests (unit, integration, and end-to-end) is presented.
What you need for this book
In order to understand the concepts presented in this book better, it is highly recommended to fork the GitHub repository, which contains the code examples presented in this book (https://github.com/bonigarcia/mastering-junit5). In the author’s opinion, touching and playing with the code is essential to achieve a quick hands-on understanding of the JUnit 5 testing framework. As introduced before, the last chapter of this book provides a complete application example covering some of the most important topics of this book. This application (called Rate my cat!) is also available on GitHub, in the repository https://github.com/bonigarcia/rate-my-cat.
In order to run these example, you will need JDK 8 or higher. You can download the Oracle JDK from its website: http://www.oracle.com/technetwork/java/javase/downloads/index.html. In addition, it is highly recommended to use an Integrated Development Environment (IDE) to ease the development and testing process. As we will discover in this book, at the time of this writing there are two IDEs fully compliant with JUnit 5, namely:
Eclipse 4.7+ (Oxygen):
https://eclipse.org/ide/
.
IntelliJ IDEA 2016.2+:
https://www.jetbrains.com/idea/
.
If you prefer to run JUnit 5 from the command line, two possible build tools can be used:
Maven:
https://maven.apache.org/
Gradle:
https://gradle.org/
Who this book is for
This book is targeted for Java software engineers. For that reason, this piece of literature tries to speak the same language than the reader (that is, Java) and therefore it is driven by working code examples available on the aforementioned public open source GitHub repositories.
Conventions
In this book, you will find a number of text styles that distinguish between different kinds of information. Here are some examples of these styles and an explanation of their meaning. Code words in text, database table names, folder names, filenames, file extensions, path names, dummy URLs, user input, and Twitter handles are shown as follows: “The @AfterAll and @BeforeAll methods are executed only once”.
A block of code is set as follows:
Any command-line input or output is written as follows:
New terms and important words are shown in bold like this: “Compatibility is the degree to which a product, system or component can exchange information with other products”.
Reader feedback
Feedback from our readers is always welcome. Let us know what you think about this book-what you liked or disliked. Reader feedback is important for us as it helps us develop titles that you will really get the most out of.
To send us general feedback, simply email [email protected], and mention the book's title in the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide at www.packtpub.com/authors.
Customer support
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
Downloading the example code
You can download the example code files for this book from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files emailed directly to you.
You can download the code files by following these steps:
Log in or register to our website using your email address and password.
Hover the mouse pointer on the
SUPPORT
tab at the top.
Click on
Code Downloads & Errata
.
Enter the name of the book in the
Search
box.
Select the book for which you're looking to download the code files.
Choose from the drop-down menu where you purchased this book from.
Click on
Code Download
.
Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of:
WinRAR / 7-Zip for Windows
Zipeg / iZip / UnRarX for Mac
7-Zip / PeaZip for Linux
The code bundle for the book is also hosted on GitHub at https://github.com/bonigarcia/mastering-junit5. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Errata
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books-maybe a mistake in the text or the code-we would be grateful if you could report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the Errata Submission Form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded to our website or added to any list of existing errata under the Errata section of that title.
To view the previously submitted errata, go to https://www.packtpub.com/books/content/support and enter the name of the book in the search field. The required information will appear under the Errata section.
Piracy
Piracy of copyrighted material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works in any form on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at [email protected] with a link to the suspected pirated material.
We appreciate your help in protecting our authors and our ability to bring you valuable content.
Questions
If you have a problem with any aspect of this book, you can contact us at [email protected], and we will do our best to address the problem.
Retrospective On Software Quality And Java Testing
The well-known testing framework JUnit has come a long way since its inception in 1995. On September 10, 2017, an important milestone in the project life cycle took place, i.e. the release of JUnit 5.0.0. Before going deep into the details of JUnit 5, it is worth reviewing the status quo of software testing, in order to understand from where we have come, and where we are going. To that aim, this chapter provides a high-level review of the background of software quality, software testing, and testing for Java. Concretely, the chapter is composed of three sections:
Software quality
: The first section reviews the status quo in quality engineering: Quality assurance,
ISO/IEC-2500,
V
erification & Validation
(
V&V
), and software defects (
bugs
).
Software testing
: This is the most commonly performed activity to guarantee software quality and reduce the number of software defects. This section provides a theoretical background of software testing levels (unit, integration, system, and acceptance), methods (black-box, white-box, and non-functional), automated and manual software testing.
Testing frameworks for the
Java Virtual Machine
(
JVM
): This section provides a summary of the main features of the legacy versions of the JUnit framework (that is, versions 3 and 4). Finally, a brief description of alternative testing frameworks and enhancers to JUnit is depicted.
Software quality
Software is the collection of computer programs, related data, and associated documentation developed for a particular customer or for a general market. It is an essential part of the modern world, and it has become pervasive in telecommunications, utilities, commerce, culture, entertainment, and so on. The question What is software quality? can generate different answers, depending on the involved practitioner's role in a software system. There are two main groups of people involved in a software product or service:
Consumers
: are people who use software. In this group, we can differentiate between
customers
(that is, people responsible for the acquisition of software products or services) and
users
(that is, people who use the software products or services for various purposes).
Nevertheless, the dual roles of customers and users are quite common.
Producers
: are people involved with the development, management, maintenance, marketing, and service of software products.
The quality expectations of consumers are that a software system performs useful functions as specified. For software producers, the fundamental quality question is fulfilling their contractual obligations by producing software products that conform to the Service Level Agreement (SLA). The definition of software quality by the well-known software engineer RogerPressman comprises both points of view:
Quality engineering
Quality engineering (also known as quality management) is a process that evaluates, assesses, and improves the quality of software. There are three major groups of activities in the quality engineering process:
Quality planning
: This stage establishes the overall quality goal by managing customer's expectations under the project cost and budgetary constraints. This quality plan also includes the strategy,
that is,
the selection of activities to perform and the appropriate quality measurements to provide feedback and assessment.
Quality Assurance (QA)
: This guarantees that software products and processes in the project life cycle meet their specified requirements by planning and performing a set of activities to provide adequate confidence that quality is being built into the software. The main QA activity is
V
erification & Validation
, but there are others, such as software quality metrics, the use of quality standards, configuration management, documentation management, or an expert's opinion.
Post-QA
: These stage includes activities for quality quantification and improvement measurement, analysis, feedback, and follow-up activities. The aim of these activities is to provide quantitative assessment of product quality and identification of improvement opportunities.
These phases are represented in the following chart:
Requirements and specification
Requirements are a key topic in the quality engineering domain. A requirement is a statement identifying a capability, physical characteristic, or quality factor that bounds a product or process need for which a solution will be pursued. The requirement development (also known as requirements engineering) is the process of producing and analyzing customer, product, and product-component requirements. The set of procedures that support the development of requirements, including planning, traceability, impact analysis, change management, and so on, is known as requirements management. There are two kinds of software requirements:
Functional requirements
are actions that the product must do to be useful to its users. They arise from the work that stakeholders need to do. Almost any action such as, inspecting, publishing, or most other active verbs can be a functional requirement.
Non-functional requirements
are properties, or qualities, that the product must have. For example, they can describe properties
such
as performance, usability, or security. They are often called
quality attributes
.
Another important topic strongly linked with the requirements is the specification, which is a document that specifies in a complete, precise, verifiable manner, the requirements, design, behavior, or other characteristics of a system, and often the procedures for determining whether these provisions have been satisfied.
Quality Assurance
Quality Assurance (QA) is primarily concerned with defining or selecting standards that should be applied to the software development process or software product. Daniel Galin, the author of the book Software Quality Assurance (2004) defined QA as:
The QA process selects the V&V activities, tools, and methods to support the selected quality standards. V&V is a set of activities carried out with the main objective of withholding products from shipment if they do not qualify. In contrast, QA is meant to minimize the costs of quality by introducing a variety of activities throughout the development and maintenance process in order to prevent the causes of errors, detect them, and correct them in the early stages of development. As a result, QA substantially reduces the rates of non-qualifying products. All in all, V&V activities are only a part of the total range of QA activities.
ISO/IEC-25000
Various quality standards have been proposed to accommodate these different quality views and expectations. The standard ISO/IEC-9126 was one of the most influential in the software engineering community. Nevertheless, researchers and practitioners detectedseveral problems and weaknesses in this standard. For that reason, the ISO/IEC-9126 international standard is superseded by the ISO/IEC-25000 series of international standards on Software product Quality Requirements and Evaluation (SQuaRE).This section provides a high-level overview of this standard.
The ISO/IEC-2500 quality reference model distinguishes different views on software quality:
Internal quality
: This concerns the properties of the system, that can be measured without executing it.
External quality
: This concerns the properties of the system, that can be observed during its execution.
Quality in use
: This concerns the properties experienced by its consumer during operation and maintenance of the system.
Ideally, the development (process quality) influences the internal quality; then, the internal quality determines the external quality. Finally, external quality determines quality in use. This chain is depicted in the following picture:
The quality model of ISO/IEC-25000 divides the product quality model (that is, the internal and external attributes) into eight top-level quality features: functional suitability,performance efficiency, compatibility, usability, reliability, security, maintainability, and portability. The following definitions have been extracted directly from the standard:
Functional suitability:
This represents the degree to which a product or system provides functions that meet stated and implied needs when used under specified conditions.
Performance efficiency:
This represents the performance relative to the amount of resources used under stated conditions.
Compatibility:
This is the degree to which a product, system or component can exchange information with other products, systems or components, and/or perform its required functions, while sharing the same hardware or software environment.
Usability:
This is the degree to which a product or system can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use.
Reliability:
This is the degree to which a system, product, or component performs specified functions under specified conditions for a specified period of time.
Security:
This is the degree to which a product or system protects information and data so that persons or other products or systems have the degree of data access appropriate to their types and levels of authorization
Maintainability:
This represents the degree of effectiveness and efficiency with which a product or system can be modified to improve it, correct it, or adapt it to changes in environment and in requirements
Portability:
This is the degree of effectiveness and efficiency with which a system, product, or component can be transferred from one hardware, software, or other operational or usage environment to another
On the other hand, the attributes of quality in use can be categorized into the following five characteristics:
Effectiveness:
This is the accuracy and completeness with which users achieve specified goals.
Efficiency:
These are the resources expended in relation to the accuracy and completeness with which users achieve goals.
Satisfaction:
This is the degree to which user needs are satisfied when a product or system is used in a specified context of use.
Freedom from risk:
This is the degree to which a product or system mitigates the potential risk to economic status, human life, health, or the environment.
Context coverage:
This is the degree to which a product or system can be used with effectiveness, efficiency, freedom from risk, and satisfaction in both specified contexts of use and in contexts beyond those initially explicitly identified.
Verification and Validation
Verification and Validation -also known as Software Quality Control- is concerned with evaluating that the software being developed meets its specifications and delivers the functionality expected by the consumers. These checking processes start as soon as requirements become available, and continue through all stages of the development process. Verification is different to validation, although they are often confused.
The distinguished professor of computer science Barry Boehm expressed the difference between them back in 1979:
Verification
:
are we building the product right?
The aim of verification is to check that the software meets its stated functional and non-functional requirements (that is, the specification).
Validation
:
are we building the right product?
The aim of validation is to ensure that the software meets consumer's expectations. It is a more general process than verification, due to the fact that specifications do not always reflect the real wishes or needs of consumers.
V&V activities include a wide array of QA activities. Although software testing plays an extremely important role in V&V, other activities are also necessary. Within the V&V process, two big groups of techniques of system checking and analysis may be used:
Software testing
: This is the most commonly performed activity within QA. Given a piece of code, software testing (or simply testing) consists of observing a sample of executions (test cases), and giving a verdict on them. Hence, testing is an execution-based QA activity, so a prerequisite is the existence of the implemented software units, components, or system to be tested. For this reason, it is sometimes called dynamic analysis.
Static analysis
: This is a form of V&V that does not require execution of the software. Static analysis works on a source representation of the software: either a model of the specification of design or the source or the program. Perhaps, the most commonly used are inspections and reviews, where a specification, design, or program is checked by a group of people. Additional static analysis techniques may be used, such as automated software analysis (the source code of a program is checked for patterns that are known to be potentially erroneous).
It should be noted that there is a strong divergence of opinion about what types of testing constitute validation or verification. Some authors believe that all testing is verification and that validation is conducted when requirements are reviewed and approved. Other authors view unit and integration testing as verification and higher-order testing (for example, system or user testing) as validation. To solve this divergence, V&V can be treated as a single topic rather than as two separate topics.
Software defects
Key to the correctness aspect of V&V is the concept of software defects. The term defect (also known as bug) refers to a generic software problem. The IEEE Standard 610.12 propose the following taxonomy related to software defects:
Error
: A human action that produces an incorrect result. Errors can be classified into two categories:
Syntax error (program statement that violates one or more rules of the language in which it is written).
Logic error (incorrect data fields, out-of-range terms, or invalid combinations).
Fault
: The manifestation of an error in the software system is known as a fault. For example, an incorrect step, process, or data definition.
Failure
: The inability of the software system to perform its required functions is known as (system) failure.
In addition to this level of granularity for defects, it is also interesting to contemplate incidences as symptoms associated with a failure perceived by the software consumer. All in all, error, faults, failures, and incidences are different aspects of software defects. A causal relation exists between these four aspects of defects. Errors may cause faults to be injected into the software, and faults may cause failures when the software is executed. Finally, incidences happen when failures are experienced by the final user or costumer. Different QA activities can be carried out to try to minimize the number of defects within a software system. As defined by Jeff Tian in his book Software Quality Engineering (2005), the alternatives can be grouped into the following three generic categories:
