Software Testing using Visual Studio 2010 - N Satheesh Kumar - E-Book

Software Testing using Visual Studio 2010 E-Book

N Satheesh Kumar

0,0
23,50 €

-100%
Sammeln Sie Punkte in unserem Gutscheinprogramm und kaufen Sie E-Books und Hörbücher mit bis zu 100% Rabatt.

Mehr erfahren.
Beschreibung

In Detail

When testing your applications on the Microsoft platform, Visual Studio 2010 offers you a range of powerful tools for different types of testing. This Microsoft product makes the testing process easier and faster.

A concise guide that will lead you through the amazing range of features offered by Visual Studio 2010 to test your software applications before going live. It is packed with simple but interesting examples to make you comfortable with Visual Studio 2010 testing environment. The book introduces you to the main types of testing available in Visual Studio for both desktop and web applications, and then walks you through deploying, running, and interpreting the results of tests.

Visual Studio 2010 is the new version of Microsoft's software development product, and includes lots of new features for developing and testing software applications. In collaboration with Team Foundation Server, it provides supporting tools for the management of the entire application life cycle, including development and testing.

This book is more focused on testing features and the supporting tools provided by Visual Studio 2010 for testing the applications.

The book begins by explaining different types of tests as part of the software development cycle, and then dives deep into providing an overview of each type of application testing using Visual Studio 2010 features. Along the way you will also learn in detail about creating and maintaining the test cases and associating the test cases with requirements using the Test Manager Tool.

Each chapter in the book concentrates on explaining each test type using the features and tools provided by Visual Studio 2010. You will be guided in using these features with real world examples with step by step explanations.

Get to grips with the amazing range of features offered by the Visual Studio for testing your software applications before going live.

Approach

This book starts with basic understanding of different types of tests. It then goes about explaining several examples with a step-by-step approach to master concepts and the features needed to help the reader understand testing clearly.

Who this book is for

If you are a developer, a software tester, or an architect who wishes to master the amazing range of features offered by the Visual Studio 2010 for testing your software applications before going live - then this book is for you.

This book assumes that you have a basic knowledge of testing software applications and have good work experience of using Visual Studio IDE.

Sie lesen das E-Book in den Legimi-Apps auf:

Android
iOS
von Legimi
zertifizierten E-Readern

Seitenzahl: 402

Veröffentlichungsjahr: 2010

Bewertungen
0,0
0
0
0
0
0
Mehr Informationen
Mehr Informationen
Legimi prüft nicht, ob Rezensionen von Nutzern stammen, die den betreffenden Titel tatsächlich gekauft oder gelesen/gehört haben. Wir entfernen aber gefälschte Rezensionen.



Table of Contents

Software Testing using Visual Studio 2010
Credits
About the Authors
About the Reviewers
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Errata
Piracy
Questions
1. Visual Studio 2010 Test Types
Software testing in Visual Studio 2010
Testing as part of the Software Development Life Cycle
Types of testing
Unit test
Manual test
Web Performance Tests
Coded UI test
Load Test
Ordered test
Generic test
Test management in VS 2010
Team Explorer
Testing tools introduction
Test View
Test List Editor
Test Results
Code coverage results
Overview of files used for test settings and testing in Visual Studio 2010
Microsoft Test Manager
Connecting to Team Project
Test Plans, Suites and Test Cases
Define test cases
Lab Center
Summary
2. Test Plan, Test Suite, and Manual Testing
Test Plan
Test Suite and Test Suite Types
Requirement-based Test Suite
Query-based Test Suite
Static Test Suite
Run Manual Tests
Action Recording
Shared steps and action recording for shared steps
Creating shared steps
Action recording for Shared steps
Add Parameters to Manual Tests
Summary
3. Automated Tests
Coded UI Test from Action Recording
Files generated for the Coded UI Test
CodedUITest1.cs
UIMap.Designer.cs
UIMap.cs
UIMap.uitest
Data Driven Coded UI Test
Adding controls and validation to the Coded UI Test
Summary
4. Unit Testing
Creating unit tests
Naming settings
General settings
Generated unit test code
Assert statements
Types of asserts
Assert
Assert.AreEqual
Assert.AreNotEqual
Assert.AreSame
Assert.AreNotSame
Assert.Fail
Assert.Inconclusive
Assert.IsTrue
Assert.IsFalse
Assert.IsNull
Assert.IsNotNull
Assert.IsInstanceOfType
StringAsserts
StringAssert.Contains
StringAssert.Matches
StringAssert.DoesNotMatch
StringAssert.StartsWith
StringAssert.EndsWith
CollectionAssert
CollectionAssert.AllItemsAreNotNull
CollectionAssert.AreEquivalent
CollectionAssert.AreNotEquivalent
CollectionAssert.AllItemsAreInstancesOfType
CollectionAssert.IsSubsetOf
CollectionAssert.IsNotSubsetOf
CollectionAssert.AllItemsAreUnique
CollectionAssert.Contains
CollectionAssert.DoesNotContain
CollectionAssert.AreEqual
CollectionAssert.AreNotEqual
AssertFailedException
UnitTestAssertionException
ExpectedExceptionAttribute
Unit tests and generics
Data-driven unit testing
Unit testing an ASP.NET application
Unit testing web services
Code coverage unit test
Summary
5. Web Performance Testing
Creating Web Performance Test
Recording a test
Adding comments
Cleaning the Recorded Tests
Copying the requests
Web Performance Test editor
Web Test Properties
Web Performance Test request properties
Other request properties
Form POST parameters
QueryString parameters
Extraction rules
Validation rules
Transactions
Toolbar properties
Add data source
Set credentials
Add recording
Parameterize web server
Context Parameters
Add web test plug-in
Add Request Plug-in
Performance session for the test
Debug/Running Web Performance Test
Settings in the .testsettings file
General
Web Performance Test
Roles
Test Timeouts
Deployment
Hosts
Setup and Cleanup scripts
Running the test
Web Browser
Request
Response
Context
Details
Summary
6. Advanced Web Testing
Dynamic parameters in Web Performance Testing
Coded Web Performance Test
Creating a coded test from a recorded test
Transactions in coded test
Custom code
Adding comment
Running the Coded Web Performance Test
Debugging the Coded Web Performance Test
Custom rules
Extraction rule
Validation rules
Summary
7. Load Testing
Creating Load Test
Load Test Wizard
Specifying a scenario
Think time
Load pattern
Constant load
Step load
Test mix model and test mix
Based on the total number of tests
Based on the number of virtual users
Based on user pace
Based on sequential test order
Network mix
Browser mix
Counter sets
Run settings
Threshold rules
Editing Load Tests
Adding context parameters
Results store
Running the Load Test
Working with the test result and analyzing test results
Graphical view
Summary view
Table view
Details view
Exporting to Excel
Using the test controller and test agents
Test controller configuration
Summary
8. Ordered and Generic Tests
Ordered tests
Creating an ordered test
Properties of an ordered test
Executing an ordered test
Generic tests
Creating a Generic Test
Summary results file
Properties of a generic test
Summary
9. Managing and Configuring a Test
Managing tests using Test Lists
Organizing the Test Lists
Test view
Enabling/Disabling tests
Toolbar options
Filtering tests
Group by
Configuring tests
Test settings
General
Roles
Data and Diagnostics
Deployment
Hosts
Setup and Cleanup scripts
Test Timeouts
Web Performance Test
Unit Test
Editing the test run configuration file
Editing the deployment section
Test properties
Summary
10. Command Line
MSTest utility
Running a test from the command line
/testcontainer
/testmetadata
/test
/unique
/noisolation
/testsettings
/resultsfile
/noresults
/nologo
/detail
Publishing test results
/publish
/publishbuild
/flavor
/platform
/publishresultsfile
Trace files
Publishing
Step 1: Create/use existing test project
Step 2: Running the test
Step 3: Create build
Step 4: Build project
Step 5: Publish result
Summary
11. Working with Test Results
Test Results
Test as part of Team Foundation Server Build
Build Report and Test Result
Creating Work Item from the result
Publish Test Results
Summary
12. Reporting
Team Foundation Server reports for testing
Bug status Report
Status on all iterations
Other out of box reports
Creating client report definition using Visual Studio 2010
SQL Server Business Intelligence Project
Analysis Services Project
Report server project
Create new Report using Microsoft Excel
Summary
13. Test and Lab Center
Connect to Team Project
Testing Center
Testing Center — Plan
Testing Center — Test
Test Center — Track
Test Center — Organize
Lab Center
Virtual Environments
Deployed Environments
Stored Templates
Stored Virtual Machines
Environments
Creating Environment from stored Virtual machines and Templates
Creating Environments from stored environments
Composed Environments
Summary
Index

Software Testing using Visual Studio 2010

Software Testing using Visual Studio 2010

Copyright © 2010 Packt Publishing

All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews.

Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book.

Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information.

First published: November 2010

Production Reference: 1241110

Published by Packt Publishing Ltd.

32 Lincoln Road

Olton

Birmingham, B27 6PA, UK.

ISBN 978-1-849681-40-7

www.packtpub.com

Cover Image by David Guettirrez (<[email protected]>)

Credits

Authors

Subashni S

Satheesh Kumar N

Reviewers

YiChun Chen

Adam Gallant

Development Editor

Rukhsana Khambatta

Technical Editors

Erika Fernandes

Arani Roy

Indexer

Monica Ajmera Mehta

Editorial Team Leader

Gagandeep Singh

Project Team Leader

Lata Basantani

Project Coordinator

Vishal Bodwani

Proofreader

Kevin McGowan

Graphics

Geetanjali Sawant

Production Coordinators

Alwin Roy

Arvindkumar Gupta

Cover Work

Alwin Roy

About the Authors

Subashni S, has a Bachelor’s Degree in Computer Science Engineering and around twelve years of experience in software development and testing life cycle, project, and program management. She is a certified Project Management Professional (PMP) and Certified Software Test Manager (CSTM). She started her career as DBA in Oracle technology and later developed many software applications using Borland software products for a multinational company based in Chennai, India, and then moved to Bangalore. She is presently working for a multinational company in the area of managing development and testing projects. She is also the co-author of the book, Software Testing using Microsoft Visual Studio 2008, by Packt Publishing.

I would like to thank my husband for helping me in co-authoring and supporting me in completing this book. I would also like to thank my other family members and friends for their continuous support in my career and success.

Satheesh Kumar N, has a Bachelor’s Degree in Computer Science Engineering and has around fourteen years of experience in software development life cycle, project, and program management. He started his career developing software applications using Borland software products in a company based in India and then moved to the United Arab Emirates and continued developing applications using Borland Delphi and customizing Great Plain Dynamics (now known as Microsoft Dynamics). He moved back to India and spent three years designing and developing application software using Microsoft products for a top multinational company in India and then spent couple of years in Project Management and Program Management activities.

Now he works as a Technical Architect in Bangalore for a top retail company, based in the United States. He works with the latest Microsoft technologies and has published many articles on LINQ and other features of .NET. He is also the author of the books, LINQ Quickly and Software Testing using Microsoft Visual Studio 2008, by Packt Publishing.

I would like to thank my wife for helping me in co-authoring and supporting me in completing this book. I would also like to thank my other family members and friends for their continuous support in my career and success.

About the Reviewers

YiChun Chen is one of the, five star contributors at MSDN VSTS, TFS forums, as well as Visual Studio forums, and is an enthusiastic TFS expert.

He is familiar with VB.NET, C# development, and is interested in new techniques of VSTS and TFS from Microsoft. He is well versed in the use of databases, including SQL Server and Oracle.

He has worked for Microsoft for about two years as a Senior Support Engineer on MSDN forums to help customers worldwide to solve technical issues on Visual Studio, Team Foundation Sever, and .NET Framework.

Adam Gallant is a Developer Tools Technology specialist with Microsoft Canada, focusing on Visual Studio 2010 Application Lifecycle Management. In his role, Adam works with Microsoft's partners and customers across Canada, helping them understand how to implement the Microsoft ALM platform, in order to deliver high-quality solutions more effectively. Adam has been with Microsoft in various roles focusing on software development for the past 16 years.

Adam can be reached through e-mail at <[email protected]>.

Preface

The Microsoft Visual Studio 2010 suite contains several features that support the needs of developers, testers, managers, and architects to simplify the development process. Visual Studio provides different editions of the products such as Professional, Premium, and Ultimate with different sets of tools. Visual Studio 2010 is tightly integrated with Team Foundation Server 2010, which is the central repository system that provides version control, process guidance and templates, automated build, automated test, bug tracking, work item tracking, reporting, and supporting of the Lab Center and Test Center.

This book helps developers to get familiarized with the Visual Studio tools and techniques to create automated unit tests, using automated user interface testing, code analysis, and profiling to find out the performance and quality of the code. Testers benefit from learning more about the usage of Test Center and Lab Center, which are very new tools in Visual Studio 2010. This books also covers different types of testing such as Web Performance Test, Load Test, Executing the Manual Test cases, and recording the user actions, rerunning the tests using the recording, Test case execution, and capturing the defects and integrating the requirements, test cases, test results, and defects together is also covered in detail. Testers also get a high level overview on using Lab Center for creating virtual environments for testing multiple users and multiple location scenarios.

Visual Studio provides user interface tools such as Test List Editor, Test View, Test Results, Test Configuration user interfaces, Test Center, and Lab Center to easily manage and maintain multiple test cases, and Test Results in integration with Team Foundation Server. This book provides detailed information on all of the tools used for testing the application during the development and testing phase of the project life cycle.

What this book covers

Chapter 1, Visual Studio 2010 Test Types, provides an overview of different types of testing which help with testing software applications throughout different phases of software development. This chapter also introduces the tools and techniques in Visual Studio 2010 for different testing types.

Chapter 2, Test Plan, Test Suite, and Manual Testing, explains the steps involved in creating the Test Plan, Test cases, and Test Suite used for manual testing. Creating the manual test by recording the user action and running the test with data inputs is also covered as part of this chapter.

Chapter 3, Automated Tests, provides the step-by-step approach to create a Coded UI test from the user action recording. It also explains the steps to provide data to the coded UI test and custom rules for the test.

Chapter 4, Unit Testing, explains the detailed steps involved in creating the unit test classes and methods for the code. It also explains different types of assert methods and parameters for testing the code. Passing a data source with a set of data and testing the code is also explained in detail.

Chapter 5, Web Performance Testing, explains the basic method of web testing using VSTS and features such as adding rules and parameterization of dynamic variables. Microsoft Visual Studio 2010 ultimately provides many new features for the Web Performance Testing such as adding new APIs to the test results, web performance test results in a separate file, looping and branching, new validation and extraction rules, and many more.

Chapter 6, Advanced Web Testing, generates the code for the testing scenario explained in Chapter 5 using the Generate Code option. This is very useful for customizing tests using the code.

Chapter 7, Load Testing, can simulate the number of users, network bandwidth, a combination of different web browsers, and different configurations. In the case of web applications it is always necessary to test the application with different sets of users and browsers to simulate multiple requests to the server. This chapter explains the steps involved in simulating the real world scenario and testing the application.

Chapter 8, Ordered and Generic Tests, explains both test types, the ones used for testing the existing third party tool or programs which can also be run using the command line, in detail. Visual Studio 2010 provides a feature called ordered test to group all or some of these tests and then execute the tests in the same order. The main advantage of creating the ordered test is to execute multiple tests in an order based on the dependencies. Generic tests are just like any other tests except that they are used for testing the existing third-party tool or program which can also be run using the command line.

Chapter 9, Managing and Configuring a Test, provides different tools that support easy ways of managing and grouping the tests. Using these tools, we can enable and disable the tests or select them to run, filter the tests from the list of all the tests created for the project, and set the properties for the individual test. The Test List Editor is the main interface provided by Visual Studio for managing all tests under the solution. This chapter explains the details of the test settings file and the tools used for managing the tests.

Chapter 10, Command Line, explains the command line tool, MSTest, used for running the test with different options and then collecting the output and publishing it to the Team Project.

Chapter 11, Working with Test Results, helps us to verify whether the test methods return the expected results but also to analyze the application quality and to verify the build. We can add the test as part of the Team Foundation Server automated build so that we can verify the build and make sure the latest code is checked in to the source control, and is working as expected. This chapter explains the process of running the tests and publishing the test results to the Team project.

Chapter 12, Reporting, explains the details of accessing the publishing and reporting test results in a specific format. Accessing different types of testing reports and creating new test reports are also explained in this chapter

Chapter 13, Test and Lab Center, is useful for creating the Test Plans and adding Test Cases to the plans. We can also associate the requirements to the test plans. The Lab Center helps us to create and configure different virtual/physical environments for the test runs, test settings such as defining the roles and configuring the data and diagnostics information for the selected roles, configuring the Test Controllers required for the test, and configure the test library to store the environment information.

What you need for this book

This book requires a basic knowledge of any of the versions of Visual Studio and Team Foundation Server. The reader must be familiar with Visual Studio IDE and integration with Team Foundation Server and have a basic knowledge of C#. To generate coded tests and customize the code, the testers should have a basic knowledge of C#. The following tools are required in order to try the samples in all of the chapters of this book:

Visual Studio 2010 UltimateSQL Server 2008 Team Foundation Server 2010Microsoft Office (Microsoft Word and Microsoft Excel)SQL Server Analysis and Reporting Services (for customizing reports)

Who this book is for

If you are a developer, a software tester, or an architect who wishes to master the amazing range of features offered by Visual Studio 2010 for testing your software applications before going live—then this book is for you.

This book assumes that you have a basic knowledge of testing software applications and have good work experience of using Visual Studio IDE.

Reader feedback

Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.

To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.

If there is a book that you need and would like to see us publish, please send us a note in the SUGGEST A TITLE form on www.packtpub.com or e-mail <[email protected]>.

If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.

Customer support

Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.

Tip

Downloading the example code for this book

You can download the example code files for all Packt books you have purchased from your account at http://www.PacktPub.com. If you purchased this book elsewhere, you can visit http://www.PacktPub.com/support and register to have the files e-mailed directly to you.

Errata

Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/support, selecting your book, clicking on the erratasubmissionform link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.

Piracy

Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.

Please contact us at <[email protected]> with a link to the suspected pirated material.

We appreciate your help in protecting our authors, and our ability to bring you valuable content.

Questions

You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it.

Chapter 1. Visual Studio 2010 Test Types

Software testing is one of the most important phases of the Software Development Life Cycle (SDLC). The delivery of the end product is based on better design, better coding, better testing, and meeting the requirements. The quality of the product is measured by testing the product based on functional and non-functional requirements with the help of testing tools and techniques. The tools are useful in simulating a real life situation and the user load. For example, testing a web application with more than 1000 user load is a very time consuming and tedious task if we do it manually. But the performance testing tool that comes along with Visual Studio 2010 can simulate this scenario and test it in a very short period of time. Visual Studio 2010 provides additional tools for testing all type of applications and scenarios such as Unit testing, Load Testing, Web testing, Ordered testing, and generic testing.

This chapter provides a high level overview on all the testing tools and techniques supported by Visual Studio 2010. This chapter covers the following topics:

Testing as part of the software development life cycleTypes of TestingTest Management in Visual Studio 2010Testing Tools in Visual Studio 2010

Software testing in Visual Studio 2010

Before getting into the details of the actual testing using Visual Studio 2010 let us find out the different tools provided by Visual Studio 2010 and their usage and then we can execute the actual tests. Visual Studio 2010 provides different tools for testing and management such as the Test List Editor and the Test View. The test projects and the actual test files are maintained in Team Foundation Server (TFS) for managing the version control of the source and the history of changes. Using Test List Editor we can group similar tests, create any number of Test Lists, and add or delete tests from a Test List.

The other aspect of this chapter is to see the different file types generated in Visual Studio during testing. Most of these files are in XML format, which are created automatically whenever a new test is created.

For the new learners of Visual Studio, there is a brief overview on each one of those windows as we are going to deal with these windows throughout all or most of the chapters in this book. While we go through the windows and their purposes, we can check the Integrated Development Environment (IDE) and the tools integration into Visual Studio 2010.

Testing as part of the Software Development Life Cycle

The main objective of testing is to find the defects early in the SDLC. If the defect is found early, then the cost will be lower than when the defect is found during the production or implementation stage. Moreover, testing is carried out to assure the quality and reliability of the software. In order to find the defect as soon as possible, the testing activities should start early, that is in the Requirement phase of SDLC and continue till the end of the SDLC.

In the Coding phase various testing activities take place. Based on the design, the developers start coding the modules. Static and dynamic testing is carried out by the developers. Code reviews and code walkthroughs are conducted by the team.

Once the coding is complete, then comes the Validation phase, where different phases or forms of testing are performed:

Unit Testing: This is the first stage of testing in the SDLC. This is performed by the developer to check whether the developed code meets the stated functionality. If there are any defects found during this testing then the defect is logged against the code and the developer fixes it.

The code is retested and then moved to the testers after confirming the code without any defects for the purpose of functionality. This phase may identify a lot of code defects which reduces the cost and time involved in testing the application by testers, fixing the code, and retesting the fixed code.

Integration Testing: This type of testing is carried out between two or more modules or functions together with the intention of finding interface defects between them. This testing is completed as a part of unit or functional testing and, sometimes, becomes its own standalone test phase. On a larger scale, integration testing can involve putting together groups of modules and functions with the goal of completing and verifying that the system meets the system requirements. Defects found are logged and later fixed by the developers. There are different ways of integration testing such as top-down and bottom-up.
The Top-Down approach is intended to test the highest level of components and integrate first to test the high level logic and the flow. The low level components are tested later.The Bottom-Upapproach is exactly opposite to the top-down approach. In this case the low level functionalities are tested and integrated first and then the high level functionalities are tested. The disadvantage of this approach is that the high level or the most complex functionalities are tested later.The Umbrella approach uses both the top-down and bottom-up patterns. The inputs for functions are integrated in the bottom-up approach and then the outputs for functions are integrated in the top-down approach.
System Testing: This type of testing compares the system specifications against the actual system. The system test design is derived from the system design documents and is used in this phase. Sometimes system testing is automated using testing tools. Once all the modules are integrated, several errors may arise. Testing done at this stage is called system testing. Defects found in this type of testing are logged by the testers and fixed by the developers.Regression Testing: This type of testing is carried out in all the phases of the testing life cycle, once the defects logged by the testers are fixed by the developers or if any new functionality changes due to the defects logged. The main objective of this type of testing is testing with the intention of determining if bug fixes have been successful and have not created any new defects. Also, this type of testing is done to ensure that no degradation of baseline functionality has occurred and to check if any new functionality that was introduced in the software caused prior bugs to resurface.

Types of testing

Visual Studio provides a range of testing types and tools for testing software applications. The following are some of those types:

Unit testManual testWeb Performance TestCoded UI TestLoad TestGeneric testOrdered test

In addition to these types there are additional tools provided to manage, order the listing, and execution of tests created in Visual Studio. Some of these are the Test View, Test List Editor, and Test Results window. We will look at the details of these testing tools and the supporting tools for managing testing in Visual Studio 2010.

Unit test

Unit testing is one of the earliest phases of testing the application. In this phase the developers have to make sure the code is producing the expected result as per the stated functionality. It is extremely important to run unit tests to catch defects in the early stage of the software development cycle. The main goal of unit testing is to isolate each piece of the code or individual functionality and test if the method is returning the expected result for different sets of parameter values.

A unit test is a functional class method test which calls a method with the appropriate parameters, exercises it, and compares the results with the expected outcome to ensure the correctness of the implemented code. Visual Studio 2010 has great support for unit testing through the integrated automated unit test framework, which enables the team to write and run unit tests.

Visual Studio has the functionality to automatically generate unit test classes and methods during the implementation of the class. Visual Studio generates the test methods or the base code for the test methods but it remains the responsibility of the developer or the team to modify the generated test methods and to include the code for actual testing. The generated unit testing code will contain several attributes to identify the Test Class, Test Method, and Test Project. These attributes are assigned when the unit test code is generated from the original source code. Here is a sample of the generated unit test code.

A Unit test is used by developers to identify functionality change and code defects. We can run the unit test any number of times and make sure the code delivers the expected functionality and is not affected by new code change or defect fix.

All the methods and classes generated for the automated unit testing are inherited from the namespace Microsoft.VisualStudio.TestTools.UnitTesting.

Manual test

Manual testing is the oldest and the simplest type of testing but yet very crucial for software testing. The tester would be writing the test cases based on the functional and non-functional requirements and then testing the application based on each test case written. It helps us to validate whether the application meets various standards defined for effective and efficient accessibility and usage.

Manual testing comes to play in the following scenarios:

There is not enough budget for automationThe tests are more complicated or too difficult to convert into automated testsNot enough time to automate the testsAutomated tests would be time consuming to create and run

The tested code hasn't stabilized sufficiently for cost effective automation.

We can create manual tests by using Visual Studio 2010 very easily. The most important step in a Manual test is to document all the required test steps for the scenario with supporting information, which could be in a separate file. Once all the test cases are created, we should add the test cases to the Test Plan to be able to run the test and gather the test result every time we run the test. The new Microsoft Test Manager tool helps us when adding or editing the test cases to the Test Plan. The following are additional Manual testing features that are supported by Visual Studio 2010:

Running the Manual test multiple times with different data by adding parametersCreate multiple test cases using an existing test case to get the base test case first and then customize or modify the testSharing test steps between multiple test cases Remove test cases from the test if not required Adding or copying test steps from Microsoft Excel or Microsoft Word or any other supported tool

There are a lot of other manual testing features that are supported in Visual Studio 2010. We will see those features explained in Chapter 2, Test Plan, Test Suite, and Manual Testing.

Web Performance Tests

Web Performance Tests are used for testing the functionality and performance of the web page, web application, web site, web services, and combination of all of these. Web Performance Tests can be created by recording the HTTP requests and events during user interaction with the web application. The recording also captures the web page redirects, validations, view state information, authentication, and all the other activities. All these are possible by manually building the web tests using the Web Performance Test editor but this takes more time and is more complex. Visual Studio 2010 provides the automated Web test feature which takes care of capturing all HTTP request and response events while recording the user interaction and generating the test.

There are different validation rules and extraction rules used in Web Performance Testing. Validation rules are used for validating the form field names, texts, and tags in the requested web page. We can validate the results or values against the expected result as per the business needs. These validation rules are also used for checking the processing time taken for the HTTP request.

Extraction rules in Web Performance Tests are used for collecting the data from the web pages during requests and responses. Collecting this data helps us in testing the functionality and expected result from the response.

Providing enough data for the test methods is very important for the success of automated testing. Similarly, for Web Performance Tests we need to have a data source from which the data would be populated to the test methods and the web pages would be tested. The data source could be a database, a spreadsheet, or an XML data source. There is a data binding mechanism in Web Performance Tests which takes care of fetching the data from the source and provides the data to the test methods. For example, a reporting page in a web application definitely needs more data to test it successfully. This is also called the data driven web test.

Web Performance Tests can be classified into Simple Web Performance Tests and Coded Web Performance Tests. Both of these are supported by VS:

Simple Web Performance Tests generate and execute the test as per the recording with a series of valid flows of events. Once the test is started there won't be any intervention and it is not conditional. Coded Web Performance Tests are more complex but provide a lot of flexibility. These types of tests are used for conditional execution based on values. Coded web tests can be created manually or generated from the Web Performance Test recording. We can choose the language for generating the code for the Web Performance Test such as C# or VB.NET. Using the generated code, we can control the flow of test events by customizing the code. A coded Web Performance Test is a powerful and highly customizable test for web requests.

Coded UI test

The previous versions of Visual Studio provided all facilities to test application functionality and performance but there is no test type to test the UI part of the application, whether it is Web or Windows. Visual Studio 2010 has a very good new feature for the UI test to access the controls in a web page and verify the values.

The Coded UI test Test can be generated from action recordings coming from MTM or can be recorded and generated directly in Visual Studio. A Coded UI test generates a UIMap object that represents the controls, windows, and assertions. Using these objects and methods we can perform actions to automate the test. The coded UI test also creates other supporting files such as:

CodedUITest.cs: A File which contains the test class, test methods, and assertionsUIMap.uitest: The XML model for the UIMap class which contains the windows, controls, properties, methods, and assertionsUIMap.Designer.cs: Contains the code for the UIMap.uitest XML fileUIMap.cs: All customization code for the UI Map would go into this file

The following screenshot shows the coded UI test with the default files created for the test:

Load Test

Load Testing is a method of testing used to identify the performance of critical situations and the maximum workload the application can handle. In the case of a desktop or a standalone application the user load is predictable and it is easy to tune the performance, but in the case of a multi-user application or a web application it is necessary to determine the application behavior under normal and peak load conditions.

Visual Studio 2010 provides a Load Test wizard which helps in creating the Load Test for different scenarios. These are the parameters set using the Load Test wizard:

Load Test pattern defines how the user will run the app at any point Test mixes define which tests will be used for testingBrowser Mix and Network Mix define the possible browser and the network configuration related to the actual Counter Sets define which performance counters are collected from the load test agents and the systemRun settings define the test run such as how long the test should be run

So before we release this web site to the customers or the end users, we should check the performance of the application so that it can support the mass end user group. This is where Load Testing is very useful to test the application along with the Web test or Unit test.

The Load Test is always driven by the collection of Web and Unit tests. The Web Performance Test is added to a Load Test to simulate multiple users opening simultaneous connections to the same web application and making multiple HTTP requests. The Load Test starts with a minimum number of virtual users and then gradually increases the user count to check the performance at the peak user load.

The Load Test can be used to test the performance of the service components to find out the servicing capacity of the component for the client requests. One good example would be to test the data access service component that calls stored procedure in the backend database and returns the results to the client application. Unit tests would be used as part of a Load Test in these cases.

With VS 2010 we have the option of capturing individual tests and saving the entire test result of individual test runs within the Load Test such as a failed Web Performance Test or a failed unit test. This is very helpful when debugging the Load Tests. Results of different tests can be saved in a repository to compare the set of results and improve performance.

VS also provides the Load Test Analyzer, which provides the summary and details of test runs from the Load Test result.

We will see more about setting the Load Test properties, working with tests, and analyzing the Load Test results later in this book in Chapter 7, Load Testing.

Ordered test

There are situations where the execution of a test is dependent on another test. In order to execute the dependent test successfully, the parent test should be executed first. Ordered tests merely consist of setting the order for all the tests in an ordered Test List. Sometimes there would be a test B, which depends on the result produced by test A in the list of tests executed. If we don't set the order that test A should be run before running test B then the entire test would fail.VS 2010 provides the feature to set the execution or running order for the tests through the Test View window. We can list all available tests in the Test View and choose the tests in an order using different options provided by Visual Studio and then run the tests. Visual studio will take care of running the tests in the order we have chosen in the list.

Once we are able to run the tests successfully in an order, we also expect the same ordering in getting the results. Visual Studio provides the results of all the tests in a single row in the Test Results window. Actually, this single row result will contain the results of all the tests run in the order. We can just double-click the single row result to get the details of each of the tests run in the ordered test.

An ordered test is the best way to control tests and run tests in an order.

Generic test

We have seen different types and ways of testing applications but there are situations where we might end up having other third party applications for testing, which were not developed using Visual Studio. We might have only the executable or binaries for the application and may not have the supported testing tool for those applications. This is where we need the generic testing method, which is just a way of testing third party applications using Visual Studio.

Generic tests are used to wrap existing tests. Once the wrapping is done then it is just another test in VS.

Using Visual Studio we can collect the test results and gather the code coverage data too. We can manage and run the generic tests in Visual Studio just like others tests. In fact, the test result output can be published to the Team Foundation server to link it with the code build used for testing.

Test management in VS 2010

Visual Studio 2010 has great testing features and management tools for testing. These features are greatly improved from previous versions of VS. The Test Impact View is the new test management tool added to the existing tools such as Test View, Test List Editor, Test Results, Code coverage Results, and Test Runs from the main IDE.

Team Explorer

Team Explorer is the application tier that is integrated into Visual Studio to connect the source control tool, Team Foundation Server. Using this explorer, the developer can browse through the Team Projects in the TFS. TFS is the central source control repository for managing and controlling Team Projects. From Team Explorer IDE, we can connect to the Team Foundation Server, and the Team Project collection, and Team Project. After connecting to the server, we can use Team Explorer to browse through the details of the Team Project.

Team Explorer provides features to work with Team Project details such as:

Accessing project process guidance documents that explain the process to be followed for the Team Project such as work items, reports, queries, and work products.Managing work items such as tasks, defects, issues, and requirements.Importing and exporting work items.Accessing the project files and source code.Adding, removing, configuring, and managing Team Projects.Creating, managing, and automating project builds. Managing favorites which are the shortcuts to the favorite node of the Team Project. Managing reports. It could be defect reporting or any type of work items reporting.

Testing tools introduction

Visual Studio 2010 provides many tools to create, run, debug, and view results of tests. The following is an overview of the tools and windows provided by Visual Studio to view test details and test output details. We will look into more details of the actual usage of these tools later, in Chapter 9, Managing and Configuring a Test.

Let us see how we can create a new test project using Visual Studio 2010 and then we will test a sample project to get to know the tools and features.

Open Visual Studio 2010 and open the solution. We will not go into the details of the sample application AddNumbers now but we will just create a test project for the sample application and see the features of the tools and windows. The application referred to throughout this chapter is a very simple application for adding two numbers and showing the result.

In the same way as adding the projects and code files to the solution, we have to create the test project and test files, and add the test project to the solution. There are different ways of creating test projects under the solution:

Select the solution and add a project using the Add | New Project.... Then select the project type as Test from the list of project types under the language which we are going to use. Then select the template as Test Project.Select the solution and then from the top toolbar menu option select the option Test | New Test.... This will list all the test type templates to offer the type of template we need for the testing. You can see the option which is a dropdown saying Add to Test Project with the options as list of <projectnames>, Create a new Visual C# test project…, Create a new Visual Basic test project..., and three more options.

For our sample testing application, we will select the second option mentioned previously and choose the simple Unit Test type.

The Add New Test dialog contains nine different template options for creating tests. Unit testing templates consist of four types, one is a basic unit test, the second one is Database unit test, the third for the general unit test, and the fourth is for using Unit Test Wizard. The following are the file extensions for each of the VS test types shown in the preceding image:

.vb or .cs is for all types of unit tests and coded UI tests.generictest is for the Generic test type.loadtest is for a test which is of type Load Test.webtest is for Web Performance Testing

After selecting the test type, give a name for the test in the Test Name field. Select the Create new Visual C# test Project… in the Add to Test Project option. Now assign a name for the new test project. Let us name it AddNumbersTestProject.

Now we can see that there are two files created, <ProjectName>.vsmdi, which is the metadata file, and the Local.testsettings file created under the solution items for the test projects. The metadata file is an XML file which contains all the information about the tests list. Let us add a couple more tests to this test project to run different tests using the tools available.

Now let us see the different windows and tools that support all the tests that we can execute using Visual Studio. We will walk through the tools by using the sample unit test.

Test View

Visual Studio 2010 provides the Test View window to view the list of all tests available in the current solution. Test View provides the option to change the column display, grouping of tests based on test properties, search for specific tests from the list, and select and run the test from the list as well. The test view window is available under the Test | Windows menu option in Visual Studio IDE. The following screenshot shows a sample of the Test View window which displays the list of all tests created in the current solution:

The Test View window has its own toolbar for the multiple operations that can be performed on the list. The leftmost option in the toolbar is the run and debug option with which we can start and debug the selected tests in the list.

The second and third options in the toolbar are for Filtering the list by choosing the column from the dropdown and entering matching text in the keyword area. When you submit filter conditions using the button to the right, the list gets filtered and only the test matching the condition is shown. For example, the following screenshot shows the list of tests filtered with Test Type as Unit Test: