Design and Implementation of a Test Framework E-Book

Rüdiger Rowold

• Herausgeber: GRIN Verlag
• Sprache: Englisch

Veröffentlichungsjahr: 2012

Beschreibung

Diploma Thesis from the year 2006 in the subject Business economics - Information Management, grade: 1,4, Baden-Wuerttemberg Cooperative State University (DHBW), language: English, abstract: Agilent Technologies is one of the leading manufacturers of test and measurement equipment. At the moment the High Speed Digital Test division is developing a new Pulse- and Pattern generator. In order to improve the quality of the firmware development process, the firmware should be automatically tested. This will help find errors faster and identify side effects of any changes in the source code. For this, a test framework that can carry out these automatic tests was developed. The test cases are defined in XML to make the processing easy. The results are also saved to XML. This allows both human and machine readability and to transform the result to a HTML formatted report.

Leseprobe

Table of Content

1 Introduction

1.1 Agilent Technologies - Company Structure

1.2 Project Environment

1.3 Task Description

1.4 Strategic Approach

1.5 Structure of this Thesis

1.6 Time Schedule

2 Basic Concepts

2.1 Test Devices

2.2 GPIB

2.3 Programming Language

2.3.1 SCPI

2.4 Driver Library

2.5 XML

2.5.1 Basic Components of an XML Document

2.5.2 XML Schema

2.5.3 XPath

2.5.4 XML Transformation

2.6 Gnuplot

3 Implementation

3.1 Input Formats

3.1.1 The Test Suite XML File

3.1.2 The Test XML File

3.2 Perl Component

3.2.1 GetResourceString

3.2.2 GetOutputDir

3.2.3 GetTestSuiteName

3.2.4 GetTestName

3.2.5 SetParameter

3.2.6 GetParameter

3.2.7 Error

3.2.8 WriteLink

3.2.9 WriteText

3.2.10 Log

3.2.11 ShouldAbort

3.2.12 Wait

3.3 The Code

3.3.1 Windows Application

3.3.2 Class Library

3.3.3 Setup Project

3.4 Creation of the HTML test report

4 Validation

5 Usage of the GUI Tool

5.1 Setup

5.2 The GUI

5.3 Demo Test Suite

6 Future Prospects

A XML Element Overview

B Code Listings

C Glossary

D Listings

Page 1

Design and Implementation of a Test Framework

Duration:3 months

Page 3

Abstract

Agilent Technologies is one of the leading manufacturers of test and measurement equipment. At the moment the High Speed Digital Test division is developing a new Pulse-and Pattern generator. In order to improve the quality of the firmware development process, the firmware should be automatically tested. This will help find errors faster and identify side effects of any changes in the source code.

For this, a test framework that can carry out these automatic tests was developed. The test cases are defined in XML to make the processing easy. The results are also saved to XML. This allows both human and machine readability and to transform the result to a HTML formatted report.

Zusammenfassung

Agilent ist ein f¨ uhrender Hersteller f¨ ur Test- und Messger¨ ate. Im Moment entwickelt die Abteilung High Speed Digital Test einen neuen Puls- und Patterngenerator. Um die Qualit¨ at des Entwicklungsprozess der Firmware dieses Ger¨ ates zu verbessern, soll diese automatisch getestet werden k¨ onnen. Dadurch k¨ onnen Fehler fr¨ uher gefunden und Nebeneffekte von Code¨ anderungen erkannt werden.

Zu diesem Zweck wurde ein Test Framework entwickelt, das diese automatischen Test ausf¨ uhren kann. Um die Definition und Verarbeitung der Test zu vereinheitlichen wurden diese in XML spezifiziert. Die Ergebnisse werden ebenfalls in XML gespeichert, was die Weiterverarbeitung zu einem menschenlesbaren HTML Test Report vereinfacht.

Page 4

Acknowledgements

I would like to thank all people who contributed to this project.

In particular I want to thank my tutors Thomas Burger and Herbert Osterrieder for the support and supervision of this project.

My special thanks goes to Anit Kurtz for her personal support during my time at Agilent. In addition I would like to thank all people who review parts of this report, in particular Dani, Sophie Gaussiran-Racine, Sebastian Mezger and Ines Haas.

Page 8

1 Introduction

1.1 Agilent Technologies - Company Structure

Agilent Technologies is a leading manufacturer of Test and Measurement equipment, Communication and Electronic Solutions as well as Chemical and Biological Analysis tools. The company is a spin off of the Hewlett-Packard Corporation. It was first listed on the New York stock market in November 1999. At the moment there are more than 19.000 employees working for Agilent in over 40 countries with customers in more than 110 countries. In the financial year 2005 Agilent produced a turnover of 5.1 billion US-Dollar [ATD05].

Page 9

The Agilent Technologies Deutschland GmbH has about 1400 employees. In Germany all Agilent divisions are represented. The Life Sciences and Chemical Analysis (LSCA) sector, which is located at the Waldbronn site, provides complete solutions for the pharmaceutical industry, especially in the sectors of health research, development of new drugs and research for new active ingredients. They also work on the quality assurance during the manufacturing process of drugs.

At the B¨ oblingen, site digital test equipment and modules are developed. The Research and Development (R&D), as well as the production of several Agilent products, are also located in B¨ oblingen.

The Network and Digital Solutions Business Unit (NDS) is part of the Electronic Measurement Group (EMG). The EMG is one of the main groups the company comprises. One of the departments of NDS is the Digital Verification Solution Division (DVS), which has its headquarters in B¨ oblingen. DVS develops measurement equipment for high speed digital applications. Its product range includes Digital Communication Analyzers (DCA), Bit Error Rate Testers (BERT) as well as Pulse and Pattern generators (PPG).

The main part of DVS is called High Speed Digital Test (HSDT) and develops the High Frequency Patten Generators and Analyzers. In this part of Agilent the project described in this thesis took place. [ADVS]

1.2 Project Environment

HSDT is developing a new Pulse and Pattern generator at the moment. The internal project name is Pulsar. The firmware running on the Pulsar is developed in parallel with the hardware. Since the firmware is a very complex piece of software developed by a team of several engineers, it is an important task to verify that the software is working as expected. Experience has shown that the cost of fixing an error is smaller the earlier the error is found. A change in one part of the firmware could also have a side-effect on other parts.

Therefore the demand for a tool that can carry out some automated tests has occur.

Page 10

1.3 Task Description

The task of this diploma thesis is to design and implement a test framework that allows the software engineers to automatically test the firmware of the Pulse Generator. The tests will not be defined in the actual code but in a number of extra XML files. This ensures the reusability of the test framework. This way, for a new product new test cases can be defined and there is no need to adapt the software.

The tests are carried out through the programming interfaces of the instruments. The available interfaces are LAN (Local Area Network), USB (Universal Serial Bus) and GPIB (General Purpose Interface Bus). Testing the instruments through several interfaces ensures that the I/O (Input/Output) paths are all working correctly. There are several different test types required, simple query/response tests as well as execution time measurements. In addition, the possibility to sweep over one or more parameters is needed to determine the allowed value range of certain parameters. The result will be saved in XML files so that it can be easily processed and transformed into another format like HTML.

For the implementation, C# will be used since it supports XML very well and allows to simply create a comprehensive graphical user interface (GUI). In addition to the test framework, some example tests will be created to demonstrate the usage of the test tool.

1.4 Strategic Approach

In the first phase, the objective of this project was discussed with the project sponsor Thomas Burger, the main point being what the test framework should be able to do. After the theoretical fundamentals and knowledge were defined, a first prototype of the test framework was developed. From the experience gained by the development and usage of this prototype, the project scope and deliverables where defined in more detail. During the whole development phase of the final test framework there was a close cooperation with Thomas Burger to ensure that the software met the requirements of the

Page 11

firmware developers.

The software was tested with different test cases and also by different users to ensure the usability.

1.5 Structure of this Thesis

The first part of this thesis explains the basic concepts of the technologies used in this project. This includes the Agilent instruments as well as the software and programming languages used.

Later on, the actual implementation is described. The format of the test definition files is laid out in detail and the code of the application is explained. Chapter 4 covers the validation process of the project. The instructions in chapter 5 explain the usage of the test framework. At the end of this thesis some future prospects are given to illustrate the parts where there is still room for further development.

1.6 Time Schedule

The following figure shows the initial schedule for this project. Not everything was completed on the exact day planned. But the major milestones were completed as planned so the project stayed as scheduled most of the time.