Homework 5: Running a testing program

You have been hired by the software quality assurance division of Megabucks Systems, Incorporated. MSI management expects you to run a small team, you will have three people on your staff, two who are expert programmers and one who is a crackerjack records management expert. But before you begin and the stock options start rolling in, you are expected to lay out very clearly and thoroughly what the plan of the testing will be. This homework is to prepare the plan, a report that convinces management to let you go ahead and run the testing as the software evolves.

The testing plan will include:

• A specifications based outline of the testing plan, in the style put forth by Tamres. With the outline, a discussion of how and why the outline points were chosen as they were. This may also include some points considered but rejected, and the reasons why.

• Plan and illustrate a scheme for organizing and documenting the tests and their status. Use tables as appropriate.

• At least one system integration test, fully implemented. With the test, give a few paragraphs explaining the features tested and test strategies. Note: whenever possible, tests should be self contained, simply returning a true or false value. They should not require a human to scan and interpret extensive output to determine if the test is passed or failed.

• At least one unit test, fully implemented. With the test, give a few paragraphs explaining the features tested and test strategies. (baseline, boundaries, stress, etc.) Note, the first requirement is to identify a "unit". Note: whenever possible, tests should be self contained, simply returning a true or false value. They should not require a human to scan and interpret extensive output to determine if the test is passed or failed.

• Report the finding of at least one bug in specification or implementation. Although this is a proposal, a "white paper" for the testing plan, it is important to convince management by examples the kinds of things the testing is going to accomplish.

• The scheme for organizing and documenting tests mentioned above should be as thorough as you can make it. Now also include a discussion of the ways you may reduce the number of test cases in the face of limited time and budget while maintaining as robust a testing program as possible.

You may choose the software development project for which to design and run the testing.

One MSI project is an implementation of STL, the C++ standard container library. The design should be organized appropriately for the whole library, but the outline, tables and tests detailed may address only a portion. Certainly major component types such as containers, iterators, and algorithms should be tested. You may restrict attention to certain containers (say associative containers), certain iterators, 3 or 4 selected algorithms.

Another project is LinBox, a library for solving linear systems of equations over finite fields. The design should be organized appropriately for the whole library, but the outline, tables and tests detailed may address only a portion. Certainly major component types such as fields, blackbox matrices, and algorithms should be tested. You may restrict attention to certain fields(say modular, and NTL based), certain blackboxes (say sparse, dense, diagonal), and selected algorithms.

You may propose to design the tests for other software. The project to be tested must be reasonably extensive and have fairly well defined specifications. Clear the choice with me before committing to the project.

It is allowable to work in pairs on this project. Pairs are expected to produce a more extensive and thorough testing plan than individuals. Clear the pair/project combination with me in advance.