Lab Module I -- Program Testing and Use of Assignment

1. To learn the scheme mechanisms for modification of the parts of a pair.
2. To practice the disciplined design of a suite of test cases for procedures.

Reading: SICP (Structure and Interpretation of Computer Programs), Chapter 3 through section 3.

Black box test design is the generation of test cases for a procedure based on the specification of the procedure only. By contrast "White box" testing is based on examination of the code of an implementation. In this lab we will stick to the more fundamental black box testing. The implementation of the procedure may be under development and changing frequently or an entirely new implementation may subsequently be put in place. The test suite is designed to be independent of the implementation details, yet have a high chance of revealing bugs in the implementation. Black box test case design may be based on three principles.

1. Design a test for each "path" through the specification.

2. Design tests for boundary conditions, as well as for "typical" inputs.

3. Design tests to detect aliasing errors. (Aliasing errors will be first discussed in exercise 2.)

The specification of the output has two alternatives, return false or return a sublist. A test case should be created to give each of these two kinds of result. Having both tests should help detect problems with an implementation in which one branch of a cond or if contains an error.

The list argument is of unspecified length and the returned value may be any sublist of the input list. One could consider a path through the specification for every combination of lengths of the input list and of the output. However this leads to infinitely many test cases! We are most likely to encounter bugs associated with boundary conditions such as when the input list is of length 0 or 1. Although not a boundary condition, it is useful to have a test case when the input list length is two or more. This is because errors commonly occur in setting up a recursive call (or second iteration through a loop). The test with input of length two or more is most likely to catch this type of problem.

In view of the above considerations we might offer this test suite:

;; boundary condition, length 0
(member-list 'a '()) ; expecting #f
;; boundary condition, length 1, both paths
(member-list 'a '(a)) ; expecting (a)
(member-list 'a '(b)) ; expecting #f
;; typical case, length > 1, all relations of input to output by length
(member-list 'a '(a b c)) ; expecting (a b c)
(member-list 'a '(c a b)) ; expecting (a b)
(member-list 'a '(c b a)) ; expecting (a)
(member-list 'a '(d c b)) ; expecting #f

In this case the principle of examining all paths through the specification would suggest trying objects of various types: number, symbol, string, pair. Each should be tried with 4 lists: proper list containing the object, proper but doesn't contain, improper containing, improper not containing. The use of different object types is likely to reveal any problems with the kind of equality test used in the implementation. The thorough use of tests of the exceptional condition are likely to yield cases where the error handling is not performed even though it is specified. This may lead to change of the specification rather than the implementation!

Important note: the first specification above does not say what the procedure has to do when the input list is improper. For that specification it would be actually wrong to design tests in which the input list is not a proper list. Such test cases would amount to criticism of the specification, not tests of it. There is a trade-off between the value of checking for errors in the input and the cost of doing so. The specification reveals which is preferred for the current procedure.

Homework exercises (Due at start of lab 22 April 01):

1. Exercise 1
   Write a suite of test cases for sign-match-sublist which is specified below. Base your selection of test cases on checking all paths through the specification and on checking of boundary conditions.

   Specification: (sign-match-sublist n L) takes a number n and a proper list of numbers L. It returns a list consisting of all elements of L, in order, which have the same sign as n. The sign of a number is -1 if the number is negative, 0 if the number is zero, and 1 if the number is positive.

2. Exercise 2
   When programs use assignment, procedures may be working with mutable objects (variables whose binding may change during the computation through use of set! or lists which may change structure through use of set-car! or set-cdr!). Common errors arise when different expressions reference the same mutable value and this fact is overlooked or improperly handled. For example consider this specification:
   (sign-match! L M) --> unspecified. Sign-match takes two lists of numbers as argument. It modifies the second list so that each element has the same sign as the corresponding element of the first list. After the procedure finishes, each element of M (1) is zero if the corresponding element of L is zero or if the original value in M was zero, (2) has the same absolute value as the original value in M, but the same sign as the corresponding element of L, if the element of L is nonzero. IF L is longer than M, it's tail is ignored. It is an error if M is longer than L. L is unchanged.

   Part a. Design test cases based on the first two testing principles. Note that you ignore the output value, since it is not specified. The result to be checked is the final value of M.

   Additionally, because M is modified, aliasing problems can arise if L is a sublist of M or M is a sublist of L. Changes to M will cause changes to L and it is specified that L must not be changed. Thus in some cases in which L and M share structure, the specification cannot be met. If the procedure cannot meet its specification an error or exception must be thrown, even if not discussed in the specification.

   Thus in this case, the specification only requires that L and M be proper lists of numbers, yet requires that upon completion L be unchanged and M be as required. Test cases should include valid inputs for which the output conditions cannot be met because of the aliasing problem, and test cases for which the output conditions can be met despite aliasing problems.

   Part b. Complete your test suite taking this aliasing issue into account.

3. Exercise 3. Write and test sign-match!.

4. Exercise 4. Do exercise 3.16
   Example: The following definitions create a structure c which has three pairs, but for which Ben's (count-pairs c) returns 5.

   (define a (cons 1 2))
   (define b (cons a a))
   (define c (cons a b))
   

   ---

5. Topping exercise for lab. Design a test suite for count-pairs, which returns the number of pairs in any list structure, proper or improper, with or without shared substructure, with or without loop-back references. Count-pairs returns zero if it's argument is not a pair.

   Mystery implementations of count-pairs will be provided in lab. You will run your tests to validate them or reveal their errors.

   Note: Be careful! Do not attempt to print some of the test cases you should create. The write and display functions, like Ben Bitdiddle's count-pairs, will over-display a list structure with shared subparts, and they will go into an infinite printing loop on structures with loop-back references.

   ---

   Flourish (optional - and hard - problem):

6. The builtin procedure length works correctly on proper lists, but will go into an infinite loop on lists with a loop back. For instance in this sequence length will go into an infinite loop:

   (define L '(1 2 3 4 5)) ;; -> (1 2 3 4 5)
   (define M (cddddr L)) ;; -> (5)
   (set-cdr! M (cddr L)) ;; now  L's tail forms a loop
   (length L) --> runs forever
   

   The resulting L above is a structure with 5 pairs. The car's are the numbers 1-5. Let us identify the pairs by their cars. The mutation done by the set-cdr! makes the cdr of pair 5 be pair 3. Thus when we "walk down" L we visit pairs in this order:

   1, 2, 3, 4, 5, 3, 4, 5, 3, 4, 5, 3, 4, ....

   The length procedure takes such a walk, hence never terminates. L consists of an initial segment: 1, 2. Followed by a circle: 3, 4, 5. Write a modified version of length that takes an arbitrary list structure and returns it's length if it is a list without loop-back. If there is loop-back, length returns a list of two numbers, (m n). The m is the number of pairs in the initial segment of the input. n is the number of pairs in the circle caused by the loop-back. So for the above L, (length L) --> (2 3).

   Ignore cars of pairs. Length is only about the sequence of pairs visited by following cdrs. Note that (eq? x y) returns true of two pairs x and y if and only if they are the same pair in memory. (By contrast, (equal? x y) returns true if and only if they are eq or they are different stored pairs, but their cars are equal and their cdrs are equal.)