CSC265 - Testing 1
Ad hoc testing versus systematic testing
-
Usually testing is ad hoc:
unplanned, left until the implementation is complete,
not maintained, and mostly manual.
Consequently, it is expensive and ineffective.
-
Systematic testing is
-
Planned:
to permit design for testability.
-
Documented:
so that the tests can be understood
-
Maintained:
so that the tests can be run on every new version
-
Exploits opportunities for automation,
so that repeating the tests is very cheap
-
Can be:
inexpensive and effective
-
The KWIC testing is planned and highly automated.
Testing tasks
-
Build the test harness:
utility programs, tools, stubs, and drivers.
-
Generate the inputs:
choose inputs likely to cause failures.
-
Determine the expected outputs from the specification.
-
Execute the test cases, capturing the actual outputs.
-
Compare the actual and expected outputs:
can be very expensive; planning pays off here.
Module versus system testing
-
Problem: hard to test modules thoroughly in their production environment.
-
Cause: poor controllability and observability
Example: LineStorage
LSGetWord(lineNum,wordNum) checks that
lineNum and wordNum are in range.
In KWIC, LSGetWord is always called with parameters
that are in range.
Thus, in KWIC, the range checking code of LSGetWord cannot be tested.
-
Solution: isolate the module under test.
-
In KWIC:
5 of the 6 modules are tested completely standalone.
There is a test suite for the system as well.
All the tests run completely automatically.
Stubs and drivers
-
Suppose that module M is to be tested in isolation.
-
Suppose that M exports function F and that F's
implementation calls function F1, exported by module M1
-
To produce a test program for M:
we must write code to invoke F and
we must provide an implementation of F1.
-
A driver for module M is
a program written to invoke M's exported functions.
Usually the driver invokes M's exported functions many times,
with a wide variety of parameter values.
-
A stub is code that serves as a substitute for functions
called by M.
Usually the stub is much simpler than the code actually invoked by
M in the full system
Example: the Input module
-
Input's job is to load the lines in stdin into the LineStorage module
Input calls LSAddLine for each line in stdin.
Input calls LSAddWord for each word in each line.
-
To test Input in isolation from KWIC:
A driver is needed to invoke INInit and INLoad.
A stub is needed for LSAddLine and LSAddWord.
-
Study the following files to see how it works:
Input/driver.c
Input/LineStorage.c