Test Execution

If the product test suite is highly automated, then test execution should be a single command which is part of the nightly build. This requires not only a mature test suite, but a reasonably mature product. There is a trade off between the frequency of test runs and analysis time required to digest the results. Analysis can be a heavy burden when large numbers of tests are failing. In the best case, planning for test execution is primarily a question of run time resources.

If the test suite is not highly automated, then humans must perform test procedures by hand. This is bad for several reasons:

Planning Test Execution

In this discussion, planning for test execution is treated as a question of run time resources.

Development Checkpoints

Test Environments

Developing test code is time consuming. If tests can be written such that a single piece of test code can run in multiple environments, then the coding investment can be substantially leveraged.

What constitutes and environment depends on the product. The implementation of environments in tests is often through environment variables passed to the test program. For example, when testing a compiler tool like Purify, the environment includes

The cross product of these environment variables makes one test program execute as dozens of tests. There are, of course, pros and cons to this:




Copyright 1998 Anne Powell

last update 2/16/98