Monitor Quality Status

Daniel Freedman

Quality status is the sum of what you know and what you don't know about the number of bugs left in the product. The tasks associated with managing quality status are

set quality goals
present quality reports
bug list management

Quality Reports

Reports on the current quality of the product should include

what product functionality has not yet been received
% test code complete
% tests failing
code coverage
critical product failures
progress towards the quality goal

From this and historical data you can attempt to extrapolate the number of unknown problems. For a new product, this will be very hard to do. For a well understood product, it is possible to make some educated guesses.

The nature of the test suite affects the quality reports. Equally, expectations of the quality reports should affect the test suite. An automated test suite should enhance reporting ability.

% of tests automated is proportional to:

accuracy of status reports

% of tests automated is inversely proportional to:

time to prepare status reports
unknown-bug latency time
verification time for resolved bugs
unverified-fix latency time

Frequency of quality reports should be determined by the project manager. Status is often presented at weekly project meetings. It may also be desirable to post reports to a readily accessible area, like a web page.

Bug List Management

Some one on the project team must be responsible for driving the bug list. This is usually the technical lead, but can often be the test lead. When a test lead is driving the bug list, their role should be to facilitate resolution of all bugs. In this context, resolution may include postponing some bugs to later releases. If the quality goals for the current release are clear, there should be few issues categorizing bugs.

Home

last update 2/16/98