Since it not possible to evaluate a test for the “all of the time” criteria it is necessary to turn to statistics to provide a measurement for the goodness of a test. And, since test is a process a good statistic to use is the capability index  (Cp) and the centering capability index (Cpk). A good explanation of the definitions of Cp and Cpk can be found at the QIMacros website, http://www.qimacros.com/qiwizard/process-capability.html

For an complete evaluation of the test it would be necessary to test a number of different boards on different test systems. However, executing the test on even one Unit Under Test on one tester will reveal a good amount of information about the test. For instance, is the Cpk less than 1? In that case there will definitely be problems. Typical industry demands are for a  Cpk  of 2 or over.

No matter what, if the tests are not evaluated and the results recorded trouble looms ahead. Having the evaluation proves that at least one moment in time the test was well behaved.

- Rick Wagner

 The beginning of the test specification would then list all of the inputs and outputs on a document and apply the performance specification. Each time an input and output is referenced, the document is updated with the reference number of the performance specification. When the performance specification is exhausted, the engineer should be able to see any inputs or outputs that were not referenced. Typically the types inputs or outputs that are missed are those that use parallel pins on a connector, or in the case of software, excess information.

 Other sources of test specifications include requirements that are not explicitly  part of the performance specification of the UUT, but need to be included in the test. An example of this type of requirement may be safety or regulatory requirements.

-Rick Wagner