Alignment to ISO/IEC/IEEE standards applicable to the test domain has been a key principle of my work. This alignment does not “add” tasks to the “as is” state of the test engineering domain, but it does restructure domain artifacts (i.e., test plan, test specification, test design specification, test case specification, test procedure specification) and the sequence of some of the tasks (early engagement in setting stakeholder expectation and concurrence for system acceptance test), as well as re-instantiating tasks (describing the verification method beyond inspection, analysis, demonstration and test) frequently over-looked. Focus has been brought on a key task within the test sub-process area supporting verification of requirements at the acceptance level of test. An exemplar under development typifies production and acceptance by all stakeholders of a requirement’s acceptance criteria beginning in the proposal phase with the key requirements of the proposed system. This methodology is not a new concept, but rather a revitalization of a time proven practice. In A. Wayne Wymore’s seminal work “Model Based Systems Engineering”, Mr. Wymore emphasizes the importance of the ‘System Test Requirement’ as an element of the ‘System Design Requirement’ in his system modeling formalism. My emphasis is also consistent with guidance for Section 4 Verifications of a System Requirements Document provided by MIL-HDBK-520A and MIL-STD-961E. The recommended practice goes beyond simply applying a verification method kind to a requirement in the Verification Cross Reference Matrix (VCRM)[i]. It requires the creation of a baseline concept for the test requirement / method, principally by defining an abstract set of acceptance criteria in the form of a test objective specification (e.g., a test requirement). This forms the basis for a test design specification which is ultimately realized by a concrete test case specification.
At the system’s SRR each requirement is associated with a concept for its verification test case. A test case is ‘named’ and ‘described’ by its test objective. This strikes a test specification baseline for the test system at SRR.
The maturation stage of the test specification is at SFR. The elements required to implement a test are described for the test context owning the test case. This forms the basis for the test architecture associated to the requirement.
Another principle motivating this work is to drive the engagement in product development of the test engineering domain much earlier in the lifecycle of the system of interest than has been the typical practice, in my experience. An example of the concept is to treat work product inspections as a “Test Case” and incorporate the test case execution in the test body of evidence. This is a concept currently in use in the European software development community. The intent is to dramatically influence and thereby reduce the accumulation of technical debt in all of its forms.
Early test engineering effort of this nature are not typical, in my personal experience, but my research and experiences suggest they hold promise for a substantive ROI. Setting the test system’s technical baseline and the test architecture it is responsible for early in the project aides in setting the expectation with stakeholders. Early setting of the baseline supports managing changes in scope and offers an opportunity to incrementally refine expectations and thereby enhance the probability of a satisfied customer stakeholder at acceptance testing.
[i] The term VCRM is inconsistent with the Glossary define by ISO/IEC 29119