Premise: A requirement expresses a precise feature set for an entity. Typically, features are framed as desired behavior (functional requirements). Features may be expressed in terms of functionality, inputs, outputs, actions, sequences, qualities and constraints. A collection of requirement statements that define an entity’s technical features is typically identified as a specification. A stakeholder requirements specification should define stakeholder needs and an intended use of the entity’s features by the stakeholder in an environment, the system’s context. This intended use is the description of how the stakeholder intends to satisfy their need by employing the entity under development to achieve a goal. This information is foundational to developing requirement acceptance criterion.
Discussion: Frequently, stakeholder requirement statements lack the precision to unambiguously produce objective evidence through test case execution that a feature complies with its specification. Requirements are frequently expressed implicitly rather than explicitly, or are poorly constructed. Implicit characteristics of a feature and poor requirement statement construction frequently results in conflict during the object’s acceptance test phase as a result of emergent requirement statement interpretation. It is not at all unusual to have stakeholders re-interpret requirement statements to obtain new features they did not originally define. One approach to clarify imprecise requirement statements is to develop explicit requirement statements from the imprecise statements and offer the improved requirements as a replacement. An alternate approach is to author acceptance criterion for implicit or poorly constructed requirements to explicitly define the criterion that support the assertion that the entity’s feature behavior satisfies its specification.
Ideally, requirements should focus on the problems requiring solutions in the stakeholder’s domain rather than focusing on the system solution. Stakeholders must be accountable for their requirement statements and agree to an acceptance criterion for each and every stakeholder requirement prior to the commencement of stakeholder requirement analysis. Acceptance criterion forms the basis for the stakeholder’s acceptance of the developed entity. This acceptance is the transfer of an entity from development to stakeholder use. Mutual agreement on precise acceptance criterion precludes discord late in the program during the entity’s acceptance test phase where the stakeholder’s satisfaction is critically important.
Assertion: Well written acceptance criterion will validate that a requirement statement is verifiable. Employ explicit acceptance criterion early in the entity’s development period and obtain stakeholder agreement with it. Agreement on acceptance criterion should be established at the stakeholder requirements review phase of a project.
Method: An approach to developing acceptance criterion is to ask questions from the user/stakeholder viewpoint. These criterions will answer the stakeholder question “How do we know the entity meets its specification?”
What measured or observed criterion will convince stakeholders that the system implementation satisfies their operational need or is there a specific operational situation where an entity’s features will provide a user a capability which accomplishes a user operational objective/goal? Is there an output provided by an entity feature that satisfies a stakeholder criterion that can be measured and subjected to a stakeholder standard?
Answering these questions provides a foundation for test case development. The first answer forms the operational domain context for a test case. A domain description defines the static structure of the population of an environment and how they interrelate. In the Department of Defense Architecture Framework V2 specification the populations of entities in an environment are referred to as “Performers”. Performer entities posses features that enable interaction with our stakeholder’s entity to accomplish a stakeholder objective and thereby satisfy a stakeholder need. How these entities are connected and how they exchange objects (e.g., information items, or flows) helps to define the acceptance criterion. What performers in the environment are interacting with the system and how are they interacting? The second answer provides both a test case scenario and the test oracle that an arbiter uses to assert if the test case has “passed” or “failed”. The test case scenario description defines the dynamic event and interactions of the entities in an environment. Acceptance criterion defines the successful outcome of a test case. Each requirement should have an acceptance criterion statement. Criterion must be measurable, either qualitatively or quantitatively. Where the measure is qualitative, it is imperative to reach agreement on defining this subjective evaluation.
Further Refinement: Acceptance criterion such as “successful” is qualitative. In this example there is a need to quantify “Success”. We can measure “profitability” by assessing the “return on investment” and stating that if each dollar investment returns a 10% profit, the standard used to assert a verdict, and then success has been achieved. Success might also be measured by the quantity of process cycles performed in a period of time.
Yes, these quantitative criterions are measures applying to non-functional requirements. However, the point is that by measuring the performance of a function, which may be a derived performance measure, there is implicit verification of the functional requirement statement.
The acceptance criterion defines the domain the entity is employed in. It describes how the entity’s features are measured or assessed. Acceptance criterion may express constraints to be imposed on the entity when its features are in use. Acceptance criterion forms the test case development framework. The acceptance criterion statements explicitly communicate the interpretation of stakeholder requirements.