Requirements testing and management for a software product

Data tells us that nearly half of the world’s software projects exceed their allocated time and budget. Frequent reasons for this are ‘scope creep’ and the need to rework the solution because of various issues that surface.

At the same time, an overwhelming majority of software defects – 56% (!) – are usually introduced during the requirements phase (see figure 1).

The later these defects are detected, the more expensive they are to fix: it is 40 to 70 times more expensive to fix an error found during integration/acceptance testing, than it is to fix an error found during requirements testing. So it is much cheaper to nip the defects in the bud by testing your requirements thoroughly before the development team acts on them.

Software requirements testing

First, you should make sure that the requirements are at all testable. To be deemed testable, a specification should only contain points that:

Make sense
Are readable
Are realistically feasible
Are “deterministic” (You can use them to determine the system’s expected behavior.)

Among various approaches to requirements testing, one name stands out. The Bender RBT system (the method and the related software) is one the oldest and most reputable solutions for specification testing in the software industry. Invented in the 1970’s by Richarch Bender, the process has since been beneficial to more than 500 government agencies and business corporations all over the world.

According to Bender RBT’s definition of a good requirements specification, it should be:

Correct
Complete
Unambiguous
Logically consistent

These KPIs are achieved by performing an ambiguity review, which consists in auditing the specification for the parameters I just mentioned.

Correctness is understood as the correct relationship between cause (input) and effect (output), as described in the cause-effect graph employed in the Bender RBT method.

Completeness is achieved by ensuring that the requirements cover all possible scenarios in which the system may be used.

Ambiguity is understood as ambiguous/unclear words, phrases, and constructs. According to Bender RBT, there are 15 classes of ambiguity. For example, the document may contain ambiguous adjectives (“some,” “most,” “usual,” “standard”), ambiguous adverbs (“approximately,” “normally”), or refer to ambiguous variables (“the data,” “the screen,” “the file.”) Phrases like “as required,” “on the other hand” could mean that the requirements contain implicit information (which should be made explicit), while temporal ambiguity can arise from words like “monthly,” “quickly,” or “after.”

If […] ambiguities are not dealt with in the Requirements Definition Phase, nearly 100% of them will show up as defects in the code.

From a Bender RBT white paper

Logical consistency is understood as a lack of logical errors in Bender RBT’s cause-effect graph that uses operators like “AND”, “OR” and “NOT” to demonstrate relationships between inputs and outputs in a system.

Good vs. bad requirements example

Let us say we want to create a contact form with three fields, a file attachment functionality, and an action button. Here is an example of incomplete and ambiguous requirements vs. complete and unambiguous requirements:

Incomplete and ambiguous	Complete and unambiguous
Wireframe: The mockup visualizes form fields with dummy text, the file attachment icon, and the button with text.	Wireframe: The mockup visualizes form fields with dummy text, the file attachment icon, and the button with text.
X The mockup does not visualize error messages or the “Success” message.	V Additional mockups are provided that visualize applicable error messages and the “Success” message.
Functional requirements: The specification states which fields should be required and mentions that the remaining fields are optional. It also contains dummy text to be used and specifies an email address, to which the messages should be sent.	Functional requirements: The specification states which fields should be required and how the requirement is satisfied for each field (the specification details precise validation rules and validation errors.) It also explicitly names the fields that should be optional. The document also contains dummy text to be used and specifies an email address, to which the messages will be sent.
X The specification is silent on the contents and the format of messages that go out to the specified email box after the form has been submitted.	V The specification lists data points that should be included in the resulting message and says where each data point comes from. For example, the “Sent by” line should be recorded in the following format: “Sent by: [Name_Field_Value]”, where Name_Field_Value equals the contents of the form’s “Name” field.
X Non-functional requirements: The specification does not list any technical requirements.	V Non-functional requirements: The specification states the maximum time the user can wait for the form to be submitted. It also states the order, in which validation scripts should be firing, and gives each validation algorithm a priority.

The omission of important details makes the requirements incomplete, and the use of ambiguous wording (e.g., “the remaining fields”) leaves room for varying interpretations.

Software requirements management

Once the QA specialists have tested the requirements for the above mentioned parameters (clarity, completeness, etc.), the next goal becomes designing the smallest possible number of test cases to assure that the solution meets the requirements.

Moving forward, further requirements management activities include:

Overseeing the QA process.
Proper versioning of the requirements.
Mapping detected defects to requirements and tracking them to resolution.
Preventing the specification from becoming problematic again.

In conclusion

Depending on the project and the team, various approaches to requirements testing may be used. It helps to include domain experts in the feedback loop: you can ask end-users to read through the test cases, or you can get feedback from programmers.

Arguably, any amount of requirements testing is better than none. Still, it’s important to have clearly-defined KPIs while performing this type of testing.

Need someone who can test your software requirements? Hire our testers