What Do You Mean “Test the Requirements”?

by Jennifer Bedell

Post image for What Do You Mean “Test the Requirements”?

Guest post by Jennifer Bedell

Everyone knows it is less costly to identify defects early in the project lifecycle, but how early can we go?

Typically, we create a project plan that allows for testers to become involved before the actual testing phase.  This gives them an opportunity to review the requirements and create test cases that will be executed during the testing phase.  But, what happens when the tester discovers gaps in the requirements?  Sure, they will ask for clarification and update their tests accordingly, but is this tracked?

A major reason for project overruns is unclear or incomplete requirements. Yet, we still have difficulty justifying additional analysis time.  People want to see results early so we move too quickly into the development phase.

If we track the questions asked about the requirements the same way we track defects that are discovered with the software, we will likely find that the root cause of many ”defects” is the requirements and not the code.  But, it is the code that needs to be fixed because it was based on requirements that may not have been complete.

By reporting on the root cause of defects, we can identify the ratio of requirements defects to code defects.  This data can then be used to demonstrate the importance of allowing adequate time for analysis, formal walkthroughs, and requirements reviews.

At first, this may appear to slow down the progress, but the advantage of discovering potential defects early will decrease re-work later in the project lifecycle.

Leave a Comment

{ 5 comments… read them below or add one }

Josh March 1, 2011 at 2:38 pm

Great points Jennifer! I am currently implementing user stories as their own separate module in our DOORS requirements management tool. These will be traceable to our requirements. I think the user stories model really helps uncover some of the hidden or missing aspects of requirements….perhaps just because it’s a different approach to the topic than traditional requirements.

The user stories are intermingled with our test cases, and we are capturing acceptance criteria for each of the user stories and so that will make it easy for us to check off what our “definition of done” is when we test.

Another item that helps here is continuous integration. In my environment, we have many systems that all interface with each other, and different teams are developing the various systems. Even though I’m working in a waterfall paradigm, we do a build and dry-run of the system (and interfacing systems) every 2 weeks. These end-to-end tests help find bugs early, and they also help uncover miscommunicated or uncommunicated requirements when the output or interaction with interfaces isn’t what another group or the end user expected. In these cases, we’ll iterate our user stories and requirements as necessary.


Adriana B. March 2, 2011 at 6:13 pm

Any type of defect measurement should differentiate between defects in requirements (i.e., incomplete, ambiguous, incorrect, inconsistent, etc.) vs. defects in code (software performs unexpectedly, functionality is missing, etc.).

In order to prevent defects in requirements, it’s always a good idea to involve as many different constituents as possible in requirements reviews (not only testers, but analysts, designers, developers, IT architects, etc.) in order to help uncover omissions, errors and other problems as early in the process as possible (when it’s much cheaper to fix).


Jennifer Bedell March 2, 2011 at 8:38 pm

Josh…I love how you have integrated User Stories into a waterfall project. It sounds like the User Stories will trace up to a higher-level business requirement so you’ve incorporated the traceability. The User Stories will certainly pick up on any gaps that may exist and also help the business to see how their processes will be affected. That would also feed in to Change Management and Training areas.

Adriana…I completely agree with your comments! Reviews as you have mentioned are always helpful in getting to clear and complete requirements.


Randy Tangco March 6, 2011 at 11:28 pm

It is a very timely article. I have always indicated to any team I work with that the beginning of a good product is a good set of requirements. What does this mean? It means that the specs should be understood one way by all the project team members. If you have at least two interpretation of a requirement, you already have a defect in the making and so there is value in catching the problem early during the requirements gathering and development part of a project. I always encourage all my team mates to participate early on in the requirements phase of a project regardless whether you are a developer or a tester or an analysis. Having a clear requirement save the project time as you execute it as it eliminates re-work. I have proven this in one of the projects I had worked on when my team mates bought into the idea of building a good set of requirements from the start.


Sridhar May 18, 2011 at 1:26 pm

One of the things that we can do is to make testers part of the requirements team. This allows them to get to know the domain better and also review it from a testability perspective.

My experience is that testers bring a systems perspective early in the lifecycle and think of various things that may cause bottlenecks later (where does the data come from, how different pieces will integrate etc).

Good post and a good practice from Josh.


Previous post:

Next post: