Do testers goldplate too?
via Flickr by by Mykl Roventine
Why is it always the developers who get blamed for goldplating? Most people who have worked in a project environment know that goldplating is one of the biggest contributors to scope creep. When you consider that the cost of change increases as the project timeline progresses, it becomes evident that, in addition to increasing scope, goldplating by a developer can also be costly.
But, what if the developers are not the only contributors to goldplating? What if goldplating is also being done by the testers? Considering the cost of change at the various stages in the SDLC, it becomes clear that goldplating by a tester could be ten times more costly than if it is done by a developer. But, exactly how can a tester goldplate?
Goldplating by a tester can occur when a tester goes beyond the stated requirements in an effort to produce a “quality” product. A tester may feel that their suggestion would improve the customer experience so they log this in the defect log. While their suggestion may do exactly what they envisioned, if it was not within the scope of the stated requirements, it becomes a form of goldplating or “feature creep”. A tester’s job is to ensure that a quality product is delivered, but many testers rely on their own definition of quality rather than using the requirements to define quality.
Quality is defined as the ability to meet specified requirements as stated by the user. The key in this definition is the user. It is the user’s perception that determines whether a product meets their expectations.
To mitigate the risk of gold-plating by testers, all defects should be triaged by a member of the project team who has a clear understanding of the requirements scope. This person can determine whether each defect is truly a defect related to the current project or an enhancement suggestion for a future release.
Feb 7th, 2010 at 5:20 am
Twitter Comment
Do testers goldplate too? [link to post]
– Posted using Chat Catcher
Feb 7th, 2010 at 8:28 am
As I once said: “Scope Creep & Gold Plating are two faces of the same coin”,
It’s not about who’s plating the gold it’s about who’s approving the plating and controlling it in the first place.
Ideally a minor project change log has to be there to monitor this kind of changes that sometimes are done to keep powerful stakeholders happy!
Thanks Jennifer..
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 7th, 2010 at 2:47 pm
Excellent point Kareem, and I like Jennifer’s point about flagging great ideas for future releases too. Ideally testers get involved earlier so they can give feedback for requirements and design decisions up front rather than just during testing.
Reply
Jennifer Bedell Reply:
February 9th, 2010 at 9:53 am
Having testers involved earlier is definitely an advantage. Even if it is simply having the tester review the Requirements and the Design documents.
Agile methods like Scrum seem to recognize this.
Reply
Feb 7th, 2010 at 8:28 am
Twitter Comment
RT @pmstudent Do testers goldplate too? [link to post] #pmot
– Posted using Chat Catcher
Feb 7th, 2010 at 9:46 am
I agree that testers do gold plate but in many organizations are a great resource for enhancing the software and in particular the UI design. Unfortunately, testers are rarely included in the requirements sessions so their ideas are not heard until too late.
I realize this is a separate topic but offer it as a reason why testers gold plate and as a mechanism for project managers to both improve the quality of their products and minimize the risk of gold plating by testers.
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 7th, 2010 at 2:44 pm
That’s a great point Mark.
Testers are important stakeholders in the project too. They should be consulted sooner rather than later because they have a lot of great ideas in terms of usability and quality of the user experience.
Reply
Feb 7th, 2010 at 10:33 am
Twitter Comment
RT @kareemshaker: RT @pmstudent Do testers goldplate too? [link to post] #pmot
– Posted using Chat Catcher
Feb 7th, 2010 at 12:33 pm
Interesting post..
@Kareem: That’s true.. Its about who is approving the plating.
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 7th, 2010 at 2:48 pm
I agree. There needs to be some system in place, even if it’s a simple log.
Reply
Feb 9th, 2010 at 1:25 am
Twitter Comment
Do testers goldplate too? [link to post]
– Posted using Chat Catcher
Feb 9th, 2010 at 9:50 am
Hi everyone,
Thank you for all the comments on this post! It is a topic I have thought about for quite some time. I struggle with where to draw the line between “defect” and “out of scope” everyday, so I thought others may be experiencing it too.
The simple answer is to manage the scope and keep track of all changes. The difficulty comes in determining when a logged defect should be addressed with the current project. Here’s an example:
While introducing a new product to a web-based insurance system, we discovered several existing calculation problems. The problems existed in the system for several years and were not an issue for the business. However, expected testing results for the new product required that the calculations be correct. Regression testing revealed that the same calculations for other products were not correct. Although this was a true defect, it was out of scope for the current project. From a tester’s standpoint, it is difficult to accept that we must sign-off on something that we know contains defects. But, introducing those defects into the current project is a form of “gold-plating” (let’s fix this while we are in there) or “scope creep” (let’s add this one more fix).
The question then becomes, do we extend the project to include these fixes or do we stick to the original scope of ensuring that the new product is correct? This is best answered by the key stakeholder after being presented with the options.
We need to be sure these types of defects are not sent directly to the developer to “fix” without having them reviewed by the appropriate parties.
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 9th, 2010 at 8:18 pm
Interesting. I liked Pawel’s twitter comment about testers not being confined by project specs as a good thing.
I’d go to the question of business value. Does fixing the problem add more value to the business than the effort to carry it out?
Here’s another question. Does the sponsor always have the final say? If I knew as a project manager that a particular direction would contribute more value to the business to the organization and yet the sponsor disagreed, do I just yield?
Na, I’m too much of a rock-the-boat kind of guy. I’ve had sponsors who frankly didn’t have the best interests of the organization or department in mind. When that happens, project managers need to have the ability to stand up for what’s best for ALL stakeholders, not just the sponsor.
Reply
Feb 9th, 2010 at 10:37 am
Twitter Comment
Rt @pmstudent.Do testers goldplate too? [link to post] <–yet another reason to include testers in design phase.
– Posted using Chat Catcher
Feb 9th, 2010 at 11:34 am
Twitter Comment
I think ANYONE can be guilty of gold-plating! RT @kripssmart @kareemshaker @pmstudent Do testers goldplate too? [link to post] #pmot
– Posted using Chat Catcher
Feb 9th, 2010 at 3:47 pm
In the projects I have worked with, I have always emphasize the approved requirements. I always indicate that we should go back to requirements if there are contentions between team members.
Reply
Feb 9th, 2010 at 6:11 pm
This post is really interesting as it is opening more possible discussions.
The first one is about on flow and responsibilities.
Should the tester decide what goes in scope? Well, no. There are a lot of things that go into the balance when deciding if the benefits of a change outweigh their costs and the person/group that can decide needs a complete view of the project.
Should the tester, though, give his/hers opinions on functionality outside the strictly defined requirements? Absolutely yes. Often the suggestions coming back from testing are in the usability area, don’t add too much overhead but add a lot of value for the users.
What we did? The feedback from testing is going to a stage called “Dispatch”. The “dispatcher” (in our case the team) cleared this area once a week by assigning the proper bugs to the developers and moving other items to “New Feature”, “Next Release”, “Needs Discussion” or “Closed”.
The second one is about the role of the tester. In my opinion you’re saving a lot of hassle later if you have good “requirements validation” done by the same people who will do the testing. And don’t let them out when you plan the UI.
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 9th, 2010 at 8:20 pm
Great point about usability. That is one of those things that can be great or lousy even though the project requirements are met.
Testers can be so valuable in terms of usability….especially when they get involved early!
Reply
Feb 9th, 2010 at 11:48 pm
Sorry for the long comment but I just thought to share my experience with this issue.
I have managed projects where the testing phase got out of control. The test team kept coming up with more things to test and more reasons to defer the Go Live date. This situation can be extremely demoralizing to the entire project team.
As Project Managers, we learned to put so much time and effort in getting the requirements right, but we neglect to invest enough effort in developing a robust Test Plan.
A robust test plan puts boundaries around testing and leads to a controlled scope. In my experience, I see these 7 weaknesses in test planning as the root cause for scope creep and gold plating:
1. No written test scope or test plan
Testers want to do a great job. However, without a written clear scope, they can end up spending valuable project hours testing unimportant stuff. Many hours can be wasted arguing with the QA team or customers about whether a test is in scope or not. Make sure to specify not only what will be tested but also what will not be tested and why. Once the plan is approved, then it should be baselined and every subsequent change should go thru change management to determine impact on cost and schedule.
2. Unclear Roles & Responsibilities
Make sure it is clear from the start of the project what the tester(s) will do in the project. Sometimes testers get confused about their role. There is a big different between testing, Quality Assurance, and Quality Control. The definition of these is not always clear or consistent. With an overzealous QA manager, getting agreement on these terms and what is applicable to your project can consume hours and lead to frustration.
3. No Change management process.
Once the test plan is baselined (approved), every change to the plan should go thru change management process. This will give you the ability to show the cost of every change and let the sponsor approve or deny. Quality has a price and to approve a change or reject it is a business decision for the sponsor to make. Force the decision to be about value and cost and remove the emotions out of the debate.
4. No clear or defined acceptance criteria
With no clear or defined acceptance criteria, too many hours can be spend debating with test team about which defects are considered showstoppers and which ones can be fixed after the Go Live. Your Go Live date can be held hostage at the discretion of the test team, especially if they report to an uncompromising Test Manager.
Make sure to agree as early as possible on types of defect or issues priorities so that it is clear:
• What showstoppers must be fixed before the Go Live. Let these be priority 1.
• What can wait until post-project if we cannot resolve before the Go Live. Make these bugs priority 2.
• What can be deferred to future phase/projects or handled as part of the ongoing maintenance and support. Make these bugs priority 3.
Only priority 1 bugs can stop the project from going live. Everything else can wait until after the Go Live.
5. Missing or incomplete Requirements Traceability Matrix
Make sure each requirement is covered by a test case/scenario. Indicate the reasons why a requirement is not covered by a test, so there is no confusion later down the road. Make sure this is reviewed by the all stakeholders and signed off on, before testing starts. This way only bugs resulting from a valid test case in the plan can be resolved.
6. Testing Tools, Techniques, and Methodologies
What testing tools will be used in this project? The organization can suddenly require that all testing get performed using a new tool/platform, after you already baselined your schedule and budget. This can add to project timeline and budget. Your project test plan should indicate what tools and methodologies will be used.
7. Applicable Standards and guidelines
What applicable standards and guidelines were in effect when the project started? The baseline budget and schedule can be impacted when the test team decides to adopt new standards and guidelines for testing.
Reply
Josh Nankivel, BSc PM, PMP Reply:
February 10th, 2010 at 2:51 am
Thanks Samad! I’ve got to get you doing some guest posting over here on pmStudent.com!
Reply
Samad Aidane Reply:
February 10th, 2010 at 8:04 am
Josh, I would be honored. Thank you so much. I will contact you.
Reply
Jennifer Bedell Reply:
February 10th, 2010 at 6:17 am
All very good points! I think my next topic will be the role of the BA and usability.
Has anyone ever worked with a Usability Analyst?
Reply
Feb 10th, 2010 at 9:34 am
Jennifer, having them view the requirements is a really good point. This is a problem I have not experienced but I see that it can pose a serious threat to scope. Great post. Thanks!
Reply
Feb 10th, 2010 at 10:24 am
Jennifer,
As I began reading the post, I thought testers can’t goldplate because of this or that. As I continued reading, you started to close the loop. By the end, you presented a post with a nice bow on top.
I’ve seen team members at every stage of the SDLC attempt to goldplate, with the best of intentions. Your mention of a triage is a very critical insight. Be it the customer or a customer representative (PM or BA), someone needs to vet anything and everything which could impact the baseline.
Without this control point, I think you’re guaranteed to see creep somewhere in your project.
Yes, we certainly want to deliver the greatest value to the customer. But, creep increases risk and that’s not value.
Excellent post!
Derek
http://twitter.com/derekhuether
http://thecriticalpath.info
Reply
Feb 10th, 2010 at 2:50 pm
In short: I disagree. You don’t want to turn your testers into machines blindly following specs/requirements.
I turned my long response into blog post: http://blog.brodzinski.com/2010/02/dont-stick-to-specs.html
Reply
Feb 10th, 2010 at 4:01 pm
Jen,
Good article, and raises some important points.
There as an important tension here: As testers we SHOULD go beyond the verification of written requirements and into the realm of validation: Can the solution fulfill its underlying need? We SHOULD be raising any issues that prevent the solution from doing so, or that limit its value to any stakeholder who counts.
BUT we must accept that our role is to provide information about product quality. It is the PMs role to manage scope and balance out time, cost and quality – and to negotiate the inevitable compromises with the customer.
An effective bug management process is essential in doing so – when it comes to sorting out the must fixes from the enhancements and the can-do-laters.
Reply
Feb 11th, 2010 at 4:17 am
This is a good topic. I’d suggest that a good developer will take any “defect” raised, compare it to the requirements and then log/update accordingly. I’ve always liked the idea of testing beyond the plan (because no plan is perfect) but I can see the danger that a developer takes any “defect” at face value and proceeds to a “fix” without referencing the requirements!
Reply