Estimating Effort: Part 3
Preparing Three-Point Range Estimates
The mechanics of preparing three-point range estimates, particularly if you are working with team members who have never used the technique before, are important. And just a reminder: we are still dealing with effort estimates; duration estimates come later.
Working one-on-one. Let’s again start with a simple case where you need an effort estimate from an individual team member regarding how much of their effort is likely to be required to complete a specific activity. The first step is to make certain that both of you have a common understanding of words like “estimate” and “complete.” For example, it is always tempting to say something like “how long do you think this will take?” This question is likely to produce a duration estimate rather than an effort estimate. Try to ask something like “how many hours of your time do you think this activity will take?”
Begin the discussion with a review of the work to be done to ensure that both you and your team member have a common understanding of the work. If there are unknowns, decide how to handle them—make assumptions, go get answers, whatever.
Once you have a reasonable level of agreement, ask the individual for their effort estimate of the most likely result. I tend to use words like “if these assumptions turn out to be true, how much of your time do you think this work is likely to take?” After they’ve answered that question, I ask “if every thing goes well, if you get very lucky with this activity, and still using the same assumptions, how much of your effort do you think this activity will take?” Then I use similar language to obtain a pessimistic effort estimate.
Remember to keep your assumptions constant. If you develop an optimistic estimate based on everything going well, and a pessimistic estimate based on nothing going well, the range of the estimates will be quite large. Very large ranges make most people uncomfortable because it appears that they don’t understand the work. In addition, it will be harder for you to assess the accuracy of the range estimate later because you’ll have to factor in all the different assumptions.
Understand that the first few conversations with any given individual are unlikely to go smoothly. They may generate additional questions or ask you to make additional assumptions at any point during the conversation. Fine. Let them. Help them climb the learning curve. My experience is that it takes most people 3-5 iterations to get comfortable with the process. After that, they’ll start giving you three-point estimates automatically.
Feel free to challenge their estimates, but do so constructively by asking clarifying questions. Do not cast aspersions on their estimates. If your estimate differs from theirs by a substantial amount, the most likely explanation is that your understanding of the work differs from theirs. Concentrate on identifying and resolving those differences.
It also seems to be important to start by asking for the most likely result, then the optimistic, and then the pessimistic. I don’t know why, but this sequence seems to work best.
Working with a group. Most of your estimates will not be developed one-on-one, but rather will be developed by small groups as part of a team planning process. The basic process remains the same for these small groups:
- Review and agree on the work to be done.
- Document the assumptions.
- Develop three-point range estimates starting with the most likely result.
Even when some of the group members don’t have deep expertise in the project’s work, their presence can help to surface unstated assumptions and to build team-wide commitment to and understanding of the effort estimates.
Other approaches to three-point range estimates. Some project managers try to expedite the development of a three-point range estimate by asking for only two numbers—the most likely and some form of variance (e.g., “10 hours, plus or minus 2”). While this statement equates to a three-point range estimate of 8, 10, and 12, in my experience, it lacks the richness of my process—it doesn’t seem to generate the same degree of thoughtfulness from the estimator.
Will this take too much time?
I am convinced that the thinking process and the conversations that support three-point range estimating contribute to the development of more accurate estimates. Short-circuiting the process may appear to save time, but it’s a false economy.
My experience is that developing three-point estimates almost always takes less time than single-point estimates because the conversational process eliminates the posturing and gamesmanship that often delays the development of estimates. I’ve have had teams in my project management training courses spend less than an hour to develop an estimate for a 4,000 hour project — an estimate that later proved highly accurate. A recent client spent less than a day to discover that a mission-critical, multi-million dollar new product development project was going to be eight months late. An investment of something less than $10,000 is expected to return in excess of $10,000,000. Time spent estimating is not a cost, it is cheap insurance.
In the next article of this series, I’ll cover some of the more difficult challenges of estimating effort on projects: what to do when management thinks it should take fewer hours, using range estimates with project management software, how to deal with the natural (and normal) biases of the estimators, understanding the difference between accuracy and precision, and how to use some basic, simple statistical methods to create an estimate for the whole project.
Estimating Effort
- Estimating Effort: Part 1
- Estimating Effort: Part 2
- Estimating Effort: Part 3
- Estimating Effort: Part 4
- Estimating Effort: Part 5
Jan 5th, 2009 at 10:34 pm
Bill,
Some good background on the purpose of the 3-point. It has been shown in clinical trials with the NAVY, that a Most Likely (ML) and a variance approach produces statistically better estimates in both novice and expert populations of estimators.
The three point estimate are random variables. In project work these random variables have specific underlying probability distributions. Triangle is the best fit in the absence of specific distribution knowledge.
Some reference for this topic that have served me well in this are include:
“Three Point Approximations for Continuous Random Variables,” Donald Keefer and Samuel Bodily, Management Science, 29(5), pp. 595 – 609.
“Better Estimation of PERT Activity Time Parameters,” Donald Keefer and William Verdini, Management Science, 39(9), pp. 1086 – 1091.
“Sources of Schedule Risk in Complex Systems Development,” Tyson R. Browning, INCOSE Systems Engineering Journal, Volume 2, Issue 3, pp. 129 – 142, 14 September 1999.
“Triangular Approximation for Continuous Random Variables in Risk Analysis,” David G. Johnson, Decision Sciences Institute Proceedings 1998
“Stochastic Project Duration Analysis Using PERT–Beta Distributions,” Ron Davis (personal communication)
“The Framing of Decisions and the Psychology of Choice,” Tversky, Amos, and Daniel Kahneman. 1981, Science 211 (January 30): 453–458
“Improved Three Point Approximation To Distribution Functions For Application In Financial Decision Analysis,” Michele E. Pfund, Jennifer E. McNeill, John W. Fowler and Gerald T. Mackulak, Department of Industrial Engineering, Arizona State University, Tempe, Arizona
“The Mother of All Guesses: A User Friendly Guide to Statistical Estimation,” Francois Melese and David Rose, Armed Forces Comptroller, 1998.
While the approach of asking ML, and the upper and lower bounds can yield results, it has also been shown through clinical research that the order in which the upper and lower bounds are asked and the words used during this asking have statistically significant effects on the answers. The work Dr. Roger Warburton at Boston College – near you – was presented at this year Society of Petroleum Engineers conference. where he and is grad studies spoke about the biases induced in 3-point estimate for reserve sizing.
The suggested approach will definitely get the team started on the process of understanding. For non-trivial projects, these estimates will produce biases in the total work effort and therefore the duration and cost, that will not be statistically correlated with the actual execution processes. this may of no interest to the project or it may be a critical risk factor depending on the level of counter correlation between cost, effort, and duration biases.
Glen B. Alleman
VP Program Planning and Controls
Reply
Jan 5th, 2009 at 11:16 pm
Bill,
One other thought. 3-Point estimates or for that matter any estimating process that provides a resulting range – either variance developed of boundary value developed is actually a “risk management process.”
The result is an exposure of the risk that the upper and lower bounds of the effort (duration or cost as well) will be different from the that provided by a single point estimate.
By eliciting these upper and lower (again through direct means or through variances), establishes that there is “variance” in the effort, duration and cost. This is a programmatic risk.
That the primary reasons for capturing these variables.
The second reason is to expose to the doubting management the notaion that any estimate is composed of random variables and if we don’t know the variance, we’re ignoring the risk.
I use the analogy of the median tempretature in Cody WY and Trinidand are nearly the same – 78 degrees. It’s the variance that is the problem. Don’t go to Cody in Feb in your shorts and tee shirt!!
Reply
Jan 8th, 2009 at 6:48 am
Glen –
Thanks for the references. I will try to track them down. I found the Navy experiment interesting, and the results in contrast to some work done at the Texas Department of Prisons where they found that a triangular distribution was the most accurate representation of costs for their projects.
In the next article, I’ll deal with some of the issues you raise.
And I love your example of Cody and Bermuda!
Duncan
Reply
Jan 8th, 2009 at 11:03 am
Bill,
Matahematically the triangle is best suited in the absensce of underlying information. It possess the least cummulant divergence for forecasting likley values drawn by a random number generator. You do not want the numbers to be correlated when produces them for modeling.
If the population statistics are known – cummulatants known – than other distributions can be used.
The paper at the SPE confernce concludes the order of asking the high, low and most likley has statistically signifcant differences. These estimates are made by subject matter experts – like the effort or duration should be.
Also, in our A&D work we ask for the duration of the workpackage and its work elements first, from the subject mater experts. Then we ask for the labor loading needed. This we we get a bounded end to end duration early in the Basis of Estimate process, that can be founded on previous work and define the pre-conditions for holding that schedule. The trouble with IT is they never seem to have any credible estimates of duration or effort, since the development of software is Ad Hoc process with little or no modeling from past performance. Even “new” development in A&D has parameteric modeling. Trying to introduce that into corporate IT is a bust, they simply don’t have the understanding of why or how this is a benefit. I blame this entirely on the agile community.
For embedded SW and ground system in defense, a customer would NEVER take an estimate that did not have some kind of parametric modeling. SEER, Price-S, and COCOMO models are mandated.
I’ll locate the paper in the archive and send it.
Reply