Category Archives: Estimation

Project Management Estimation Techniques

How To Estimate Project Work

Recently my team and I tried out a modified hybrid method of doing project estimation. I’d like to share it with you today.

This method is essentially a combination with the Delphi method of estimation that I have used before, and a agile method of estimation. Check out my friend Travis’ post on Cost Estimation for more on what prompted me to try something different.

Planning poker is the normal method that my team uses to come up with project estimates. Planning poker is excellent for getting everyone on the same page. I think this is a good idea and works well, but I like to try new things to see what works best. I very much believe that experimentation on a continuous basis is the only way you can discover things that work better what you doing.

One of my primary goals for this experiment is to find a method which avoids the phenomenon of anchoring as much as possible.

Let’s see what it looks like.

Step One: Deliverables Defined

Make sure that you have your work breakdown structure defined properly. If you’ve read my book, you know that the work breakdown structure is extremely important to me, regardless of whether you are working on a waterfall project or an agile/leave project.

If you don’t start out with a good definition of your scope, then everything else down the line is for naught. Garbage in, garbage out.

Step Two: Relative Sizing Delphi

The first round I used for my teams this time relied on the relative sizing method that many agile teams use when trying to come up with estimates for their projects.

So there was a range of T-shirt sizes between XXS to XXL and everything in between that my team to select from.

Whenever using any kind of relative sizing model, you need to make sure that your teams are comfortable with it first. Be sure to explain to them that these are completely relative, which means that one person’s L is different then another person’s L. And that’s okay.

The primary goal of this first round is to make sure that we have a good discussion and that all appropriate questions are raised about what exactly it is that we’re supposed be doing. It’s really kind of a refinement of the work breakdown structure process.

Step Three: Evaluate Relative Results

After you’ve gotten the relative sizes right now it seems to them all into a spreadsheet so that you can see them side-by-side. You’ll also have a list of questions to go track down the answers to from Step Two.

In some cases there will be a lot of agreement and consistency in the relative estimates, and other cases there will be wide variances. With relative sizing you must remember that one person’s L is going to be equivalent to someone else’s M since no calibration of the estimates have taken place. That’s fine. It’s when we have people who think a feature or item is an XS and others who think it’s a L, XL, or XXL where we need to revisit the discussion. Similar to what we do with planning poker, I facilitate discussion by asking questions when we have a high variance involved.

Step Four: Absolute Estimates

In this step we do two important things.

First, we discuss the results of the relative sizing step, without any reference to who estimated what. Remember, we are trying to eliminate anchoring. The focus here is to further refine the understanding of the scope, and have the questions answered from the previous session.

Second, we do a new delphi round, this time using hours. I like to use effort estimates, and base them on an ever-increasing scale to reflect that larger estimates are more uncertain. When between two estimates, tell the team to always round up. The hours I use are 5,8,13,20,40,80 – anything that ends up being larger than 80 hours can probably be broken down more discretely and you can iterate that estimate.

Step Five: Analysis and Final Results

Finally, I put all the effort estimates side-by-side again. I look at the optimistic and pessimistic values, take the median for likely, and calculate PERT on each feature/item.

Now there is some judgement involved. I look for cases where the person(s) with the most domain knowledge on a particular item differ widely from the PERT calculated value. Specialty and experience of the estimators is an important factor. If I have one person who is really a subject matter expert on a particular item and the rest of the team is not, I value the SME’s estimate more.

This assessment happens for each line item, and in the end I have a rollup I can use as well as standard deviation and other metrics to not only back up the estimates, but provide input for risk or reserve needs. I add things like documentation, meeting time and general overhead not directly related to implementation, and testing may be derived from implementation effort and historical numbers or estimated separately as it’s own set of activities.

We’ll see how this experiment turns out. What do you think about this approach?

[photo by?Improve It]

Oracle Primavera Enterprise PPM Virtual Summit

Guest post by Travis Anderson

Earn PDUs while learning how Oracle Primavera is adamant about riding the world of project failure. Get started by visiting the registration screen at Oracle Primavera Enterprise PPM Virtual Summit. http://vshow.on24.com/vshow/primavera2010/#home

Once you get your login information, go to the Auditorium and watch Joel Koppelman, Sr. VP & GM of Oracle Primavera Global BU, as he delivers an informative key note presentation called Enterprise Project Portfolio Management: Helping You Manage Change in a World of Constant Change. Joel indicates that projects create change, result from change, and require change. Project Managers need to be ready to handle change by being tolerant and equipped to respond in a dynamic and collaborative fashion. The processes and tools of yester year are simply no longer effective.

Also visit the Resource Center, Exhibit Hall, and Networking Lounge to read white papers, listen to podcasts, and learn valuable insights on industry trends.

I hope you enjoy this virtual summit. Please share your perspectives by leaving a comment.

6 Books To Make You a Better Project Manager

As 2010 draws to a close, the need for predictions about what will happen in 2011 about.? I have even offered my own guesses and plans here, here, and here.

For this year-end post however, I offer some tools of skepticism to you, dear reader.? When it comes to predicting the future, we are all terrible at it.? Worse yet, confidence in our predictions and the accuracy of those predictions are inversely correlated.? The more confident we are, the more wrong we are.

“I do not pretend to start with precise questions.? I do not think you can start with anything precise.? You have to achieve such precision as you can, as you go along.”

– Bertrand Russell

Out of the thirty or forty books I read in 2010, several focused on the topics of decision making, estimation, and expert opinion. These are some of the books I’ve come across that have helped change my own thinking about expert opinion, our inability to predict the future, and the pitfalls we all fall prey to in our decision making process. I won’t give an in-depth review, but encourage you to check these books out and read the summaries and customer reviews for yourself.

Fooled by Randomness and The Black Swan

I like Nassim Nicholas Taleb. He cuts right to the chase and backs up what he says, and has a rebellious nature that I identify with and enjoy. I tend to think that if ‘everyone is doing it’ it’s probably wrong. Taleb can come off as a bit arrogant if you are not prepared for his tone, but if you appreciate brutal honesty I think you will enjoy his works.

In Fooled by Randomness, Taleb discusses probability and how it is misunderstood and illustrates points via thought experiments and short stories. I find some of his views on the role of randomness to be a bit over the top, but very insightful nonetheless. Misunderstanding statistics is something we are very good at as human beings, and this book illustrates the point well.

The Black Swan focuses on predicting the future, and how bad we are at it. A main focus are the large-scale outlier events we can not predict and usually forget to even attempt to include in forecasts. The lesson I drew from the book was to be humble about my ability to predict the future in any way.

How We Decide

In How We Decide, Jonah Lehrer cited many interesting psychological and neuroscience studies demonstrating that we don’t always make decisions in the manner we think we do. I wrote a bit about anchoring in project estimation as a result.? He also demonstrated that for some types of decisions and situations, intuitive decision making is better suited while in others it can lead us astray. I must say I was more interested in the cited studies and follow on research I did based on this book than the book itself, but it was still a pleasant read.

Sleights of Mind

I bought Sleights of Mind primarily because of my fascination with how human psychology works and the ways in which we can be fooled. If you like magic and science, this is definitely a book for you. In terms of managing projects, this book offered me some additional insight into how people can be fooled. Although the focus here is sensory, the aspects of self-delusion and justification are extremely pertinent when considering how decisions are made in the real world. It shows how we can remember X happening even though Y actually happened, which is a good thing to know when trying to plan projects using past experience.

Future Babble and Expert Political Judgment

These I haven’t read yet but plan to in 2011, but I have done some independent looking into Tetlock’s research on ‘expert opinion’ and it is absolutely astounding.? The bottom line is that human beings and in particular experts are really bad at predicting the future.? Gardner draws upon Tetlock’s research and I think I will probably go after Future Babble as a summary of this question, and hold off on getting Tetlock’s book unless I feel I need to go that route.? From the reviews it seems that Expert Political Judgement is rather dry, more of an academic book.

In general, most experts do worse than random chance at predicting the future and even the ones who are slightly better than chance aren’t better by much.? Additionally, looking at groups of people, one who is better than random chance and the other who is worse, there is one correlate.? Those who used a single tool or approach when generating estimates were more confident in their predictions and less comfortable with uncertainty.? They also were the “worse than random chance” group.? The other group was more comfortable with uncertainty, more tentative about their own predictions, and used an array of tools and approaches rather than relying on one ‘ultimate’ tool.

Scheduling as Premature Elaboration: You?re Doing It Wrong

Scheduling is what project management is all about, right?

Among the plethora of project management tools available, what aspect is most widely promoted?

Jumping right into MS Project or any other scheduling tool is a mistake.

Projects like this are built on very unstable footing, and it’s likely they will fall apart in some way.

It’s just not safe.

If you haven’t fully developed a good Work Breakdown Structure (WBS)/PBS, requirements, and Basis of Estimates (BOE) before you start scheduling (and subsequently estimating costs and setting a budget), you’ve done it wrong.

So please, don’t open up a scheduling tool the moment you start a new project.? For me, there is a general order of operations to acheive project planning which is built on a sturdy foundation.? I don’t care if it’s waterfall, agile, whatever.? There are pieces between steps that go back and forth a bit before moving forward, but in general:

  1. Why – (business case, charter)
  2. What – (charter, WBS, requirements, use cases/user stories)
  3. How/Who – (ConOps, Trade Studies, Design, Basis of Estimates)
  4. When – (schedule, prioritized backlog)
  5. Iterate – (progressive elaboration, sprint cycle)

[All wrapped inside a Project Management Plan/Approach, based on proven system engineering/industry practices,? and supported by risk and configuration management.]

Note that MS Project or other scheduling tools don’t enter the picture until Step #4.? I have never heard a convincing argument as to why anyone would think of scheduling anything until you had a good grasp on the foundational prerequisites I list in steps 1-3 above.

So what do you think?? Does my take on this topic match up with your own, or are you mad at me now because I’m talking about you?? Either way, please leave a comment and let’s discuss what you think.

Project Estimation: Mapping Size and Complexity

Complexity in project estimation is important, and yet many if not most project managers seem to ignore it from what I’ve seen.

I have been thinking and reading up quite a bit on relative methods for eliciting estimates for projects.

I’ve enjoyed Planning Poker mostly as a means to get a team working more as a cohesive whole, thinking about the project in it’s entirety and how their own work relates to the big picture.? The process has been able to spark many “ah ha!” moments for my team members when someone who was new to a particular component or area of work became exposed to it.? Sometimes a fresh set of eyes can be very illuminating.

My first steps with teams who are new to these processes are to focus on pseudo-relative (meaning mainly absolute) sizing estimates since that is what they are accustomed to being asked for.? The next step is? talking in relative terms about complexity and size in a way we can calibrate and that makes sense to the teams.

I am going to try something new and see how it goes.

First, Planning Poker

First we? will conduct planning poker, looking at relative complexity of the stories we are estimating.? A story that everyone is familiar with will be selected and discussed as the starting point for our discussions, around which other stories will be judged in relative terms.

If you are new to planning poker, there are many great resources available to learn more about it.

Next, Relative Effort Mapping

Normal planning poker sessions result in a story point estimate you stick with.? What I would like to do here is extend and validate relative mappings with a visual tool used in a collaborative way with the team.

What’s this fancy tool?

A whiteboard and some post-it notes.? Sophisticated, I know…? just like how I do Kanban.

Each of the stories will be indicated on a post-it note.? Use the same colors to avoid any unconscious grouping tendencies.? On the whiteboard draw a 2-axis graph with complexity on the X-axis and size on the Y-axis.? Starting with the calibration story in the center, review each story quickly and have the primary story owner (who will be working/lead on it) place it where they think it belongs on the matrix after having already done the planning poker session.

Pretty simple.? Unnecessary?? Perhaps.

I’d like to experiment with it and see if it yields any new insights or strengthens the understanding of the work to be done within the minds of the team.? My hope is to discover interesting discrepancies between the results of planning poker and the relative size and complexity of the same stories on the matrix.? When experimenting, discrepancies always lead to new insights and questions to be tested further.? If I find no significant or systematic discrepancies over time I will likely abandon the practice, as that would indicate a lack of efficacy.

What’s That Got To Do With The Price Of A Cordless Keyboard?

The pre-frontal cortex of our brains can mislead us. ?Big time. I am continuing my journey into the nature of project estimation and the irrationality of the human mind, in order to get better at it. ?A few of the books I’ve listened to over the past few weeks include:

There is an example from that last book I want to talk about today.

MIT economists led by Dan Ariely did an experiment with their business school graduates , and later on executives and managers at the MIT Executive Education Program.

It was an auction with various products including wine, a wireless keyboard, and chocolate truffles.

Before they bid on miscellanous items, they were asked to write down the last 2 digits of their social security numbers.

Then, they were asked whether or not they would be willing to pay that amount (the last 2 digits of their social security number) for each of the products.

Finally, students wrote down the maximum amount they would be willing to pay for each item.

Results

It’s obvious that the last 2 digits of your social security number should have nothing to do with the value you place on random objects, right? ?Someone with two last digits of 10 and another person with two last digits of 90 might be expected to bid similar amounts on the same item, on average.

Here’s an example of what happened.

For the cordless keyboard, the group who had social security numbers ending between 80 and 99 bid an average of $56. ?Those who had social security numbers ending between 1 and 20 had an average bid of $16.

FOR THE SAME PRODUCT ???

In Project Management

This phenomenon is reflected in project estimation. ?I’ve recently had short-term estimates turn out to be completely off, and I’m talking about within 2 weeks of actually doing the work. ?WHY?

One explanation could be lack of specific information. ?With a complex interconnected software system it can be difficult to tell what the “real” impacts of something will be until you get in there and start looking at the specifics.

Another explanation is this anchoring effect. ?And anchoring doesn’t just happen when you give someone a number right before they estimate…it can be coming from many sources, with or without your knowledge as the project manager.

I don’t believe this means that expert opinion is useless for estimation; at least not totally. ?I’ve been thinking through some techniques that could be used to eliminate the influence of anchoring to a large extent, and if you have any ideas on the topic I’d love to hear them.

How Expectations Mess Up Project Estimates

Glen Alleman recently pointed me to a paper by Jorge Aranda among other material on software estimation.

I sat down and read Anchoring and Adjustment in Software Estimation and it was well worth my time.

by Andrew Stawarz via Flickr

by Andrew Stawarz via Flickr

To cut to the chase, the subjects were tasked to give estimates for software tasks in a controlled manner, in 3 groups with various “anchoring” methods being used.? The only difference between the groups was the expectation statement by the manager before estimation.

Group 1 (control – no explicit anchor given)

?I?d like to give an estimate for this project myself, but I admit I have no
experience estimating. We?ll wait for your calculations for an estimate.?

Group 2 ( ‘2 months’ condition)

?I admit I have no experience with software projects, but I guess this
will take about 2 months to finish. I may be wrong of course, we?ll
wait for your calculations for a better estimate.?

Group 3 (’20 months’ condition)

?I admit I have no experience with software projects, but I guess this
will take about 20 months to finish. I may be wrong of course, we?ll
wait for your calculations for a better estimate.?

You will need to read the full paper to see all the goodies (and to determine if you think it is relevant to your domain), but I would like to summarize some of the results I found striking.

These were the results among all participants, and there are other slices of the data available in the paper including only experienced participants and also by estimation method chosen.

‘2 months’ condition

  • mean – 6.8 months
  • median – 6 months
  • standard deviation – 3.7

control – no explicit anchoring

  • mean – 8.3
  • median – 7
  • standard deviation – 4.4

’20 month’ condition

  • mean – 17.4
  • median – 16
  • standard deviation – 5.6

The results in general coincide with my own experience on this matter.? An important point to note is that even though they were supposedly estimating the exact same software requirements, it is very likely that the ‘2 month’ group would have produced a significantly different product than the ’20 month’ group.

Food for thought.

When you and your team are putting together estimates, what influences are creating these anchors?? From my experience there are many of them, some of which are likely to be arbitrary or set (even inadvertently) without sufficient knowledge or experience.? They may be coming from stakeholders, sponsors, the project manager, or even a team member/lead.

My favorite example of this is when a team is asked to provide a “back of the envelope” estimate without really understanding the scope yet.? It produces a bad estimate and sets a rather arbitrary anchor for future estimates.

What do you think?

Project Estimation Methods

Project estimation can be a headache for new project managers. ?There are lots of opinions out there and not much that weighs various methods…rather you find people who feel strongly about one way or another. ?To give a quick overview of some ideas and then call for your input, I recorded this video. ?Please leave a comment and share your thoughts on this topic!

Struggling with Cost Estimation? Check This Out.

Once upon a time, I was searching for some solid guidelines on cost estimation to improve the way my contract was doing it.

I found a few great resources, and one of them was the US GAO Cost Estimating and Assessment Guide.? It contains a wealth of knowledge and guidelines for major government projects, and also a lot of lessons that non-government and small projects can use.

Here is a link to the guide, and a video below where I introduce it for you.

I record in HD so you can watch the HD version and read everything clearly on my screen (hit the full-screen button in the lower-right)

Click the maximize button to watch in full screen HD

Once Bitten, Twice Shy

Great White flashback to 1987 - Is he wearing a robe?

To my wife’s dismay, this is not a post about the 80’s hair band, Great White.

(Thank goodness!)

This is about project communication and estimation.


Mark had done his best.? Being new to project teams, he tried to estimate his tasks as well as he could back when Jack (the project manager) asked.

Now it was several months into the project, and was becoming painfully obvious that Mark wasn’t keeping on schedule.

After a particularly painful project status review, Jack called Mark into his office.

“Mark, I expect you to perform at the level you committed to.? There is no excuse for this, you are making our entire project team look bad.? Your tasks were nowhere near the critical path before, but now you ARE the critical path.”

Now that’s some whiz-bang project communication right there.

Mark walked out of Jack’s office, dejected.

After struggling through that project (Jack went out and got an extra person to help with Mark’s tasks, which resulted in another lecture and brought the project in over the original budget) it was time for another one.

After the whupping he took last time, Mark knew what to do.? This time when doing estimates, he remembered all the things that got in his way last time he hadn’t anticipated.? He came to an estimate on each task.? Then he doubled it.

Sue was running the project Mark was on this time.? Jack stopped Sue in the hall one day as they were passing by each other.

“Say Sue, you have Mark on your new project, right?”

“Yep.? Why?”

“If I were you, I’d take his estimates and double them.? I went through hell on my last project because his estimates were way too low.”

“Wow.? Thanks for the tip Jack!”


Of course this is a simplified caricature, but this short story illustrates some key issues that occur on projects often, to a greater or lesser degree.? As? a new project manager, this kind of scenario is something to watch out for.

  • Group estimates could have been used to get clear on scope and assumptions and bring in more experience
  • Instead of chewing out Mark, Jack should have sought out the root cause and took responsibility upon himself
  • Hopefully, Sue will use the information Jack gave her to sort out the root cause with Mark and ensure solid estimating practices are used.
  • Is Sue’s sponsor going to pad the estimates even further?
  • Will Parkinson’s Law come into play with these heavily padded estimates, so that they will be a self-fulfilling prophecy?

The worst part is probably that Mark has been bitten, and in the future will be very unlikely to trust a project manager who might want to implement something like Critical Chain Project Management, where padding is removed from individual tasks and it’s OK to go over on individual tasks, because you are actually shooting for a 50% chance of completion on time.

No, a barrier has gone up between Mark and his project managers.

Once bitten, twice shy.

Do you have similar stories to share?? How about additional comments on this scenario?