Recently my team and I tried out a modified hybrid method of doing project estimation. I’d like to share it with you today.

This method is essentially a combination with the Delphi method of estimation that I have used before, and a agile method of estimation. Check out my friend Travis’ post on Cost Estimation for more on what prompted me to try something different.

Planning poker is the normal method that my team uses to come up with project estimates. Planning poker is excellent for getting everyone on the same page. I think this is a good idea and works well, but I like to try new things to see what works best. I very much believe that experimentation on a continuous basis is the only way you can discover things that work better what you doing.

One of my primary goals for this experiment is to find a method which avoids the phenomenon of anchoring as much as possible.

Let’s see what it looks like.

Step One: Deliverables Defined

Make sure that you have your work breakdown structure defined properly. If you’ve read my book, you know that the work breakdown structure is extremely important to me, regardless of whether you are working on a waterfall project or an agile/leave project.

If you don’t start out with a good definition of your scope, then everything else down the line is for naught. Garbage in, garbage out.

Step Two: Relative Sizing Delphi

The first round I used for my teams this time relied on the relative sizing method that many agile teams use when trying to come up with estimates for their projects.

So there was a range of T-shirt sizes between XXS to XXL and everything in between that my team to select from.

Whenever using any kind of relative sizing model, you need to make sure that your teams are comfortable with it first. Be sure to explain to them that these are completely relative, which means that one person’s L is different then another person’s L. And that’s okay.

The primary goal of this first round is to make sure that we have a good discussion and that all appropriate questions are raised about what exactly it is that we’re supposed be doing. It’s really kind of a refinement of the work breakdown structure process.

Step Three: Evaluate Relative Results

After you’ve gotten the relative sizes right now it seems to them all into a spreadsheet so that you can see them side-by-side. You’ll also have a list of questions to go track down the answers to from Step Two.

In some cases there will be a lot of agreement and consistency in the relative estimates, and other cases there will be wide variances. With relative sizing you must remember that one person’s L is going to be equivalent to someone else’s M since no calibration of the estimates have taken place. That’s fine. It’s when we have people who think a feature or item is an XS and others who think it’s a L, XL, or XXL where we need to revisit the discussion. Similar to what we do with planning poker, I facilitate discussion by asking questions when we have a high variance involved.

Step Four: Absolute Estimates

In this step we do two important things.

First, we discuss the results of the relative sizing step, without any reference to who estimated what. Remember, we are trying to eliminate anchoring. The focus here is to further refine the understanding of the scope, and have the questions answered from the previous session.

Second, we do a new delphi round, this time using hours. I like to use effort estimates, and base them on an ever-increasing scale to reflect that larger estimates are more uncertain. When between two estimates, tell the team to always round up. The hours I use are 5,8,13,20,40,80 – anything that ends up being larger than 80 hours can probably be broken down more discretely and you can iterate that estimate.

Step Five: Analysis and Final Results

Finally, I put all the effort estimates side-by-side again. I look at the optimistic and pessimistic values, take the median for likely, and calculate PERT on each feature/item.

Now there is some judgement involved. I look for cases where the person(s) with the most domain knowledge on a particular item differ widely from the PERT calculated value. Specialty and experience of the estimators is an important factor. If I have one person who is really a subject matter expert on a particular item and the rest of the team is not, I value the SME’s estimate more.

This assessment happens for each line item, and in the end I have a rollup I can use as well as standard deviation and other metrics to not only back up the estimates, but provide input for risk or reserve needs. I add things like documentation, meeting time and general overhead not directly related to implementation, and testing may be derived from implementation effort and historical numbers or estimated separately as it’s own set of activities.

We’ll see how this experiment turns out. What do you think about this approach?

[photo by Improve It]

Original link: How To Estimate Project Work

No related posts.

Guest post by Travis Anderson

Earn PDUs while learning how Oracle Primavera is adamant about riding the world of project failure. Get started by visiting the registration screen at Oracle Primavera Enterprise PPM Virtual Summit. http://vshow.on24.com/vshow/primavera2010/#home

Once you get your login information, go to the Auditorium and watch Joel Koppelman, Sr. VP & GM of Oracle Primavera Global BU, as he delivers an informative key note presentation called Enterprise Project Portfolio Management: Helping You Manage Change in a World of Constant Change. Joel indicates that projects create change, result from change, and require change. Project Managers need to be ready to handle change by being tolerant and equipped to respond in a dynamic and collaborative fashion. The processes and tools of yester year are simply no longer effective.

Also visit the Resource Center, Exhibit Hall, and Networking Lounge to read white papers, listen to podcasts, and learn valuable insights on industry trends.

I hope you enjoy this virtual summit. Please share your perspectives by leaving a comment.

Original link: Oracle Primavera Enterprise PPM Virtual Summit

No related posts.

As 2010 draws to a close, the need for predictions about what will happen in 2011 about.  I have even offered my own guesses and plans here, here, and here.

For this year-end post however, I offer some tools of skepticism to you, dear reader.  When it comes to predicting the future, we are all terrible at it.  Worse yet, confidence in our predictions and the accuracy of those predictions are inversely correlated.  The more confident we are, the more wrong we are.

“I do not pretend to start with precise questions.  I do not think you can start with anything precise.  You have to achieve such precision as you can, as you go along.”

- Bertrand Russell

Out of the thirty or forty books I read in 2010, several focused on the topics of decision making, estimation, and expert opinion. These are some of the books I’ve come across that have helped change my own thinking about expert opinion, our inability to predict the future, and the pitfalls we all fall prey to in our decision making process. I won’t give an in-depth review, but encourage you to check these books out and read the summaries and customer reviews for yourself.

Fooled by Randomness and The Black Swan

I like Nassim Nicholas Taleb. He cuts right to the chase and backs up what he says, and has a rebellious nature that I identify with and enjoy. I tend to think that if ‘everyone is doing it’ it’s probably wrong. Taleb can come off as a bit arrogant if you are not prepared for his tone, but if you appreciate brutal honesty I think you will enjoy his works.

In Fooled by Randomness, Taleb discusses probability and how it is misunderstood and illustrates points via thought experiments and short stories. I find some of his views on the role of randomness to be a bit over the top, but very insightful nonetheless. Misunderstanding statistics is something we are very good at as human beings, and this book illustrates the point well.

The Black Swan focuses on predicting the future, and how bad we are at it. A main focus are the large-scale outlier events we can not predict and usually forget to even attempt to include in forecasts. The lesson I drew from the book was to be humble about my ability to predict the future in any way.

How We Decide

In How We Decide, Jonah Lehrer cited many interesting psychological and neuroscience studies demonstrating that we don’t always make decisions in the manner we think we do. I wrote a bit about anchoring in project estimation as a result.  He also demonstrated that for some types of decisions and situations, intuitive decision making is better suited while in others it can lead us astray. I must say I was more interested in the cited studies and follow on research I did based on this book than the book itself, but it was still a pleasant read.

Sleights of Mind

I bought Sleights of Mind primarily because of my fascination with how human psychology works and the ways in which we can be fooled. If you like magic and science, this is definitely a book for you. In terms of managing projects, this book offered me some additional insight into how people can be fooled. Although the focus here is sensory, the aspects of self-delusion and justification are extremely pertinent when considering how decisions are made in the real world. It shows how we can remember X happening even though Y actually happened, which is a good thing to know when trying to plan projects using past experience.

Future Babble and Expert Political Judgment

These I haven’t read yet but plan to in 2011, but I have done some independent looking into Tetlock’s research on ‘expert opinion’ and it is absolutely astounding.  The bottom line is that human beings and in particular experts are really bad at predicting the future.  Gardner draws upon Tetlock’s research and I think I will probably go after Future Babble as a summary of this question, and hold off on getting Tetlock’s book unless I feel I need to go that route.  From the reviews it seems that Expert Political Judgement is rather dry, more of an academic book.

In general, most experts do worse than random chance at predicting the future and even the ones who are slightly better than chance aren’t better by much.  Additionally, looking at groups of people, one who is better than random chance and the other who is worse, there is one correlate.  Those who used a single tool or approach when generating estimates were more confident in their predictions and less comfortable with uncertainty.  They also were the “worse than random chance” group.  The other group was more comfortable with uncertainty, more tentative about their own predictions, and used an array of tools and approaches rather than relying on one ‘ultimate’ tool.

Original link: 6 Books To Make You a Better Project Manager

No related posts.

Scheduling is what project management is all about, right?

Among the plethora of project management tools available, what aspect is most widely promoted?

Jumping right into MS Project or any other scheduling tool is a mistake.

Projects like this are built on very unstable footing, and it’s likely they will fall apart in some way.

It’s just not safe.

If you haven’t fully developed a good Work Breakdown Structure (WBS)/PBS, requirements, and Basis of Estimates (BOE) before you start scheduling (and subsequently estimating costs and setting a budget), you’ve done it wrong.

So please, don’t open up a scheduling tool the moment you start a new project.  For me, there is a general order of operations to acheive project planning which is built on a sturdy foundation.  I don’t care if it’s waterfall, agile, whatever.  There are pieces between steps that go back and forth a bit before moving forward, but in general:

1. Why – (business case, charter)
2. What – (charter, WBS, requirements, use cases/user stories)
3. How/Who – (ConOps, Trade Studies, Design, Basis of Estimates)
4. When – (schedule, prioritized backlog)
5. Iterate – (progressive elaboration, sprint cycle)

[All wrapped inside a Project Management Plan/Approach, based on proven system engineering/industry practices,  and supported by risk and configuration management.]

Note that MS Project or other scheduling tools don’t enter the picture until Step #4.  I have never heard a convincing argument as to why anyone would think of scheduling anything until you had a good grasp on the foundational prerequisites I list in steps 1-3 above.

So what do you think?  Does my take on this topic match up with your own, or are you mad at me now because I’m talking about you?  Either way, please leave a comment and let’s discuss what you think.

Original link: Scheduling as Premature Elaboration: You’re Doing It Wrong

Related posts:

1. Rolling Wave Planning and Progressive Elaboration

Complexity in project estimation is important, and yet many if not most project managers seem to ignore it from what I’ve seen.

I have been thinking and reading up quite a bit on relative methods for eliciting estimates for projects.

I’ve enjoyed Planning Poker mostly as a means to get a team working more as a cohesive whole, thinking about the project in it’s entirety and how their own work relates to the big picture.  The process has been able to spark many “ah ha!” moments for my team members when someone who was new to a particular component or area of work became exposed to it.  Sometimes a fresh set of eyes can be very illuminating.

My first steps with teams who are new to these processes are to focus on pseudo-relative (meaning mainly absolute) sizing estimates since that is what they are accustomed to being asked for.  The next step is  talking in relative terms about complexity and size in a way we can calibrate and that makes sense to the teams.

I am going to try something new and see how it goes.

First, Planning Poker

First we  will conduct planning poker, looking at relative complexity of the stories we are estimating.  A story that everyone is familiar with will be selected and discussed as the starting point for our discussions, around which other stories will be judged in relative terms.

If you are new to planning poker, there are many great resources available to learn more about it.

Next, Relative Effort Mapping

Normal planning poker sessions result in a story point estimate you stick with.  What I would like to do here is extend and validate relative mappings with a visual tool used in a collaborative way with the team.

What’s this fancy tool?

A whiteboard and some post-it notes.  Sophisticated, I know…  just like how I do Kanban.

Each of the stories will be indicated on a post-it note.  Use the same colors to avoid any unconscious grouping tendencies.  On the whiteboard draw a 2-axis graph with complexity on the X-axis and size on the Y-axis.  Starting with the calibration story in the center, review each story quickly and have the primary story owner (who will be working/lead on it) place it where they think it belongs on the matrix after having already done the planning poker session.

Pretty simple.  Unnecessary?  Perhaps.

I’d like to experiment with it and see if it yields any new insights or strengthens the understanding of the work to be done within the minds of the team.  My hope is to discover interesting discrepancies between the results of planning poker and the relative size and complexity of the same stories on the matrix.  When experimenting, discrepancies always lead to new insights and questions to be tested further.  If I find no significant or systematic discrepancies over time I will likely abandon the practice, as that would indicate a lack of efficacy.

Original link: Project Estimation: Mapping Size and Complexity

No related posts.

The pre-frontal cortex of our brains can mislead us.  Big time. I am continuing my journey into the nature of project estimation and the irrationality of the human mind, in order to get better at it.  A few of the books I’ve listened to over the past few weeks include:

• The Black Swan: The Impact of the Highly Improbable – Nassim Nicholas Taleb
• A Whole New Mind: Why Right-Brainers Will Rule the Future – Dan Pink
• How We Decide – Jonah Lehrer

There is an example from that last book I want to talk about today.

MIT economists led by Dan Ariely did an experiment with their business school graduates , and later on executives and managers at the MIT Executive Education Program.

It was an auction with various products including wine, a wireless keyboard, and chocolate truffles.

Before they bid on miscellanous items, they were asked to write down the last 2 digits of their social security numbers.

Then, they were asked whether or not they would be willing to pay that amount (the last 2 digits of their social security number) for each of the products.

Finally, students wrote down the maximum amount they would be willing to pay for each item.

Results

It’s obvious that the last 2 digits of your social security number should have nothing to do with the value you place on random objects, right?  Someone with two last digits of 10 and another person with two last digits of 90 might be expected to bid similar amounts on the same item, on average.

Here’s an example of what happened.

For the cordless keyboard, the group who had social security numbers ending between 80 and 99 bid an average of $56.  Those who had social security numbers ending between 1 and 20 had an average bid of $16.

FOR THE SAME PRODUCT ???

In Project Management

This phenomenon is reflected in project estimation.  I’ve recently had short-term estimates turn out to be completely off, and I’m talking about within 2 weeks of actually doing the work.  WHY?

One explanation could be lack of specific information.  With a complex interconnected software system it can be difficult to tell what the “real” impacts of something will be until you get in there and start looking at the specifics.

Another explanation is this anchoring effect.  And anchoring doesn’t just happen when you give someone a number right before they estimate…it can be coming from many sources, with or without your knowledge as the project manager.

I don’t believe this means that expert opinion is useless for estimation; at least not totally.  I’ve been thinking through some techniques that could be used to eliminate the influence of anchoring to a large extent, and if you have any ideas on the topic I’d love to hear them.

Original link: What’s That Got To Do With The Price Of A Cordless Keyboard?

No related posts.