Wishful thinking – the inevitable outcome of "not knowing"?

Daniel Kahneman gives a great example of this in his book “Thinking, Fast and Slow”.

He describes a project, many years ago, where he convened a team to design a new high-school curriculum and to write a textbook for it (ironically, it was about judgement and decision making).  They had had several team meetings to construct an outline of the syllabus, had written a couple of chapters, and even run a few sample lessons.  They decided to do some planning, and to create an estimate of how long it would take to submit to finish draft of the textbook. Kahneman knew that one of the most effective ways of estimating is not to start with discussion, but to get everybody to individually submit their judgment, so has asked everybody to write down their estimates, and then collected these in.  Estimates ranged from 1 ½, years to 2 ½ years with a median of two years, to finish and submit the first draft.

Then he had another bright idea. He asked one of the curriculum experts on the team whether he could think of any examples of similar projects in the past, and how long they had taken.

“He fell silent” Kahneman writes. “When he finally spoke, it seemed to me that he was blushing; embarrassed by his own answer: ‘You know, I never realized this before, but in fact not all the teams at a stage comparable to ours ever did complete their task. A substantial fraction of the teams ended up failing to finish the job’”.  That fraction was 40%.

Kahneman then asked how long it took those who actually had finished the job.

“I cannot think of any group that finished in less than seven years” he replied, “nor any that took more than 10”. 

(Note that this guy himself had, shortly before, estimated it would take about two years!).  Then Kahneman asked how the current team ranked compared to the others (perhaps they were much better and could finish much faster!).  “We are below average” he replied “but not by much”.

So now the team had new knowledge, that comparable or better teams often fail at this job, and those that succeeded took four times longer than the groups estimate.

So what do you think the group did?

Kahneman tells us

“our state of mind when we heard Seymour is not well described by stating what we “knew”.  Surely all of us now “knew” that a minimum of seven years and a 40 per cent chance of failure was a more plausible forecast than the numbers we had written on a slips of paper.  But we did not acknowledge what we knew.  The new forecast still seemed unreal, because we could not imagine how it could take so long to finish a project that looks so manageable.  All we could see was a reasonable plan that should produce a book in about two years. ……….  The statistics that Seymour provided were treated as base rates normally are – noted and promptly set aside.  We should have quit that day.  None of us were willing to invest six more years of work in a project with a 40 per cent chance of failure …..  After a few minutes of a desultory debate,  we gathered ourselves together and carried on as if nothing had happened”.

The book was eventually finished eight years later.  The initial enthusiasm for the idea in the ministry of education had waned by the time the text was delivered, and it was never used.

This is a very interesting story.  Not only did the group not know what they didn’t know, they were unable or unwilling to accept the new knowledge when it was presented.  They ignored it, and continued with the wishful hope that they would be finished in two years.

Kahneman concludes from this that there are two different approaches to forecasting, the inside view and the outside view.  The inside view is based on “what you know that you know”, and what the team knew was that they had made some good progress already, albeit completing some of the easiest chapters at a time when enthusiasm was at its peak, and they extrapolated from this good progress.  But they didn’t know what they didn’t know, and they didn’t foresee the bureaucracy, the distractions, and the conflicts that would eventually arrive.  However because they were anchored to their inside view, they would not accept the outside view, even though the outside view was based on reliable baseline statistics.

The rest of Kahneman’s book, which I highly recommend, explores other aspects of the psychology of decision-making, and gives many examples of how people will make wrong decisions as a result of over confidence through limited data.

There are many are implications here for knowledge management; the need to access outside knowledge through activities such as peer assists, the need to collect baseline data on performance to act as a “reality check” for optimistic teams, and the need for continuous project learning to recognise when predictions are optimistic, and to renegotiate the prediction.

 Without effective knowledge management, and without effective knowledge-based decision making, project predictions will be, as Kanhneman’s project was, based largely on wishful thinking.