How to calculate the transmission efficiency of lessons learned

We can look at lesson learning as a supply chain; identifying new pieces of knowledge and supplying them to other knowledge workers so they can improve their work. If it works well, knowledge is gathered, accumulated, synthesised into guidance, and used to inform future operations.

This is something I tried to measure recently with a client, using a survey.

I divided the lesson supply chain into three steps, shown below

1. Capture and documentation of lessons from project experience;
2. Update of guidance documents based on new lessons;
3. Review of guidance documentation in future projects.

I therefore asked survey respondents to estimate in what percentage of cases the following happened, giving them the options to selecting 0%, 20%, 40%, 60%, 80% and 100%. The average estimate from the survey responders is shown below.

• Any given project documents lessons (46%)
• Any given lesson is used to update guidance documents (32%)
• Any given project makes use of guidance documents (43%)
• Any given project seeks for documented lessons (44%)
• Any given search finds useful lessons (36%)

The effectiveness of transmitting lesson through search is 46% (capture) x 44% (search) x 36% (find) = 7%.

These are pretty low figures! Even if both routes were independent and there was an overall success rate of 13%, that still means that 87% of project knowledge was never transmitted other than via human memory.

Let’s compare that with an organisation that treats lessons seriously. The diagram and numbers below are not from a survey, but from published statistics in another organisation. This organisation does not require users to search for lessons, but has a well-resourced supply chain to embed lessons into procedures and guidance.

• Any given project documents lessons (100% – lesson capture is mandatory part of every project, and supported by dedicated resources)
• Any given lesson is used to update guidance documents (93% – this figure was tracked on a quarterly basis and the “closure rate” for lessons varied between 88% and 95%)
• Any given project makes use of guidance documents (100% – all activity was directed by operational procedures which were reviewed and use to make the action plans)