3 ways to estimate the value of lessons learned

Many organisations attempt to assign value to lessons in a lessons management system, and there are three ways you can do this. 

A screen sub-panel from the lessons management hub
showing value assigned to lessons

Assigning value to lesson-learning has three main advantages;

  • It reassures the people using the system that there is value in lesson learning. A panel on the front page of your lessons management system, such as the one shown here, reassures people logging into the system that sharing and re-using lessons is a valuable thing to do.
  • Lessons can be high-graded according to value, with the most valuable lessons getting highest priority.
  • It reassures management that there is value in lesson learning, and makes them think twice before axing the lessons management team and closing down the system.

However there are also counter-arguments;

  • Assigning value to lessons can be subjective (see below)
  • Value is yet another piece of metadata that needs to be added when documenting a lesson

If you decide to go down the path of assigning value to lessons, you can estimate this value in three ways;

  1. You estimate a projected value, where you look at the impact the lesson had on the project where the lesson was identified (measured through lost time, saved time, wasted cost, saved cost etc), and then you forecast  that forward by estimating the frequency of recurrence. Imagine a new way of working was developed which saved a project $100k on a particular activity. Imagine that activity is repeated in other projects a total of 10 times a year. If you document that new way of working as a lesson, and/or embed it into project procedures, then the projected value of that lesson is $1 million per year ($100k times 10, assuming all the future projects re-use the lesson). This of course is only an estimation – maybe the activity is repeated only 5 times, or 20 times. In some cases it might save $50k, in others $200k.
  1. You record an actual value, where the lesson is re-used (either directly, or through embedding into process) and you can then measure the improvement that results. This is more accurate than the projected value, but it sometimes can be difficult to isolate the impact of one lesson when many lessons are applied together.  The reporting is often anecdotal – “We started our project by reviewing and adopting all the lessons from the past. The project was delivered in X time/cost which is  saving of Y compared to the benchmark”. This approach was used by the Ford Best Practice replication system, where manufacturing-plant contact who received a lesson needed to report what had been done with this knowledge in their local plant and, if they applied the lesson, what value it has added.
  1. You record an aggregate value, by looking at the improvement in results over time.  There are several examples on this blog of the aggregate improvement that comes through learning and re-use of knowledge, for example in Trinidad offshore platforms, Oil wells in Oman, Nuclear power stations in Korea, and jumping frogs in California.

If you can, assign value to lessons. This reassures both managers and workers that lesson-learning is a good investment of time and effort.

View Original Source (nickmilton.com) Here.

How to learn like an ant

Social and organisational learning is so easy that even ants can do it, and we can learn from the principles they apply.

If you look at an ant trail from the nest to a source of food, it is pretty direct. The food may have been found by a random foraging ant, but pretty soon the whole colony has been alerted to the food, and developed a path that approximates to the quickest route between the food and the nest.

So how do ants collaborate to build the “best route”? Do they design it, or do they evolve it through continuous improvement? And if the latter, exactly how are these continuous improvements made? How do ants LEARN to make a straight path?

  1. When ants move, they leave a scent trail behind. It’s a fading trail – immediately after depositing, it starts to fade. Initially their trail is pretty random, and may zigzag all over the place. Other ants follow the trail, following the strongest scent, but not 100% faithfully. There’s a little bit of variation built in.
  2. Because of that variation, ants will sometimes find a shortcut, and cut out one of the zigzags. Wherever they cut the corner and get there faster, their scent is stronger, and becomes the dominant trail. Other ants follow them. If their new way takes longer (a longcut), their scent has faded, and others don’t follow. For the ants – faster is better as it is more efficient.
  3. So the improvements in the trail are reinforced, and the trail gets progressively better and straighter. Over time, the trail becomes straight.

For ants, the organisational memory lies in the trail itself, embedded in the scent. For organisations, the organisational memory lies in the processes. We can follow the principles of Ant-learning, if we replace the scent-trail with the operational procedures.

  1. We record our knowledge in operational procedures, which represent our current best approach to doing a particular task. Initially the procedures may not be particularly effective, and people are allowed to vary from the procedures if they find a better way.
  2. The variations are recorded as process improvements or lessons learned and embedded in improved procedures, like the new scent trail along the shortcut.  There needs to be some way to know whether the new variations are better. Perhaps they improve efficiency, or quality, or cost, or customer satisfaction. You need to be able to tell what “better” looks like.
  3. Other teams follow the new better procedures (like the ants following the stronger scent) so that these become the dominant procedures – until the next improved variation is found. Over time the procedure becomes optimal. 

You can see this learning process at work in any continuous improvement process, whether it is a drilling crew improving their rig procedures, a lean manufacturing team improving their manufacturing procedures, or an Army improving their doctrine. They all learn like an ant – recording and embedding the improvements until the trail is straight.

View Original Source (nickmilton.com) Here.

6 benefits of reflective team learning

I was reflecting recently after a major lessons capture exercise from a multi-million Euro project of the benefits of this sort of reflective team learning.  It struck me that there are actually 6 areas of benefit from this practice.

  1. Firstly, the team members learn themselves, as individuals. By listening to the reflections and analysis of their colleagues, they gain new learning and new insight which will help them improve their personal repertoire of skills and understanding.
  2. The team members are able to share their thoughts and feelings in a non-judgmental environment. They find this emotionally valuable, and it can be a healing process after a traumatic project.
  3. The teams identify improvements they can make to their own team process. By reflecting on the past, they can improve their own ways of working together. 
  4. The conversations and analysis can be recorded, and turned into valuable learning material for other teams.
  5. This documented learning material is not only an asset in its own, right, it also allows other teams to know what lessons have been learned, and who to contact to learn more. Documented lessons are often of most value when they are the catalyst for conversations (and we know that conversations are approximately 14 times more effective than written material in transferring knowledge). 
  6. Finally the lessons and experiences of the team can drive improvements in the way the whole organisation operates. In a recent exercise we identified 30 major learning points, and about half of these were associated with requests of recommendations for Headquarters, to embed the lessons into new process or new structures. 

View Original Source (nickmilton.com) Here.

KM and Hansei, where "no problem" becomes a problem

Effective learning within an organisation requires consistent and rigorous self-analysis, in order to pick up learning points and points of improvement. In Japan, this process is known as Hansei.

はんせい、Hansei
Hansei, by Jim O’Neil, on Flickr

Although alien to many in the west, Hansei is an important part of the Japanese culture. Han means “change” and Sei means “to review”, so the whole thing means “introspection” or “reflection for the purposes of change”.  This translates into a behaviour , instilled from childhood, of looking for mistakes, admitting responsibility, and implementing change.(When Japanese children do something wrong, for example, they are scolded and are told: hansei shinasai – Do hansei!).

Hansei can become very public, as the footage of the crying Toyota CEO shows. As a response to poor safety performance, the CEO admitted responsibility, apologised, promised change, and wept – behaviour lauded in Japan but deemed strange in the West. In the West we would probably avoid responsibility, deny the mistake as “fake news”, and insult our detractors.

Hansei is at the heart of Kaizen – the “learning from experience” approach seen in Japanese industry. It may part of the reason why Japanese companies succeed so well at Kaizen as a core component of Knowledge Management, while other cultures struggle. Where a European company might see lesson-learning as a witch-hunt, for example, a Japanese company would see it as a way to put right the embarrassment of self-acknowledged failure. Where a US company might fear a blame culture, Hansei means that individuals already accept any blame and if they fear anything, they fear the lack of ability to make restitution.

How do we develop Hansei?

In non-Japanese cultures we have not been brought up with Hansei. Seeing our leaders accepting full responsibility for mistakes and sincerely, with emotion, promising change is something exceedingly rare (name me one example!).  However this is a behaviour we would dearly love to promote at work, so that mistakes are not hidden, but lead to learning and change.

So here are six things we can do to begin to develop Hansei behaviours.

1. We can understand the current culture, and recognise the barriers. One of the 10 cultural barriers is defensiveness – an unwillingness to take responsibility and examine your mistakes. Our cultural assessment service allows you to see whether this is a major barrier in your own organisation.

2. We can build reflection into the work process. After Action Review, for example, is a Kaizen activity that can be embedded into the working pattern, to trigger reflection and change on a regular basis.

3. We can adopt no-blame learning processes. The Retrospect is widely recognised as a no-blame lesson-learning process for use at the end of projects or project stages. The open questioning within the Retrospect gives people the opportunity to examine what went wrong, and to suggest how this might be improved.

4. We can ask the team leader to set the tone. If we are concerned about lack of openness in a Retrospect, we can work with the team leader before hand to identify an area where they can openly admit to making a mistake, and explore how to avoid this happening again. When the leader sets the tone, the rest of the team may follow.

5. We can ensure all learnings lead to action. We must make sure that everyone present in a Retrospect or After Action review can see that their admissions, introspections and lessons will lead to action. Lessons will not just rot away in overstuffed databases, but become embedded in changes to process. Knowing that your mistakes can be turned into successes for others can make Retrospects into something like group therapy. This is the positive outcome of Hansei.

6. We can become intolerant of complacency. Another of the 10 cultural elements, complacency is the feeling that “we did alright, there was no problem, we don’t need to change anything”.  Here is what the Toyota website says about “no problem”:

“Even if a task is completed successfully, Toyota recognises the need for a hansei-kai, or reflection meeting; a process that helps to identify failures experienced along the way and create clear plans for future efforts. An inability to identify issues is usually seen as an indication that you did not stretch to meet or exceed expectations, that you were not sufficiently critical or objective in your analysis, or that you lack modesty and humility. Within the Hansei process, no problem is itself a problem“.

This type of thinking – where “no problem” is seen as symptom of a lack of introspection and a lack of analysis, and something to challenge rather than to feel smug about – may be what separates the  true learning cultures from the also-rans.

View Original Source (nickmilton.com) Here.

Why a no-blame culture needs no-blame processes

We hear a lot about the importance of a “no-blame culture” in Lesson-learning, but a no-blame culture won’t work unless you have no-blame processes as well. 

Image from wikimedia commons

Learning lessons in an organisation requires a culture of openness, so that people are willing to explore honestly and openly what happened, and what might be learned.  This is particularly important when learning from projects or incidents where things have gone wrong and where mistakes have been made.  If people feel they are likely to blamed for the mistakes, and that their career or reputation might suffer as a result, they are unlikely to discuss the issue openly.

When we discuss the topic of lesson-learning with clients, we often hear the concern that lesson-learning meetings could be seen as “witch-hunts” – in other words, a search for someone to blame. To reassure them, we talk them through the lesson-learning process we use; the Retrospect.

The Retrospect is an externally-facilitated team meeting, held after the end of a project or project stage, where the team identifies and analyses learning points through discussion and dialogue, in order to derive lessons and actions for the future. The facilitator leads an inclusive discussion to identify

  • What went well and what did not go to plan in the project
  • Why the successful elements succeeded, and why the failures and mistakes happened (looking for root cause)
  • How future projects can repeat the success and avoid the failure. 

The key questions are therefore What, Why and How; very open questions which allow full exploration of the lessons learned.

Note that there is no Who question. 

We do not really care who was the hero or who was the villain. This is a no-blame process, as well as a no-praise process. We are searching neither for Witches or for Knights; only for the truth.

An open, no-blame culture requires open no-blame processes such as the Retrospect.

View Original Source (nickmilton.com) Here.

Why "Knowledge for action" is better than "knowledge for storage"

Knowledge has to lead to action in order to add value. 

call to action by Sean MacEntee on Flickr

As the blogger Bill Wilson says (in the context of root cause analysis) “Learning without action is mere mental trickery, while action without learning is simply useless physical exercise”.  If knowledge management is to deliver more than mere mental trickery and to live up to its promise of adding value, then it must lead to action.

A few years ago we worked with a client who was developing a lesson learning system from projects. The collection of lessons has been going well, but the client had the firm view that lessons should be stored in a library that future projects could review if they wanted. For them, the knowledge would be stored “for future reference”.

Of course, few people have time to read through the lessons, and there are now so many lessons that reading through them is becoming more and more daunting. 
We are now helping the client to move to a different philosophy, where lessons are forwarded to the owners of the organisational processes, so they can continue to update the processes, procedures and guidance in the light of the new learning. This is “knowledge for action”, and if we assume that people follow the updated guidance, it should result is less “useless physical exercise” and to more efficient ways of working.

This philosophy is that wherever possible, every piece of new knowledge should lead to an action. The action might be;

  • Fix a problem,
  • Investigate further (especially if the learning is not yet clear),
  • Document a new procedure, process or guidance document,
  • Update an existing documented procedure, process or guidance document,
  • Update a training course or other training or e-learning material,
  • Circulate the lesson for others to decide on an action.
Communities of practice, as well, should focus on creating and managing actionable knowledge. Actionable knowledge can be stored on the community wiki, and includes

  • Advice and guidance
  • Good practices
  • Improved practices
  • Solutions to problems
  • Answers to questions
  • New approaches
  • Recommendations
  • Tips and Tricks
Non-actionable knowledge is
  • Interesting articles
  • Links to interesting articles
  • Musings
  • Quotes and aphorisms
  • Descriptions of what you are doing (unless you analyse this to bring out actionable learning)
  • Descriptions of what you have done (unless you analyse this to bring out actionable learning)
  • Large document stores
Communities that circulate non-actionable knowledge, or “knowledge for interest” are classified as Communities of Interest rather than communities of practice, the clue being in the title. 

CoPs deliver more value when they focus on solving the problems of the members than when they circulate “interesting links and ideas”. CoPs that operate through a Pull process – where members with problems or issues ask questions and receive recommendations and support from other members – know they are adding value.  Each answered question represents a solved problem; knowledge which the person who asked the question can immediately put into action.

So when you are sharing knowledge in a CoP, ask yourself whether you are sharing “something that others will find interesting” or “something that will help people do their job better” – something actionable.

And when you are designing lesson learning systems, make sure each lesson leads to action, rather than being retained “for interest”.

We recommend “knowledge for action” rather than “knowledge for storage” as being a far more effective system.

View Original Source (nickmilton.com) Here.

How to calculate the transmission efficiency of lessons learned

The lesson learning system should be a supply chain for new knowledge. But how do you calculate its effectiveness?

We can look at lesson learning as a supply chain; identifying new pieces of knowledge and supplying them to other knowledge workers so they can improve their work. If it works well, knowledge is gathered, accumulated, synthesised into guidance, and used to inform future operations.
But how well does this supply chain work, and how can you calculate its efficiency?
This is something I tried to measure recently with a client, using a survey.
I divided the lesson supply chain into three steps, shown below
  1. Capture and documentation of lessons from project experience;
  2. Update of guidance documents based on new lessons;
  3. Review of guidance documentation in future projects.
Now this particular client  did not have a clear supply chain for lessons, so many people accessed lessons through search.
I therefore asked survey respondents to estimate in what percentage of cases the following happened, giving them the options to selecting 0%, 20%, 40%, 60%, 80% and 100%. The average estimate from the survey responders is shown below.

  • Any given project documents lessons (46%)
  • Any given lesson is used to update guidance documents (32%)
  • Any given project makes use of guidance documents (43%)
  • Any given project seeks for documented lessons (44%)
  • Any given search finds useful lessons (36%)
We can then use these figures to estimate the effectiveness of lesson transmission, as shown in the diagram above, and described below.
The effectiveness of transmitting lesson through guidance is 46% (capture) x 32% (update) x 43% (review) = 6%.


The effectiveness of transmitting lesson through search is 46% (capture) x 44% (search) x 36% (find) = 7%.

These are pretty low figures! Even if both routes were independent and there was an overall success rate of 13%, that still means that 87% of project knowledge was never transmitted other than via human memory.

Let’s compare that with an organisation that treats lessons seriously. The diagram and numbers below are not from a survey, but from published statistics in another organisation. This organisation does not require users to search for lessons, but has a well-resourced supply chain to embed lessons into procedures and guidance.

  • Any given project documents lessons (100% – lesson capture is mandatory part of every project, and supported by dedicated resources)
  • Any given lesson is used to update guidance documents (93% – this figure was tracked on a quarterly basis and the “closure rate” for lessons varied between 88% and 95%)
  • Any given project makes use of guidance documents (100% – all activity was directed by operational procedures which were reviewed and use to make the action plans)
The overall transmission efficiency is 93%

Please note that these figures only reflect the documentation of lessons and their transmission to future knowledge workers; they don’t include the loss of knowledge in the documentation process, or the issues of transmission of understanding from writtenprocedires into the human braid – they only look at the effectiveness of the supply chain of written lessons.

This effectiveness can be measured, through surveys or through lesson tracking, and we can see that is can vary between very ineffective (as low as 6%) or extremely effective (as high as 93%).  If you are applying lessons learned, then aim for high effectiveness!

View Original Source (nickmilton.com) Here.

Why you should never leave your lessons in the project report

Probably the worst place to store project lessons is to leave them in the End of Project Report.

Tombstone created using http://tombgen.appspot.com/

I know this is still the default approach for many project organisations, and 19% organisations who run KM programs still use this approach (according to our KM survey). However this approach was linked to the second-lowest satisfaction score for lesson learning, second only to “we don’t store our lessons anywhere”. So leaving lessons in project files is only slightly better than not storing them anywhere.

But why is this the case?

It makes sense for the project team to leave the lessons in the report, because lessons are a project output, and surely all project outputs are documented in the report? Once the report is written, the job of the project team is done.

However although the job of the lesson writer may be over, the job of the lesson-seeker is only just starting.

People searching for lessons (whether they are people in future project teams, or subject matter experts maintaining process documentation) will be searching for lessons to help with a specific issue, risk, process or task. They would like to find all the lessons associated with that issue, risk or task in the same place. They almost certainly will not know which projects learned lessons on which topic. They don’t want to have to go back through 20 or 30 project reports, looking through the lessons learned section of each one, just in case there is something of relevance.

Therefore they don’t bother.

The lessons remain unread; the project report becomes a tomb in the lessons graveyard. Learned once, and destined to be relearned in future.

Two of the basic principles for effective lesson learning are that you need to 1) capture lessons individually, and 2) store them centrally. As I pointed out in yesterday’s blog post, lessons are a knowledge output from a project, and should be stored with the rest of the knowledge, not the rest of the project files. In order to make it easy for the lessons to be found, you need to store the individual lessons under themes or topics in a lessons management system until they make their way into updated practice and procedure, guidance and training.

 If lessons are being learned from many projects, for example from Retrospects and After Action reviews, then these lessons need to be stored somewhere where they can be compared, reconciled, and actions assigned, notified, tracked and managed. They need to be captured in a consistent format, and stored in a central system.

 This is often done in some sort of Lessons Management System, set up either for a major business stream such as Sales or Engineering or for the entire organisation, to store and track lessons from many projects.

An effective lessons management system should be structured according to the needs of the “knowledge user”. People who come to access lessons from the database should be able to find what they are looking for very easily. If they don’t find relevant and useful knowledge within a few minutes, they will leave and never come back.

Think about the needs and interests of the knowledge user, think about how the knowledge will be re-used, and think about how you should structure the database to most easily give them access to what they need.  The knowledge seeker is  looking for lessons about procuring steel pipe for example, or for all the lessons to do with safety while working at height, or for all the lessons to do with partnering with a particular national authority; in other words, lessons to do with the particular issue that they face. The lessons therefore need to be grouped into a structure,  index or taxonomy, based on work activity or work process. Think about how supermarkets gather and present produce, and use ideas such as this to stock your knowledge supermarket.

In addition, the lessons management system should be able proactively to route lessons and associated actions to the relevant process owner, in order that the lessons can be embedded in improved process, and can be declared properly “learned” and the learning loop can be closed.

Leaving them in the project report graveyard destroys the learning loop at the first step.

View Original Source (nickmilton.com) Here.

Lesson learning at NASA – video

From the AFAC lesson management forum last week, this video below from David Oberhettinger, NASA’s Jet Propulsion Lab, talking about lesson learning in their human and robotic space exploration program

David makes some interesting points, such as

  • the need to dedicate individuals for lessons capture (although NASA allows engineers to submit lessons themselves, he says he has never seen this happen in 25 years)
  • lessons need verification and quality control – particularly verifying the facts of what happened
  • lessons recommendations need to be “infused” into procedures and training to ensure closed-loop learning. The key documents are the JPL design principles and the JPL flight project practices, which are built up over time from the combined and synthesised lessons
  • weekly meetings of the lessons learned committee, for the past 35 years

View Original Source (nickmilton.com) Here.

Every lesson is a story

Capture your lessons as stories, that way they will be remembered more easily.

Native American Storytelling. Photo By: Johnny Saldivar

One of the way humans learn is through stories. Stories can Inform, Educate, and/or Entertain (transmit information, knowledge, and entertainment) and some of the best stories do all three.

A post from the Farnham Street blog entitled “The structure of a Story” tells us that a great story has structure, and includes Context, Action and Result; where Context includes Where, Who, What do they Want, and What’s getting in the way. As the blog says

“It all hangs on the central Context, Action, Result (CAR) structure. It starts with the Subject, Treasure, and Obstacle, and concludes with the Right lesson and link back to why it’s being told”.

When we transfer Knowledge, story is one of the best formats. That is equally true of lesson-learning as it is of the other elements of KM. Yet all too often, lessons fail to meet the test of a good story.

As the Farnham Street blog says, aspirant storytellers usually start with the Action, because that is the exciting part, and omit the context, and as a result the reader of the story or lesson cannot easily identify with nor internalise the learnings.

When we teach lesson-learning to organisations, we say that Every Lesson is a Story, and we recommend that the lesson has at least 6 components

1) Context. Here we explain what was expected to happen, but didn’t. This is the “What did we want” step of the story. It is amazing how many project teams or facilitators skip this step. Because “what was expected to happen” is so obvious to them, they expect it to be obvious to the reader, but of course it seldom is. 

2) Outcome. Here we explain what actually happened – the Action step of the story. What went wrong? What actually happened? 

3) Root cause. Here we explain WHY it happened – what the Obstacles were, what was missing that should have been there, or (in a success story) what was done to achieve the result. 

4) Impact. Here we explain or try to quantify the result, either negative or positive. 

5) Lesson. Based on the root cause, what is the learning for future projects? This is the “Right Lesson” part of the story.

6) Action to Embed.What action do we take to embed that lesson into business process, so mistakes are never repeated, and successes always copied.

A lesson with a story-based structure such as this takes about a page to write down or a couple of minutes to tell, compared to typical bullet-point one-liner lessons – told in seconds, forgotten almost as quickly.

However if we treat every lesson as a story, then we realise that it must be presented as a Narrative, with Context, Action and Result, concluding with the Right Lesson.

This is the simple structure that conveys the message.

View Original Source (nickmilton.com) Here.