The link between Big Data and Knowledge – the classic Walmart story

This is a short analysis of the story of WalMart’s preparation for hurricanes, in order to explore the link between Big Data and Knowledge.

Hurricane Irma Food Distribution
Hurricane Irma food distribution, by City of St Pete on Flickr

One of the earliest and most famous “big data” stories is around Walmart and it’s analysis of sales data related to hurricanes.  Let’s look at this story through the lens of “Observation, Insight, Lesson” to see the link between Big Data and Knowledge.

According to the New York Times in 2004 –

A week ahead of (Hurricane Frances) landfall, Linda M. Dillman, Wal-Mart’s chief information officer, pressed her staff to come up with forecasts based on what had happened when Hurricane Charley struck several weeks earlier. Backed by the trillions of bytes’ worth of shopper history that is stored in Wal-Mart’s computer network, she felt that the company could “start predicting what’s going to happen, instead of waiting for it to happen,” as she put it.

The experts mined the data and found that the stores would indeed need certain products – and not just the usual flashlights. “We didn’t know in the past that strawberry Pop-Tarts increase in sales, like seven times their normal sales rate, ahead of a hurricane,” Ms. Dillman said in a recent interview. ” 

And the pre-hurricane top-selling item was beer.”Thanks to those insights, trucks filled with toaster pastries and six-packs were soon speeding down Interstate 95 toward Wal-Marts in the path of Frances. Most of the products that were stocked for the storm sold quickly, the company said. Such knowledge, Wal-Mart has learned, is not only power. It is profit, too.

This story is a classic example of the power of Big Data, but let’s look at it through a Knowledge lens.  The Military model of knowledge development is that it comes through a 3-step process of Observation, Insight, Lesson, and that the Lesson then leads to Action. The New York Times suggests that the big data algorithms delivered knowledge, but I suggest that what they delivered was Insight, and that the Knowledge is applied when you know what to do with those insights.

  • The “trillions of bytes’ worth of shopper history” is a collection of Observations
  • The correlation of shopper behaviour with imminent hurricane landfall is an Insight
  • The lesson, on which action will be based, comes from an analysis of the insights with a background understanding of Walmart’s business and strategy. 

I would disagree with the New York Times. I think that what the data-mining experts found was Information in the form of a Correlation. The Knowledge would come in knowing what to do with that information (Knowledge, after all, being what enables correct action).

What would you do, dear reader, if you were a Walmart Executive who had been given this information on beer and pop tart sales? What action would you take?

  • Should you (as the New York times suggests) stock your stores with extra supplies to increase profit?
  • Should you (as this article suggests) put Pop-Tarts near the store entrances during hurricane season so that sales can “soar”?
  • You could even consider raising the margin on these items, following the principles of supply/demand pricing?

All of these solutions will increase sales and improve profit, but would Walmart really want to use a hurricane solely as an opportunity to increase profit? That might not go down so well with the public.  As Peter Drucker said, “Information only becomes knowledge in the hands of someone who knows what to do with it” and luckily Walmart has the knowledge they need to use this information wisely.

What Walmart actually do 

Walmart have built their own “best practice” for disaster relief (here, from page 177)  In the US this includes

 As a result WalMart is seen as a lifesaver, and it’s disaster response procedure has been compared favourably with that of FEMA. They have won the hearts of the public, and of the administrators.

Walmart are winning on three counts. They have plenty of observations in the form of shopper data, they analyse this data to derive insights, and they have knowledge (based on experience and codified in best practice) that allows them to take the correct actions. 

It is that knowledge that allows them to know what to do with the insights they mine from Big Data

View Original Source ( Here.

The moment KM was born – a story

This is a reprise from the archives – a story of when KM was launched at British Petroleum through an early meeting of supporters and enthusiasts.

It’s an extract from my book Performance Through Learning (available here), telling of a key moment in the history of the BP KM journey – the moment that KM was launched as a “movement”. I have highlighted that moment in bold below.

The setting is a basement conference room in Milan in the late 90s, where everyone in the organisation who was already doing something KM-related was brought together for three days, to share experience, to map out the collective way forward and to launch a corporate program.

“The room was too small for the number of tables, and the pillars made it difficult to see the stage. People needed to crane their heads to get a clear view, and the air was stifling from the heat of the video lights (the whole event was being captured on video, and web-cast live onto the BP Intranet). And yet there was a tremendous buzz in the room”.

Three days in Milan – 1998 

“The three days had been an intense experience, as KM enthusiasts from around the BP group had stood up on stage and described the approaches they were already applying to managing knowledge. 

“Ray King had told us of his online community of computer modelers, Tony Kuhel told us about the Olympus knowledge base in BP Oil, John Minge beamed in by videoconference to talk about knowledge sharing on Texas drilling rigs, and many others shared their successes and plans. 

“Then there were the outside experts – Colonel Ed with his ‘war stories’ from the Army, Larry Prusak talking animatedly and powerfully for an hour without notes, John Henderson with his view of the future Knowledge Economies. Finally there was the input from the rest of the company, as people all over the world used the Knowledge Management website to raise issues and ask questions, which were read out loud to the assembled delegates in Milan every morning”

The moment of commitment 

“It was now the afternoon on the third day. The buzz was still high. We had spent the late morning in breakout groups, working some of the critical issues which needed to be overcome if KM was ever to be a way of life for BP. Jim Shannon, a video producer from the Alaska office, was standing on stage reporting the results from his group’s discussion. 

“Jim explained how the top levels of BP need to articulate the scale of the challenge and the potential benefits, and then cascade the KM ideals down to the production floor. People need to be involved, and the Knowledge Management community, exemplified by the attendees at this meeting, needed to identify and understand those people they will depend on for success. 

“How many of you people attending the meeting” Jim asked “are willing to go back and become champions within your businesses, to evangelize and lead Knowledge Management? Stand up if you are willing to be an evangelist for KM”. 

“There was a moment’s pause, and then chairs began scraping back all around the room as every delegate rose to their feet in a silent movement of commitment to the KM cause”.


“After the meeting was over, the KM team travelled out to a restaurant on the shores of Lake Como where I led the group in our own retrospect of the conference. Each of us could identify our personal success factor. 

“For Nigel Gibbs, it was “’the sense, level and quality of community that we developed. Transforming 80 individual learners to a community of learners was the biggest shift I’ve encountered in such a session. The challenge now, for me, is to shift the community of learners to be a learning community.” 

“Barry Smale noted that we avoided the event being a ‘Show and Tell’ of KM Team presentations. “This was an objective in the organization of the event and worked extremely well since the delegates got ownership.” 

“Georgie Dicker recalled the continual buzz of enthusiasm in the room and how she ended up in a nightclub with a bunch of the delegates where they were still talking about Knowledge Management at 3 a.m. in the morning. 

“All of us agreed that Milan had been the right event at the right time and had sparked an excitement and commitment that would form the foundation of BP’s move towards a fully knowledge-enabled company”.

View Original Source ( Here.

Uplifting Story of how Nelson Mandela understood the value of knowledge

Here is another post from the archives – an uplifting story of the recognition of the Value of Knowledge from the Great Man himself – Nelson Mandela

Image from wikimedia commons

The text below is taken from the excellent book “playing the enemy”, by John Carlin – a book about the transition in power in South Africa from the old apartheid movement to Nelson Mandela’s new government, and the role that the 1994 Rugby World Cup played in this process and in the reunification of national spirit.

There is a little section in this book that made me light up as I read it, because it shows the importance Nelson Mandela places on knowledge and experience.

The scene is just after Mandela’s appointment as president, and John Reinders, chief of presidential protocol under De Klerk, and the former head of the prison service, is in his office packing up his desk.

 “I came in early that morning to collect my things”, Reinders recalled. “All us whites had applied for jobs elsewhere, sure that they would be asked to leave. Quite a few of us meant to go and work for De Klerk in the deputy presidency”.  

Reinders was packing away his mementoes of 17 years spent running the presidential office, when suddenly he was startled out of his reminiscences by a knock at the door. It was another early riser. Mandela. 

“Good morning, how are you?” he said, stepping into Reinders’s office with outstretched hand. “Very well, Mr. President, thank you. And you?” “Well, well but ….” Mandela said, puzzled, “what are you doing?” “I am collecting my things and getting ready to go, Mr. President”. “Oh, I see. And may I ask where you are going?” “Back to correctional services, Mr. President, where I used to serve.” 

“Mmmm,” said Mandela, pursing his lips. “I was there 27 years, you know. It was very bad.” He grinned as he repeated, “very bad!” Reinders, flummoxed, offered him a half-smile back. “Now,” Mandela continued, “I would like you to consider staying here with us.”  Reinders examined Mandela’s eyes with astonishment. 

“Yes. I am quite serious. You know this job. I don’t. I am from the bush. I am ignorant. Now, if you stay with me, it would be just one term, that is all. Five years. And then, of course, you would be free to leave. No, please understand: this is not an order. I would like to have you here only if you wish to stay and share your knowledge and your experience with me.” 

Mandela smiled. Reinders smiled, wholeheartedly now. “So,” Mandela continued, “what do you say? Will you stay with me?” Amazed as he was, Reinders did not hesitate. “Yes, Mr. President. I will. Yes. Thank you.” 

At which point his new boss gave him his first task: to gather together all the presidential staff, including the cleaners and gardeners, in the cabinet room for a meeting. The new president walked among them, shaking hands with each of the hundred or so people assembled, saying a few words to each, in Afrikaans where appropriate. Then he addressed them all. “Hello, I am Nelson Mandela. If any of you prefer to take the severance package, you are free to leave. There is no problem. But I beg you, stay! Five years, that is all. You have the knowledge. We need that knowledge, we need that experience of yours.” 

Every single member of the presidential staff stayed.

View Original Source ( Here.

Quanitified KM value story #120 – Petroleum Development Oman

Our list of KM Value stories reaches number 120, with this story from Petroleum Development Oman (PDO). See the complete list.

Image from wikimedia commons
The story was presented last week at the UK KM Summit by Hank Malik, who gave an excellent presentation on how standard project-based Knowledge Management has been developed and deployed at PDO.  The elements of KM Hank described included 
  • Lesson Learning, 
  • Best practices, 
  • Knowledge Assets, 
  • Content Management, 
  • Knowledge Retention and 
  • Communities of Practice; 
in fact all the core elements of project-based KM, plus a complete Governance layer and a Processes toolbox.
Hank described a whole set of acheivements from the KM program, including the following quantified benefits delivered to date:
  • 5,554 Lessons Learned captured to date across over 50 projects,
  • Over $740 M savings,  and $80M realised for in-country value.

Well done to Hank and the KM team at PDO!

View Original Source ( Here.

3 principles for an effective learning system (example from the Army)

Here are three powerful principles about building an effective learning system. 

The principles come from  iraqhis interesting article about CAVNET – an online forum for the junior leaders of the US Army 1st Cavalry.

It’s worth reading the whole article, but I share below some of the points made by Major Patrick Michaelis, the founder of CAVNET, who was Battle Command Officer and Task Force CKO for the 1st Cavalry Division in Iraq at the time the article is written (2005). These points are important, in helping define an effective learning system.

  • Firstly, Michaelis had a clear view of the purpose of the learning system. The enemy was adapting faster than the Army at the tactical level, and company-level leaders had little chance to physically interact. Therefore an online system was needed.
  • Secondly, he had a clear view of the scope – – “designed to prepare for the next patrol, not the next war”, and focusing on “actionable” (contextually-based) knowledge which could be incorporated into the patrol plan, prep, and execute cycle. This meant focusing on the HOW – the tactics.
  • Thirdly, it had to fit the tempo of life in the Army. As Michailis says, “One of my baseline evaluation criteria is that it had to compliment a commander’s battle rhythm rather than complicate it … It was my belief that if a commander could not get on, get information, post information, and get off within about 10 minutes, it would be useless”.

These three principles resulted in a simple but very effective way of sharing knowledge, over and above the AARs at unit level, and the development of Army Doctrine.

Does the system work? Michailis provides two anecdotes to illustrate the value delivered by CAVNET

“A leader posts a report that his unit experienced an IED that was cloaked by a poster of Moqtada al-Sadr. On the other side of the city, a commander taps into the CAVNET and reads the post. Though he is in another part of the city, he has been involved in operations that require removing posters posted on IIG [Iraqi interim government] projects. He briefs up his leaders before they execute a normal combat patrol. One sees a poster that mirrors the description given by the original post. Instead of ripping it down, he calls EOD [Explosive Ordinance Disposal], who discovers that it is rigged as an IED.

“In another instance, a scout platoon leader from 1-8 Cav.  was given the mission to conduct sniper operations. He had never really executed a mission like it before. He looked on the CAVNET, where a commander from 1-9 Cav, in another part of the city, had posted notes and TTPs from employment of snipers over the past months. The Scout Platoon Leader from 1-8 was able to integrate what he had read from the CAVNET into his planning, preparation, and execution cycle”.

View Original Source ( Here.