Measuring knowledge management programs

Organization-wide knowledge sharing programs require significant investments and will entail major management effort, as well as behavioral changes throughout the organization over a significant period of time. In large organizations, an underground current of information sharing and water-cooler conversations will provide a running commentary on the progress of implementing the initiative.

In this flow of anecdotal information, it will be difficult for anyone in the organization to get an accurate sense of what is happening, given the likely large scale of activity, unless systematic efforts are in place to provide reliable information on both the progress and shortfalls in performance. Without measurement, there is an ever-present danger of premature abandonment of successful efforts, or alternatively, of complacent continuation of unsuccessful efforts when course correction is needed.

Putting in place a system for measuring progress will therefore be an essential step for a sustainable knowledge-sharing program. The organization must be prepared to accept some ambiguity, or at least to rely on non-traditional measures, when it tries to evaluate the impact of knowledge-sharing. Measuring that impact, either in terms of return on investment (for private companies) or development impact (for public sector institutions), remains problematical. In principle, inputs lead to activities, which generate outputs, which in turn produce outcomes, which in turn result in overall impact. But each link of this chain poses its own measurement difficulties, and measurement becomes the art of the possible.

In the early stages of the program, the focus will inevitably be on inputs (such as dollars invested, or staff recruited) and activities (such help desks or communities established, or knowledge resources made available).

As the program gathers momentum, the focus of measurement will increasingly shift to outputs (numbers of queries responded to, amount of material downloaded from the web, or usage of electronic tools) and outcomes (such as changes in turnaround times or unit costs, or comparisons with competitors).

Ideally, one would like to be able to go on to measure the impact of knowledge sharing, but it is important to recognize that few, if any, organizations, have been able to establish the clear causal links between inputs and outcomes that would convincingly demonstrate impact.

One can show correlations between inputs and outcomes, but the causal links between the inputs and outcomes are particularly difficult to ascertain in a corporate environment when many factors and changes are simultaneously at work, and the management is trying to integrate the elements, not differentiate them.

There is thus a measurement paradox: the more the organization is successful in mainstreaming knowledge sharing as the normal way of conducting the business of the organization, the more difficult it will be to isolate the impact of any particular actions or expenditures in knowledge management. Nevertheless, the measures of inputs, activities, outputs and outcomes can go a long way to reassure skeptics that the effort to share knowledge is worth it.

Are the difficulties in measurement a fatal flaw in knowledge management? Not necessarily. Once there is a realization that the choices facing a global organization are binary - either to share knowledge or to die - the task becomes, not one of justifying whether to undertake knowledge sharing, but rather how to conduct it more effectively.

There is a need for better measures to track progress, but measurement again is the art of possible, rather than the application of old measures to new kinds of activities: As Karl-Erik Sveiby says, "If we measure the new with the tools of the old, we will not "see" the new. Knowledge flows and intangible assets are essentially non-monetary. We need new proxies."

Nevertheless efforts to apply old and inappropriate measures to knowledge as a thing that can be measured continue unabated. Knowledge managers will be expected to talk of knowledge as something that can be valued (even if not absolutely) and tracked by accountants as a form of intellectual capital, something that can be traded for a certain value, something that accountants can put on the balance sheet and track from quarter to quarter, a resource that can be extracted from stores of information, the way a precious mineral is extracted from a rock. Whatever the truth of the matter, they will be pressed to present knowledge as though it is an object, rather than what it really is, namely, a living, dynamic volatile aspect of human activities.

References:

http://www.sveiby.com.au

Stephen Denning, The Springboard: How Storytelling Ignites Action in Knowledge-Era Organizations. Boston, London, Butterworth Heinemann, October 2000.

Stephen Denning: The Leader's Guide to Storytelling (Jossey-Bass, 2005) chapter 8.

Stephen Denning: The Secret Language of Leadership: How Leaders Inspire Action Through Narrative (Jossey-Bass, October 2007)