The Self-Optimizing Agile Organization needs metrics & measurements

For a while ago, I wrote about the characteristics of the self-optimizing agile organization.

In this post I’ll take a look at the role of metrics and measurements in general, and in a subsequent post on how the self-optimizing agile organization can (and must!) take advantage of a sound  (asop to today’s typically ignorant/naive “bean-counter flavor” ) usage of metrics and measurements.

In fact, without metrics and measurements it is impossible to optimize anything.

Let me start by stating that I’m by no means a proponent of today’s typical,  naive and clueless usage of metrics and measurements in business (or society in general), where management  (or politicians) measure “everything” and practices “management by spreadsheets”, a.k.a. “management by KPI’s”, (or “management by objectives”, which was the previous fad word).  The belief of many “modern” managers seems to be that “the more data, the better”, without taking into account the relevance nor the quality of the data. And now, with all the buzz about “Big Data“, I’m sure this trend will explode! The main problem with the “Big Data” approach is that the number of “false positives” will radically increase, due to the radically increasing number of spurious (i.e. random)  correlations in any massive data set. These spurious correlations result from the risk for confusing noise for signal whenever you are working with massive data sets, and as we all know by looking at e.g. the web or the media, the amount of noise increases much faster than the level of signal in modern society.  So, in the era of Big Data, extreme care must be taken to avoid the trap of believing that correlation always implies causalisation!

Unfortunately, unless you as a manager have a solid background in your business domain, you are pretty much forced to practice mgmt-by-spreadsheets, since you really don’t have the prerequisites to operate in a more effective and productive way. A “manager” who does not possess any knowledge of – or interest in –  the domain in which he or his customers operate, is to me a complete joke, an extremely well paid administrator instead of a true manager or leader. Unfortunately, in much of today’s business, these “empty suit” type of managers are much more frequent than those who practice “Go See”, i.e management by actively participating in “the trenches”.

Anyways, I’m getting carried away by my frustration with all the “empty suit”-mgmt types I frequently encounter, let’s try to get back to the topic:

The problems with metrics – Gaming the numbers

First, as we all know, metrics and measurements can be severely misleading (“lies, damn lies, and statistics”), either by ignorance or mistake, or by those involved deliberately gaming the numbers. A few examples:

  • I recently saw an article in Swedish newspapers that the number of train delays in Sweden has been declining over the past year.  It sounded as progress, until I learned that the reason for the improved statistics was that the definition of a “delay” had been changed from 5 minutes late to 15 minutes late – in other words, by changing the definition of the metric by increasing the range,  those responsible were able to demonstrate progress. 
  • Swedish Police, famous for their inability to resolve (or prevent) crime, can regardless of their high level of ineffectiveness demonstrate a high level of “success” by measuring activity (effort) by focusing their efforts on catching motorists speeding ever so little on empty freeways, or performing alco-tests outside the teetotaler HQ.
  • A large multinational in “technology, information & knowledge” business assesses the skills and knowledge by their staff by measuring the number of powerpoint presentations the staff has consumed during an annual “competence building” period.
  • A very successful football (soccer) player, known for his spectacular and high precision long range passes,  gets a new coach who employs the “scientific method” by measuring the success/failure rate of each pass the players make on the field. All of sudden the star player stops making any passes at all, to the great detriment of his and his team’s overall performance, because he got serious heat from the new coach for “his pass success rate being below bar”.

There are many more examples of misleading metrics, but those listed above should suffice to bring forward the message that extreme care must be taken when it comes to what is measured and why.

We can perhaps bring some clarity into the mist of metrics and measurements by an attempt to categorize metrics into a few groups:

  • metrics for activity ( e.g. “busyness”, efficiency)
  • metrics for process (e.g. compliance)
  • metrics for execution (e.g. skills, techniques)
  • metrics for performance (e.g. utilization)
  • metrics for results (e.g. financial targets)
  • metrics for business outcomes (e.g. customer value)

In a follow-up post, I’ll explain how the self-optimizing agile organization can apply the various types of metrics to improve its overall performance.

Advertisements

About swdevperestroika

High tech industry veteran, avid hacker reluctantly transformed to mgmt consultant.
This entry was posted in Agile, Business, Complex Systems, Leadership, Management, Organization, Systems and tagged , , , , , , , , . Bookmark the permalink.

2 Responses to The Self-Optimizing Agile Organization needs metrics & measurements

  1. Fredrik Ferm says:

    Your comment on the football player got me to think about the difference between metrics that are absolute numbers and metrics that are a quotient number.

    Usually, what you want is to achieve as much as possible of something positive (positive for who is a whole different topic worth exploring). But to set this number into perspective and compare it with something else it is usually divided by something else, which gives us a quotient number. Now, a shrewd player of the system has two variables to play with, which can give effects other than the ones we want.

    For the football player example, a much more interesting metric would be the number of successful passes (absolute number) instead of the success rate of the passes that he tries (quotient number). Who cares how many passes he tries to do as long as he delivers a significantly high number of successful passes…

    • Very true! Unfortunately it seems that many bean-counter types have a hard time understanding the difference between absolute and relative numbers… I’d be more interested in making a profit of 10.000$ on an investment of 100.000$ (10%), than making a 100 % profit on an investment of 1$… Thus, the larger absolute gain (10k asop to 1$) appeals much more to me than the relative numbers… 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s