The Dark Side of Metrics?

There was an interesting article in the New York Times a few weeks ago called "Are Metrics Blinding Our Perceptions?" In it, author Anand Giridharadas questions whether the prevalence of computers and dashboards have resulted in too many metrics. He wonders if ours is simply a world with too much data -- where metrics are distorting policy, altering the flow of charitable donations, and even driving how parents take care of babies.

It’s a good read, and raises some interesting questions. I think the issue boils down to this: yes, computers and the internet have made it easy for anyone to create dashboards and share metrics, which is why it's more critical than ever that metrics are presented with their correct context and that they are thoughtfully reviewed before using them to make decisions.

In years past, a business dashboard could have taken many "man-months" to create. It would have involved coming up with a well-thought out strategy with strategic metrics (be it for a government, company, or whatever), developing attractive reports with help from graphic designers, professionally printing and hand assembling the reports at great cost on a monthly or quarterly basis.

Today, you can take any old collection of metrics and make them look pretty good with a fraction of the effort. In short: any idiot can create a decent-looking dashboard these days.

But of course more measures, more dashboards, and more reports aren’t going to make the world any better on their own. Garbage in, garbage out holds true for metrics, dashboards, and any process in the world. There are lots of software tools that make it easier to make the garbage coming out look nicer, but from a business perspective, it remains essential to ensure that the strategy, objectives, measures, and initiatives that end up on a scorecard or dashboard are the right ones. And that there aren’t too many. And that they have owners responsible for their performance. And that appropriate goals have been set. All of the things that this blog is about.

So, yes, it's true that metrics can be taken out of context or relied upon too heavily without applying common sense. But what's also true is that when the right metrics are on a functional -- and, yes, ideally attractive -- dashboard, they can mean the difference between excellence and failure.

Just take a look at Datron World Communications. This fast growing, high performing organization has focused on reviewing and managing the right measures systematically on dashboards. Using this approach, they were able to drive incredible tangible business results. In fact, they just won the Overall Business Performance Leadership Award from Ventana Research.

Art Barter, Datron's CEO, confirms that this focus on a dashboard with the right metrics works: "Let’s be good at these three or four things. Let’s understand what impacts results in measurable ways, and let’s stay focused on those things, instead of trying to do 10, 15, or 20 things that we think might impact the results." He credits this approach with $40 million of new business, as well as higher levels of trust in the organization, and improved customer satisfaction levels. You can read the full story in an interview that BPM Magazine conducted with Mr. Barter.

What do you think about the pervasiveness of measurement in your organization? Has it gone too far? Or gone off-track?

Comments

[David Dougherty]

I remember a presentation by a field animal scientist who was decrying how the prevalence of statistics was being confused as the main activity of science. Her argument was that pure analysis needed to be tempered with a deep, first-hand experience and understanding of the domain that is being measured. That the marriage of the two was what is needed.
I think this applies to these sophisticated tools as well. Leaders who have experience in their market will know better what questions to ask and what numbers smell of ephemera.