How to Measure the Hard to Measure: Part 1 - Project-based Objectives

If you are a frequent reader of The Glue, you know that we constantly espouse the importance of linking and monitoring strategic projects to the business objectives they support.  So, to "Improve Claims Resolution" we might link the "Implement Automatic Claims Adjudication System" project and include a review of its status in our business reviews.  Straightforward, makes sense, and gets results.

Typically in this situation one would have the accountable owner of the project make frequent, brief, and to-the-point status reports. Ideally, their reports would consist of:

• the timing, budget, and quality status of the project (red=late, over budget, below quality, etc.)
• Highlights of milestones completed on time
• What is late, why, and what we are doing about it
• Potential upcoming problems

There are times, however, when completing the project is really viewed as a business objective in and of itself.  One example of a project as a business objective is when a project is so big and strategic to an organization across so many different levels, that its completion is viewed as a top level strategic objective.  A huge, transformational, IT project that touches many parts of operations might be an example.  Completion of a integration plan after a merger might be another.

Keep in mind here that I am not talking about project management. What we are interested in is boiling down the project (or program*) status from hundreds or thousands of tasks, milestones, budget lines across tens, hundreds, or thousands of people. 

When it comes to the boots-on-the-ground details of measurements for projects, there are tons of sources.  William Malek's new book offers great insight to executing projects within a strategic environment and our friends over at PowerSteering Software offer a nice package to make the whole process easier and better.

All of the following measurements are useful when looking at large complex projects during a strategic business review:

• milestones on time
• projected days after/before final (or other important) milestone
• cost vs. budget
• delivery quality
• component projects by stage gate vs. plan
• % milestones planned complete that actually are complete

There are also LOTS of ways to take these and other project measures and come up with a synthetic index representing project health, a "project score" of sorts.  Though useful to an extent (watch for a post on index measures later in this series) I think the best project snapshot measure is much simpler. 

In my view, the best indicator of the status of a complex project is the professional judgment of the person responsible for delivering the project or program of projects. Now this isn't to say that all of the project measures above are not important. They are. In fact, some of the project measures should be on the strategic objective for the project. 

But at the end of the day, the person responsible for the project should have the best handle on the project.  Their judgments of status on timing, budget, and quality can be converted to numbers or stoplight indicators as needed by a specific scorecard scheme and included in the business review.

I am also not saying that the project measures should not be included in the business review.  In fact, reconciling the project measures to the owner's status reports and explanation is what the business review should be all about. Looking at the trends over time of the project measures and what the owner is saying about performance should also be a key component of the business review. 

Relying on someone's professional judgment to accurately represent the status of a critical strategic project might seem a bit "soft". But the fact is, with many of the critical project measures visible as well on the scorecard, disconnects between the owners professional judgment and the "hard" measures of the project WILL become evident, if they exist. 

At the other end of the spectrum is the hundred (or more) page project "status" report, presented to management.  I've seen organizations working billion dollar projects with a staff of twenty putting together such a document (with an accompanying 60 slide presentation).  Three things strike me about such an approach:

1. No Senior Executive will dedicate the many hours to carefully review such a document
   
2. Without a careful review, it is impossible to get a good feel for how the project is going
3. From an accountability perspective, it is really easy to hide when things are not going well

So, the trick in designing your project oriented strategic objective and measurement scheme is to pick the right trade off between the huge "dog-and-pony" project status report and the carefully reviewed (as part of the larger strategic business review) professional judgment of the person accountable for delivery.  My suggestion: trust but verify.   

*Program in this case means a grouping of many related projects.

Comments

[Project Management Software]

Relying on someone's professional judgment to accurately represent the status of a critical strategic project might seem a bit "soft". But the fact is, with many of the critical project measures visible as well on the scorecard.

[Smith Marz]

Web based project management software can increase the productivity and effectiveness of the organization. As such software can be accessed through the Internet. So, there is no requirement of desktop installation or updating. Project management tools are specially designed to organize all your projects’ activities along with productivity effectiveness.

[Ken Masuda]

Your blogs have been very informative. Regarding projects, I would say that professional judgment plus data is what is needed.
One basic chart that we use has time on the Y axis, cost on the X axis and graphs the project charter cost and time, and the actual and projected costs and time. It's a basic, visual, how are we doing display.