Careful — What You Measure Is What You Get

"You cannot manage what you cannot measure" is a common mantra of today's business world. But to really make a difference on projects, you also have to make sure you're measuring -- and communicating -- the right things.

A policy introduced to measure the performance of our local hospitals a couple years back offers a salient lesson. Our state government decided to incentivize hospitals by rewarding good performance and penalizing poor performance using a standard set of KPIs.

The plan created many vested interests:

  • The government's desire to look good by reducing patient waiting time
  • The hospitals' desire to achieve the maximum budget income for the year
  • The administrators' desire, both in the hospital and the government, to avoid rocking the boat 
But an audit found KPI-induced behaviors, in many cases, were worse for patients than if nothing had changed in policy.

For example, one key measure was the time patients wait in the casualty/emergency area before being admitted to the hospital. So to avoid a fine for failing to admit the patient within the prescribed maximum time, some administrators were transferring patients from emergency care to the operating theater's waiting area. 

The action meant a reduction in the level of care, with the patient being moved to an area with little monitoring capability. Throughput in the operating theater was also diminished due to overcrowding and skilled staff having to spend time on patient care rather than surgery.

The government had its data and the hospital system responded to the stimulus of the KPI, but everyone forgot the key objective: enhanced patient care. 

There are a number of important lessons in this story to consider when setting up project dashboards and the like:

  • The KPIs you choose communicate to stakeholders what you think is most important. What is easy to measure is not necessarily important.
  • What you choose to measure will change behaviors. Focus on things that matter, such as value and benefits, not easy-to-measure statistics, such as time and cost. 
  • Make sure the data is validated. 
  • A KPI system cannot solve the problem, but it can be a powerful facilitator of solutions if it's set to measure the right statistics and ask the right questions. 
Simply identifying a problem and creating a KPI is not enough! Work with the project team to make sure an effective solution is crafted and then measure the effectiveness of the solution. This is far more challenging than simply processing monthly reports on easily accessible information such as schedule performance, but it can really contribute to the overall performance of your organization. 

Finally, remember that if you pick the wrong KPI, you will get behavior changes, often times for the worse. It's better to have an informed conversation with key stakeholders over value and what really matters. 

What messages are you sending with the metrics you choose to measure success?
This entry was posted in Program Management, Project Management. Bookmark the permalink.

Comments are closed.