Measurements are great - except when they are poorly created.
What makes an effective measurement?
It should be objective. There can be some subjectivity, but it should be well defined.
It should be something that can be measured regularly. Something that you measure once and then can’t measure again for months isn’t effective. You want to be able to see progress.
It should be controllable. It has to be something that the individual or team that is responsible for the measurement directly impacts with their performance.
A recent sports story is a perfect example of a bad measurement.
If you aren’t familiar, the University of Iowa football team did not do well last year when it comes to scoring points. They didn’t want to fire the Offensive Coordinator (no surprise since he is the head coach’s son), but they didn’t want to pay him all that money either. What they did was force him to take a cut in pay, and if he met certain metrics, he could earn his higher salary again.
While that story sounds messed up - it gets worse.
The university decided on 2 metrics that they would use to make the determination. The first was average points per game and the second was wins. If the team averaged over 25 points per game and wins at least 7 games, his higher salary is restored.
Let’s look at the criteria to see if it fits.
Objective - points and wins are objective. You either get the points and wins or you don’t.
Regular - the metrics are regular as you can see the results after every game.
Controllable - this is where the whole thing falls apart. The contract simply states points per game is the metric, regardless of how they are scored. Let’s say the defense or the special teams has a great year and scores a ton of point - that all counts towards the metric. The offensive coordinator has nothing to do with those aspects of the game, but could benefit from it. In terms of wins, while a team needs to score points to win, there are a lot of other factors that come into play. Let’s say the offense does great and averages 50 points per game, but the defense is horrible and gives up 55 points per game. That could lead to 3 wins which would hurt the individual doing a great job.
The above doesn’t even take into account other factors. Maybe one game is played in a snowstorm where the final score is 3-0. Not only will that hurt the points per game average, but they could easily lose that game.
How about if several starters and key players get hurt in freak accidents. Can we really blame the coach for not having the best players available?
As you can imagine, this list of variables that are beyond the coach’s control could go on for quite a while. Any of them could help or hurt the coach in meeting the metrics. The only thing they have in common is that the person being measured has no control over them.
The lesson - when selecting metrics to measure, particularly if they impact pay or even whether someone keeps their job, make sure they are effective and appropriate.