Thanks to Tom Raggett for this link from Joel on Software about the pitfalls of measurement when applied to knowledge workers:
It seems like any time you try to measure the performance of knowledge workers, things rapidly disintegrate, and you get what Robert D. Austin calls measurement dysfunction. His book Measuring and Managing Performance in Organizations is an excellent and thorough survey of the subject. Managers like to implement measurement systems, and they like to tie compensation to performance based on these measurement systems. But in the absence of 100% supervision, workers have an incentive to "work to the measurement," concerning themselves solely with the measurement and not with the actual value or quality of their work.
Is a worker who starts to "work to the measurement" ever likely to have concerned themselves with the "actual value or quality of their work" in the first place? I would say: only one who is easily jaded. And if they find themselves jaded, why aren't they moving on? My concern about such measurement is not in the dysfunction it engenders in workers, but in the dysfunction it suggests in the managers. Surely a manager has no need of such a system if they are truly in touch with the work they are supposed to be managing. Measurement systems and their kin are brought into play because something is inherently wrong. Managers who have lost faith in their ability to justify their own intuitive assessment of performance (ie who have become jaded themselves) are the ones who will grasp at measurement systems. If they stick to these, then no-one can accuse their own intuitive decisions of being sexist / racist or in any way skewed. A manager capable of being literate about his/her own intuitive knowledge of a situation, and who can defend their decisions, is one to be kept hold of. Managers that blindly accept performance measurement as just another quality system to be implemented, are jaded, should probably have moved on, but sadly, probably never will.
I totally agree. If you are not connected with the work you are managing and you require abstractions of this nature to follow it, then there is a problem. I guess, though, that it also depends on what you measure and how this is expressed.
I think it's endemic of the pyramid model of the 'manager's manager' - they need 'some way' way of 'benchmarking' and rolling up statistics to their boss (and boss's boss). I think this is most evident in the increasing prevalence of 'competencies' - where staff are grouped in arbitary sets and 'benchmarked'. Ugh. And obviously it doesn't work, competencies get commitee'd to become incredibly generic ("Shows ability to think creatively" etc.).
Reliance on these sorts of frameworks leads to a huge abdication of management responsibility and actually desire/ability to understanding what goes on with your employees, as David and Lee said.
The fix? I would cut out a huge swathe of 'managers' and push those responsibilities to the edge - I reckon most people can work co-operatively as a team and manage themselves and their performance. It does work - http://www.bbc.co.uk/worldservice/learningenglish/work/handy/semler.shtml
And empower employees to tell real stories about what they do and how they did it - people do cool stuff, but these stories always gets lost in 'the numbers'.