this space intentionally left blank

January 2, 2008

Filed under: tech

The Measurement Obsession

Raymond Chen's post on the Old New Thing about "if you can't measure it, it doesn't exist" sounded awfully familiar to me. He comments on one person's attempt to add metrics to their blog:

This smells like "I must make this quantitative and measurable so I can make it a review goal to increase my blog's 'impact' by 25%." In my opinion, blogging isn't like that. Blogging is more about creating an atmosphere. Sure, individual entries may solve specific problems, but the cumulative effect is the goal. Using a survey to measure the impact of a blog entry is like having somebody fill out a survey after you give them a ride home because you want to determine the impact that one action had on how nice a person they think you are.

Questions about measuring the impact of blogs will never go away because Microsoft is all about measurement. Many people believe that if you can't measure it, then you can't claim on your annual performance review.

Of course, it probably sounds familiar to you too, because it's not strictly limited to Microsoft. Sadly, I think smart people are particularly susceptible to this thinking. The Bank is a lot like this, in my experience: how do we measure our results? they constantly ask. Managers are capable of turning that question into a 14-hour business retreat. And are they actually measuring what they technically wanted to measure? For example, is a survey really the best way to evaluate a learning program? Given the Bank's reputation, will anyone actually rank your lesson plan badly, for fear of losing funding or offending an important institution? In some cases, people might be aware of these problems--but it's politically useful to pretend that they don't exist.

Or we can bring it back to blogging. I don't know how I measure excellence here, exactly, but it's not by comment count. If it were, this would be a very different blog, because I get a much higher response when I mouth off about the hot tech issue of the day, or focus on some geeky, easily-linkable pursuit. I don't usually consider that kind of thing my best work--but if you think that seeing those comment counts doesn't make me sometimes rethink what I'm going to write, you're wrong.

Often measurement is a good thing. But sometimes it gets in the way of doing good work in the first place, or it encourages people to aspire to good measurements, instead of good results.

Future - Present - Past