Photo

You get what you test for…

9/29/15 • 3 min read

This post was originally posted on September 29, 2015 on Sketchbook B. It’s reposted here as part of a project to move some of my favorite writing to my new site.

A few years ago, I stopped at a fast food restaurant in rural, upstate South Carolina where a car was circling the drive thru over and over and over. She’d drive up to the ordering station and wait a second. Then the car would drive to the window, pause, and then drive around again. This happened a couple of times. It was very strange.

When I walked in, I nodded toward the car at the window and asked what was going on. The woman shrugged, obviously embarrassed, and said “Just improving the drive thru wait times…”

Lots of people wonder how something like the Volkswagen ¹ emissions test scandal can happen. It’s actually very simple:

1. Set a specific goal.
2. Figure out a test to measure it: a survey, a timer, a standardized test, etc.
3. Make sure everyone knows how important the test is.
4. Provide rewards for people who achieve the test objectives. And penalize people who don’t meet the test objectives.
5. Smart people will eventually figure out how to do well on the test, even if it now has nothing to do with the original goal.
6. Management pats themselves on the back for great test results.
7. The test no longer reflects reality.

This cycle is everywhere in business today and manifests itself in different ways. Most of the time, it doesn’t involve fraud and cheating. But this process is why:

1. Engineers design products (cars, cell phones, etc.) that meet specific test objectives, but not real life usage.
2. Customer service reps in banks, car dealerships and retail stores remind you over and over to answer “highly satisfied” on a random survey.
3. Teachers teach to the test.
4. Managers make short sighted decisions to stay under budget.
5. Advertisers make really annoying online ads to improve click through rates.
6. Fast food workers fake drive thru response times.

If you connect success on a metric to an employee’s financial well being, that employee will change their behavior to meet the test objectives. And yes, sometimes, if the pressure is great enough, they cheat.

I’m sure those fast food workers wanted to have good drive thru response times to make their boss happy. I doubt there was a financial incentive. Maybe if the boss was happy, their life would be easier. Or perhaps they were afraid of being fired. Whatever the reason, they cheated to meet their test objectives. They lost sight of — or more likely, were never told — the reason for the test.

The test itself isn’t the problem. Testing and evaluation is a key part of management in a corporate environment. But today, it’s really easy to test and measure everything. Over reliance on metrics in management makes the test more important than the goal. And that never ends well.

You’ll simply get what you test for.

Bob Wertz is a creative director, type designer, Ph.D. student and researcher living in Columbia, South Carolina. He’s been blogging since 2008.