What Gets Measured Gets Improved (Or Not)

Conventional wisdom says that measurement is a necessary part of improvement, so you can track your gains. Sometimes, just the act of measuring something leads to improvement because the people doing the work will put forth more effort. However, sometimes measurement can actually thwart improvement efforts in ways quite unexpected. Don't let your attempt to measure your results get you the wrong results.

We will look at measurement from two viewpoints:

1. how to establish a measure that will work, and

2. how to correct a measure that is producing undesirable results.

It may seem that performance measurement should be fairly simple to do, but there are a number of factors which can lead not only to incorrect measures, but often incorrect actions.

Nearly every manufacturing company is interested in keeping scrap low, so they measure scrap daily. In many cases the source of the scrap data is a person writing a scrap ticket. Since that person knows that low scrap is desired, they will often “underestimate” the amount of scrap, especially if they will get in trouble for a large quantity of scrap. This can lead management to believe there is minimal scrap, only to find the inventory loss is going through the roof. This is a game that is played in many organizations, whereby everyone publicly conspires to falsify scrap performance, knowing that honesty was far more painful than deception.

I used to run a production department in an automotive metal stamping plant. It was easy to “hide” scrap metal parts by simply chopping them up and sending them out to the bailer with metal trimmings. One day my boss confronted me at the beginning of the shift with, “We have over 2.5 million in factory loss year to date. Do you know what that's from?” “Sure,” I responded, “we're cheating on scrap.” I was instructed by my red-faced boss to do something about it right away and met with each of my supervisors to stress the need to report scrap accurately.

My boss previously had held my job, where he had a reputation for running a tight operation with no tolerance for scrap. He would have each one of his supervisors bring their scrap containers to the main aisle and would proceed to chew them out for their scrap. It didn't take very long before the scrap containers showed up empty or with only a few parts, often attributed to the setup person. My boss thought he was making a difference, but he was simply teaching his supervisors to lie to him.

As a result of this lying, significant scrap problems were neither recognized nor fixed because the overall scrap appeared so small. I eventually got our scrap numbers to be reported honestly and did take a lot of heat from multiple sources, including an accountant who had to explain to the corporate office just why our scrap had increased so much in the last month. He found that my explanation was not suitable to report to the home office.

“What Could Go Wrong?”

When a measurement is imposed, especially when people will be judged (or compensated) based on that measurement, you can expect things to change. If you aren't quite careful about it, the change will not be what you wanted.

I was assigned the challenge of increasing on-time shipments at a furniture manufacturer. This company had six plants and a distribution center (DC) in west Michigan. Typically a customer order was split between multiple plants, which would send their product to the DC for consolidation and shipping. The manufacturing plants were measured by percent completions, which were typically 95% or better. The DC was measured by on-time shipment percentage, which often ranged between 65-75%.

Since the plants were doing so well, I decided to start my investigation at the DC and discovered that on Monday mornings the DC had over 98% of the product it needed to ship, but could not begin shipping most orders because there was something missing from almost every order. As the week progressed, more orders were shipped as the plants would send their delinquent items.

Something seemed very suspicious in the plant reporting, so I obtained reports of all of the missing product from the plants and reviewed them with each plant manager. They all agreed that the missing product was correct. When accounting for the actually missing product I calculated 85% completions, yet the plant was reporting 98%. Why? They made “adjustments” for situations that they felt were out of their control, which amounted to nearly everything. They viewed the completion number as a “blame number” rather than an honest measure of performance. I had uncovered a “dirty little secret” that resulted in some painful explaining and some stern redirection from their VP.