One of the pivotal experiences which provided the catalyst for starting this blog occurred in Fall of 2012 when I was living with my soon to be first wife and helping to raise her son. He was in his 1st grade and was struggling with a behavior to a degree where the school put him on a daily behavior report card system.
The report card consisted of him receiving a score card every day with each school subject and recess. Under each subject he would receive either a smiley face or frown face for behavior and work and a single face for recess. The report card allowed for a maximum of 13 Smiley faces though he typically received around 3-4.
After begging, bribing, scolding, pleading, and any other “Hail Mary” attempt to try to get him to behave I had a Eureka moment. After each day we attempted a different reward (bribe) or punishment in desperation to try and correct the behavior. Being an experienced process engineer I figured there would be no way to determine which input variable would lead to the desired output if we were consistently turning the proverbial knobs on a daily basis. I recognized the school had gifted us with a measurement system allowing us to trial a single incentive system and evaluate its effectiveness over a period of time and use statistics to measure the difference. This would prevent us from chasing “noise” and ensuring we would be able to identify real progress. Thus, we set a goal of 10 Smiley Faces per day and worked out an incentive program with his after school day care teacher.
Lo and behold the number of smiley faces increased (see the example graph below) and he drastically reached a point where he was averaging nearly 10 smile faces per day. Being the
obnoxious data savvy engineer, I performed a t-test and the resulting p-value was significantly below 0.05 indicating the change in performance was statistically significant. For the next several days I practiced my interview with Oprah and Dr. Oz while brushing my teeth as I had successfully deployed process engineering and management fundamentals to solve the fundamental challenge in parenting.
A few days (or maybe a couple weeks), I ended up picking up the boy from school and ran into his teacher. Proud of my accomplishments I had to ask, “How has his behavior been?”. Waiting in anticipation for an “Atta Boy” moment his teacher paused and said, “he is receiving a lot more smiley faces; however, by the end of the day he is a real terror”. Note: She likely put it more politically correct, but you get the gist.
Defeated, I had my first real world lesson in Goodhart’s law: “When a measure becomes a target, it ceases to be a good measure”. Children 1, Engineer 0.
Goodhart’s Law: https://en.wikipedia.org/wiki/Goodhart%27s_law