Project Nesso, Chapter 3: Troubleshooting

“The loftier the building, the deeper must the foundation be laid.”   Thomas à Kempis (http://www.brainyquote.com)

Some of what my doctors prescribed in 2014 made a lot of sense.  You’re having pain in your head – let’s take an MRI and see what’s going on in there.

But the approach to medication seemed very rudimentary.  Let’s try drug A and see if it helps.  No?  Let’s take you off of A and try drug B.  No?  On to drug C.  Trying every possible drug that has been shown to help a headache one by one seemed terribly inefficient.  It began to dawn on me that there is a parallel practice in software development, specifically in the area of troubleshooting.  I began to wonder if the lessons I’ve learned there could apply to medicine.

Troubleshooting bugs in software typically involves a few different tools in a developer’s toolkit:

Tool in the Toolkit What does it look like in practice?
Gathering Facts Step through the problem with the client.  Determine if I can reproduce it.
Analysis Can I find a pattern in when the issue occurs, and what the rest of the system was doing when it occurs?
Research Look online for other people who have had this issue, and what they did to solve it.
Experimentation The client did X, Y, then Z to get the issue.  What if I changed the order, or skipped Y – do I get the same error?
Collaboration Talk through the issue, my tests, and my theories with another developer.

When all of those techniques fail, I’m reduced to what I’ve termed “code roulette” where I just try changing things almost at random, and see what the outcome is.  It’s like Experimentation, but without a theory to guide it, which makes it the least scientific and least efficient way to fix a bug.  Historically, this approach is far less likely to lead to a solution to (or even insight into) the problem.  As a result I only pull it out of my toolbox when I’ve exhausted all of my other options.

How does the approach to troubleshooting recurring headaches stack up against troubleshooting a software bug?

Tool in the Toolkit Software Development In Practice Headache Treatment In Practice
Gathering Facts Step through the problem with the client.  Determine if I can reproduce it. Do the headaches occur every day?  Do you notice the headaches getting stronger as the day goes on?  Does your family have any history of headaches?  Let’s get you in for an MRI to see what’s going on in your head.
Analysis Can I find a pattern in when the issue occurs, and what the rest of the system was doing when it occurs? Can you keep a headache journal, to track when you get headaches, and how bad they are?  Is there a pattern to when they become more severe?
Research Look online for other people who have had this issue, and what they did to solve it. Read the medical literature to see what the research has to say about them.
Experimentation The client did X, Y, then Z to get the issue.  What if I changed the order, or skipped Y – do I get the same error? If you get more sleep, is your headache the next day better, worse, or unchanged?  Would cutting processed meats out of your diet affect them?
Collaboration Talk through the issue, my tests, and my theories with another developer. Let me refer to you to a specialist.
Roulette Let me change each setting, one by one, until we find one that fixes the problem. Let me prescribe each drug for you, one by one, until we find one that is effective.

Not only is the roulette approach inefficient, the feedback look with medication is a couple of orders of magnitude longer.  With code roulette, the time between trying something and seeing the effect is usually measured in seconds or minutes; with “drug roulette” it’s measured in weeks.

This analogy had been brewing two years, but it really solidified in February 2017.  It drove me to ask what I believe is a foundational question: is this really the state of the art for prescribing drugs, or was this just one patient’s experience?

Advertisements