IT and baseball: no silver heuristics

Along the lines of my last post that discussed avoiding slogans and “lazy thinking” in IT, let’s talk about the increasingly popular word “heuristic”. I think we can all agree that developing software is anything but simplistic. So why aren’t we more skeptical when people propose adopting simplistic heuristics for developing software? Let’s look more closely at this manner of thinking, with a specific example.

In a recent exchange, a #NoEstimates advocate declared that one example of someone making a decision amidst uncertainty, without estimating, was the act of catching a fly ball. My response was that there are in fact many estimates involved in that activity, whereupon the #NoEstimates advocate put forth essentially the notion that a fielder uses the following heuristic instead:

“One good way to catch a fly ball is to hold up your gloved hand at a constant angle, and run so that the falling fly ball is directly aligned with your glove. If the fly ball appears above your glove, it is going to go over your head: move back. If it appears below your glove, it is going to fall in front of you: move forward. Left of glove: move left. Right: move right.”

But really: watch any major league baseball game and look for a single instance, just one, where the outfielder is doing anything even vaguely resembling the above. (However, one can certainly conjure up the specter of hapless Little Leaguers, coached in this #NoEstimates heuristic-driven technique, desperately waving their gloves back and forth in front of their faces, flinching while fly balls thud to the ground all around them). You won’t find a real-world example of the above-described heuristic, because using just that heuristic will not lead you to predictable success in catching most fly balls.

In fact, you don’t need to be a major leaguer to understand that there is a whole host of empirical observations and, yes, estimates, engaged in immediately and in an ongoing fashion by our model outfielder when confronted with “the crack of the bat”. These activities are based on accumulated knowledge and experience, as well as evaluating current parameters. Here are just a few of these assessments/estimates, none of which can be dispensed with via the substitution of a heuristic:

• Where am I right now: did I already “play in” towards the infield, based on my earlier estimate of where this batter typically hits the ball?
• Is the ball headed my way? Do I start running immediately? Which direction?
• What’s the ball’s speed, direction, spin, trajectory? There’s no “one size fits all” here: left fielders know, for example, that balls hit to them by left-handed batters tend to behave differently from balls hit by righties.
• Should I adjust for wind, thinking about a similar situation in the first inning, where a fly ball like this plummeted suddenly because of high winds?
• Do I need to call off our hotshot center fielder who might run into me while I’m trying to catch the ball? Or does the center fielder maybe have a better chance of catching this ball than I do, and I should back off?
• If I run as hard as I can to make the catch, will I run into the wall? (in essence, a cost/benefit evaluation)
• My knee is still throbbing from yesterday’s slide into second. Can I viably run at full speed?
• Can I dive for the ball, or do I need to position myself for an immediate throw after I make the catch, to try to nail a runner who’s off the bag or who will attempt to advance after the catch by “tagging up”?

Fly ball situations are complex, in other words. They’re simply not solvable by a sole magic heuristic, and neither is software development. In complex situations requiring judgment and uncertainty, there aren’t easy shortcuts or silver bullets. Judgment calls require, well, judgment. There aren’t many ways (if any) to avoid putting yourself on the line. But there are most certainly ways of deluding yourself that you can escape accountability for judgment calls by using some kind of pseudo-automatic “paint by numbers” approach. It’s once again the seductive lure of the oh-so-easy answer: the no-muss, no-fuss heuristic.

But a heuristic, at least in most cases, just isn’t sufficient as a replacement for judgment amid complexity: you don’t get something for nothing. Insistence on the efficacy of a heuristic can often serve as an example of what H.L. Mencken famously said: “there is always a well-known solution to every human problem — neat, plausible, and wrong”. 

#NoEstimates advocates, who I’ve observed quite often gravitate to selective redefinitions of basic words and concepts to support their arguments, might insist that the fielder trying to catch the ball is “assessing” various factors, not really “estimating”. But that’s mincing words: the fielder is most definitely amassing (even if implicitly, instinctively) a series of judgments and weighing possible reactions, based on both new data and historical knowledge of the situation, judgments that will figure into the many decisions that she has to begin executing immediately. She is constantly tuning and modifying those judgments as the play transpires: she may suddenly decide to dive for the ball, or, recognizing that making the catch is unlikely, to pull back so the ball can drop in front of her and she can then be sure to hold the runner to a single. She is, in fact, estimating, in an uncertain, complex situation: juggling cost, benefit, impact, risk. Because, after all, that recurring pattern of assessment, decision, action, and adjustment is really all that estimating actually is.

Think about when someone says, “the outfielder misjudged the fly ball”: no, it isn’t the case that the outfielder somehow failed to apply a heuristic. Rather, the outfielder failed to estimate appropriately at the start, and/or failed to adjust her original estimates well enough to make the catch. And, do note that there’s little chance of that outfielder making the catch at all if she decides that it’s useless in general to engage in all those complicated acts of estimating I’ve enumerated, based on someone having told her that the heuristic is all she needs.

It should be clear by now: other than in the brevity of the time span involved, the process of catching a fly ball has numerous parallels to building software to achieve a functional goal:

• In software development, you’re a professional who’s done this many hundreds or thousands of times before; faced with a new request, you make dozens of hourly judgment calls and trade-offs, as you explore, design, and start to build what’s needed.
• You decide, based on new data and historical knowledge and experience, what parts of the job are easy, what parts are risky, what parts will require extraordinary effort, etc.
• As new information becomes available during the process, you adapt and change your approach as necessary, but your aim is still to catch the darned ball.

There may of course be situations in software development where heuristics can be of use. But when people insist that a heuristic will suffice to solve a tough problem, be at least skeptical. Consider if they’re actually presenting a form of silver bullet as the answer for a complex situation.

In short, don’t wave your glove at a fly ball just because someone has told you that it’d be low effort, low risk, or maybe even the One True Way to succeed.

Lagniappe:

• Daniel Kahneman, Thinking, Fast and Slow, 2011.
• “Outfield 1 – Pro tips for how to play outfield and how to catch a fly ball”