“Refuse to lose”: how executive pressure contributes to IT failure

“We went live before the system was ready”.  It’s a common excuse/explanation that I hear from IT people when they tell war stories about system launches that failed miserably. Implicit (and sometimes explicit) is the add-on statement: “and we told them so beforehand, too.”

There are obviously many things (and many parts of the org chart) that contribute to a failed launch, but here I’d like to focus on what drives this particular kind of launch-before-readiness, where the views of the rank-and-file are unheard or ignored.

In a nutshell: it’s management pressure. Sometimes that pressure comes from middle management, sometimes from the very top, and often from both.

Why do executives apply the pressure? There are a number of reasons:

• They actually believe it’s the best way to keep the team producing and working hard. They fear that extending the deadline will cause a let-up.
• Their own bonus and/or job is on the line if there’s a delay. They, or their management, regard a launch delay as a failure, just as much as a bad launch. At the very least, a delay means losing face.
• Testosterone (for either gender) sets in: a “refuse to lose” mentality. To some extent, that’s what has gotten them where they are.

Deadlines matter, of course, a lot, and people who are lackadaisical about deadlines simply shouldn’t work in IT. That said, if you bull-headedly insist on adhering to deadlines, to the exclusion of facts-based input, it makes you all the more likely to delude yourself that you’re really ready, just because the launch date has arrived and the pressures are heavy. If the stakes aren’t high, that may even be fine. (But, are the stakes ever not high in these situations?)

Let me tell three anecdotes that illustrate management “refuse to lose”:

• Earlier in my career, I was a middle manager, wrestling with a project that was both controlled by and staffed with a single “Big 5” consulting vendor’s people, with millions of dollars per month of consulting costs being racked up.  The high-profile initial system launch was a week away, and according to the bug tracker, there were still over 100 open major bugs. I sat down with the vendor’s partner-level project head. I don’t remember the precise conversation, of course, but it went something like this from him: “Oh, we’ve already got 39 of those 100 fixed in dev, 44 others are being worked by our best people and will be resolved by tonight, so really, there are only about 15. Four of those we think have workarounds so there are something like 10. I don’t think we’ll be getting many more bugs. We usually resolve bugs in a day or two, so even if each one takes two days, we’ll be able to close those ten and be ready for launch in a week.” I sat in awe (and frustration) at this hand-waving, this magic with math, and at his reluctance to think things through without an agenda. Here, “refuse to lose” had developed blinders, simple math had turned fuzzy and infused with vested interest, and IT best practices (not to mention common sense) had flown out the window.

• I worked as VP of Operations under a dev-oriented CTO. We had a late night launch, and the CTO was there in person, hovering over my sysadmins who were attempting to install the latest release into production with little success. The installation had failed, starting with the very first run of the script. “No, try this: run just those three lines of the installation script we provided you, then type these two commands, then run these other five lines.”  When that didn’t work, he came up with another idea, and then another. The next day, he sent out a company email declaring victory and congratulating his dev team, because the system had come up finally, even though it had run for a total of only two hours before crashing hard. Installation by seat of the pants. Refuse to lose.

• I arrived on the scene as the new CTO of a company in the midst of a complete replatforming project, completely retooling and replacing their sole product. The project had been underway for about nine months already; just before the point of my arrival, the team had been told by senior management that their launch date was going to be in six weeks, no ifs, ands, or buts.  I proceeded to do what I call a quick “temperature check”: I asked to see key indicators of readiness, such as bug counts over time and test case execution reports.

As you’ve surely guessed, I quickly determined that the facts didn’t even remotely support the possibility of a launch in a mere six weeks. About 40% of the code was still to be written; less than half of the test cases had been executed at all, with an even lower percentage having passed.  I marshaled my facts and figures, and went in to tell the CEO of this sorry state.  Judging from my experience, I told him, it looked like it would be a number of months, not weeks, before the product could be successfully launched. I remember him sputtering “that’s unacceptable”, and pressuring me on what I could do to, yes, bring off a launch in six weeks. (Cutting to the end of the story: we launched, successfully, nine months later).  In this case, a mixture of testosterone and probably some job-on-the-line aspects were driving this CEO to ignore facts and look for ways to bull through to success despite obvious counter-indicators.

The tragedy of the space shuttle Challenger has become almost cliché along these lines: a combination of executives feeling pressure and ignoring communication from lower ranks about major problems. “They also disregarded warnings from engineers about the dangers of launching posed by the cold temperatures of that morning and had failed to adequately report these technical concerns to their superiors.”  For whatever reasons, NASA management gave the green light for launch despite the adverse weather conditions. In essence, they “refused to lose”, and by doing that, they lost in a major way.

Takeaways:

• Don’t let your executives turn you, and don’t let them become, riverboat gamblers, where failure is a strong possibility through a premature launch. The company culture must be shaped so that it views a spectacularly failed launch as much more onerous than a delayed launch.
• Don’t let “groupthink” settle in, where your people are afraid to speak up. Any and every member of the team should have a chance to “pull the emergency brake”.  Especially, holding a formal “go/no-go” meeting is essential, where any stakeholder or project participant can voice concerns and point out facts that speak against readiness.
• That said, do be careful of the pessimist amid your ranks, who typically voices dire predictions on every project, and who sooner or later will be proved right. Insist on facts, not “Eeyore-ism”.
• Let your team watchwords for these situations become the following: candor, and facts. Bring both to the meetings, or go home.
• The best way to avoid letting pressure drive a premature launch: establish your launch criteria clearly and openly, with quantitative measurements, way in advance. If you don’t have legitimate facts-based ways to tell if you’re ready (i.e., not just the launch date arriving, and not your testosterone level that morning), well then, you’re not ready.

Without a doubt, “refuse to lose” can be a great attitude, up to a point. No executive, no team, should ever become blasé about missing a target. But “refuse to lose” can’t ever be allowed to become “refuse to face facts”. When it does, you don’t just lose face: you just plain lose.