In the immortal words of Pogo, “we have met the enemy, and he is us.”
The long-held stereotype of IT portrays us as uncooperative, unable to integrate socially, always arguing over nits, and deeply, intractably immersed in our own tunnel vision and parochial perspective.
That stereotype has held us back, as individuals and as a profession: it’s actually one root cause for the oft-lamented situation of the CIO needing a seat at the executive table and not getting it.
Now a whole movement has arisen that unfortunately reinforces that negative perception of IT people, a movement that coalesces around the Twitter hashtag #NoEstimates (all movements need a Twitter hashtag now, it appears). It started with long screeds about how inaccurate estimates are for software development: they’re nothing more than guesses; “there is nothing about them that makes them necessary, or even beneficial to the actual creation of software”. They’re a “wasteful and deceptive practice“, lies, and needing them is even comparable to how heroin users need their heroin. You can’t predict the future, these estimate detractors insist. The #NoEstimates rhetoric has become increasingly harsh, and replete with drastic imagery: estimates are a “game of fools“; an “inherited disease for the industry“; “we predict like gypsy ladies.” In what seems at times to be some kind of “top the previous hyperbole” competition, estimates have even been referred to as “management by violence”.
This sort of overreaction, this IT resistance to estimates, isn’t really wholly new, of course. More than once, I’ve watched in horror as an IT person earnestly explained to a company’s senior management how predicting systems delivery is so very difficult and so very filled with uncertainty, justifying how (for example) they couldn’t possibly commit to having a system deployed in a particular quarter. One time, I witnessed the VP of Sales in particular having zero patience with that: “Hey, I get asked to set my sales quotas way in advance based on my best professional judgment; what makes you folks in IT think that you should be an exception?” Business runs on goals and commitments, and on the “best judgment” estimates that help create those goals and commitments. And everyone in the room seems to understand that, except IT.
Setting achievable, concrete goals is healthy, in business and in life. Making solid, reasonable commitments is healthy. Taking responsibility for meeting one’s commitments all or at least most of the time is natural and should be encouraged. And if you’re perceived as constantly wailing that you’re different and special, in fact so special that you believe you can’t really be expected to state when you’ll be done? That’s a non-starter. This blog post is an introduction to how some IT practitioners have been pulled in by the seductive but ultimately wrongheaded NoEstimates claims, and what the resulting implications are for our industry.
What’s #NoEstimates about? Well, its proponents are actually quite slippery on providing any solid definition, reflexively pushing back whenever anyone tries to summarize it for them, and they often prefer to vaguely say it’s just “a hashtag for a concept about alternatives to estimates and how they might help make better decisions.” However, it seems to boil down to a virulent and unshakable base conviction that estimates are utterly horrible, for all the reasons stated above and more. Using estimates allegedly leads to chronic abuse of developers by management, cements inflexibility into projects, and disturbs developer “flow”. The sole alternative to estimates that the NoEstimates advocates clearly identify, however, seems to simply restate the age-old Agile principles of small teams, user stories sliced into doable chunks, use of “drip funding” so as to achieve a predictable fixed cost, and software delivery “early and often”. They point out that software project scope tends to change drastically and frequently during such frequent delivery, and argue that this scope variability allows projects to be declared as done and successful, long before and without ever having actually delivered all the upfront-identified desired functionality.
Over the next two blog posts, I’ll first lay out the reasons to see considerable value in a software development estimating process and its outcomes, and then respond to the myriad NoEstimates complaints that are levied against estimates as used for software development. But we’ll start here, as an intro, just with invoking basic common sense about life and business.
Every day in life, spoken and unspoken estimates drive what we do. Can I make it to the dry cleaners and to the grocery store to get some milk before I have to pick up my child from school? Can I cross the street without being hit by the car I can see approaching? I need to get more done on that key task: is it worth getting someone to help me, or should I just do it on my own?
As in life, so too in business: pretty much everything we do involves us formulating and weighing implicit estimates: what will this take? Will we make it in time? Will it be worth it? And so on. The only thing up for debate is when and how much we formalize them (i.e., write them down, or tell them to others). The extent we tend to formalize a given estimate depends: on the issue, the stakes, the team, the time frame, etc. There’s no single “fits all” answer.
Whenever there is any manner of management/business scrutiny (and there always is), estimates in some usually more explicit form are part of the landscape. In fact, pretty much all proposals to senior management will cover or evoke these specific questions in some way: “How much will this cost?” “When can we have it?” “What’s our expected benefit?” I’ve never seen these questions not get asked by the boards, the CEO, COO, and CFO of even smallish companies. They’re the most natural questions in the world when undertaking to have anyone do work of any substantial size in any domain. In fact, a board or CEO that didn’t ask these questions about a major pending project would be guilty of dereliction of duty.
In short, looking at it from a macro level, it is easy to argue that one simply can’t do business, at any meaningful scale and over a reasonable period of time, without constantly assessing probable costs and impacts, and then deciding when to take judicious risks. That’s in fact what management does.
The NoEstimates claim is that software development is somehow a valid exception to this natural and pervasive use of estimating as a tool to help make decisions. Making such a claim bears a heavy burden of proof. And my thesis here is that the NoEstimates conversation hasn’t even come close to meeting that burden of proof; my next two posts will go into detail on why not.
Next post: my list of the specific reasons why it makes sense to use estimates in business and software development.
Lagniappe:
- Previous post: “Towards a more balanced list of content about #NoEstimates“.
“Whenever there is any manner of management/business scrutiny (and there always is), estimates in some usually more explicit form are part of the landscape. In fact, pretty much all proposals to senior management will cover or evoke these specific questions in some way: “How much will this cost?” “When can we have it?” “What’s our expected benefit?” ”
—-> It might not be a proposal to senior management, but Google’s idea of 20% time, or the common practice in Silicon Valley of the “hack night”, sometimes called the “Federal Express night”, seems violate these “principles.” With the Federal Express night, the idea is that new software absolutely, positively, needs to get there overnight.
Senior Management doesn’t care what it is.
Or, more accurately, Senior management has made a tactical investment of the programmer’s time into R&D. They don’t know what the product will be up-front. A bit like a classic Venture Capitalist for oil in Texas in the early 20th century, they going to drill a bunch of holes and if they get one gusher, it will more than pay for all the misses.
Goranka Bjedov, a performance engineer at Facebook, tells me that Facebook’s Photo Album feature was a hack-day project.
So, there you have a decision that seems to support no estimates that has a positive business outcome. I suppose you could argue that senior management had some estimates, and those estimates were some sort of long-term idea of ROI based on the aggregate productionalization of R&D ideas … but that’s not what the authors of the hashtag meant.
Thanks for commenting on substance here, Matt: it’s notable to me that you’re among the very few NoEstimates advocates to do so, in the three years since I wrote these posts.
My main response here is “of course” — it’s never been in doubt (and I’ve frequently stated) that “no estimates” will work just great in any situation where no one cares what they get, when they get it, or how much it costs. Google implements the 20% policy for (at least) two reasons: employee satisfaction/motivation, and as a broad portfolio bet, where only a small portion of such “hobby” projects will likely pay off in terms of real revenue. If none of them pays off, like ever? They’ll shrug; it’s a BET they’re making, not a certainty. And eventually, they *might* eliminate the program altogether if there are no wins over a significant period of time. Because cost matters in business.
In terms of a specific business decision, not a general HR-oriented policy? Well, a good friend of mine is a software engineer at Google. She described to me a situation where someone worked on a 20% side project for well over a year, then presented a value-based business case to his boss. Next thing he knew, the 20% side project was declared to be his full-time project because it was deemed to have the requisite value/impact vs. cost. You can bet that then there were established goals, deliverables, timelines, and you can bet that estimates were involved in all of those. Because as the portion of my post that you quoted states, “pretty much all proposals to senior management will cover or evoke these specific questions in some way: “How much will this cost?” “When can we have it?” “What’s our expected benefit?” ” Basically, all major business decisions come back to those questions.