accurate estimations

src: Accurate Estimations - Cam Hashemi - 2020-12-28

Abstract

Estimating is hard for both technical and social reasons:

• technical: we don’t know what we don’t know ⇒ hard to estimate a “buffer”
• social: peers / clients may judge you for understimating/overstimating the tasks/projects

Rather than giving a single-pointed estimate, we can give two points to create a range of targets to hit: best case & worst case.

By estimating with 2 numbers instead of one, we expands our range of accuracy, which can be called the “confience intervals”.

We’re constantly asked to give estimations:

How long will it take?
How much will it cost?
What time should we leave?
How much do you want?

Estimations are information about the unknown. We constantly use this information to make decisions: allocating resources, changing strategies, and choosing partners. But despite all this practice, we’re horrible at accurate estimations.

Why Estimations are Hard

Estimations are hard for both technical and social reasons.

We don’t know what we don’t know. Naive estimators fail to account for surprises. They estimate based on known factors and best-case scenarios. These estimates may be perfectly accurate beforehand, but they’re instantly broken by the first surprise.

Experienced estimators account for this problem by ‘adding some buffer’. But even then, how much should they add? Even knowing that surprises can happen, it’s impossible to know how many will happen or the impact of those surprises. Choosing the right amount of buffer is a lot like making the right estimation in the first place. We still don’t know what we don’t know. And adding too much buffer can be as expensive as failing to account for surprise at all.

In addition to this technical problem, there’s a strong social problem. Let’s imagine two common scenarios.

In the first scenario, you’re giving an estimate to your team. You perfectly estimate that a project will take three weeks; but your manager gives you a puzzled look. Your teammate snickers, claiming they could do it in a week, tops. The feeling is that you must be either lazy or incompetent to give such a padded estimation. Your newly shortened estimate is wrong, so you go on to extend the project’s deadline twice in three weeks. Rather than holding you accountable to that one-week estimate, your manager commends you for being able to handle the unforeseen surprises on such a complicated project.

In the other scenario, you’re giving an estimate to a potential client. You perfectly estimate that a project will take three weeks; but your competitor only estimates it’ll take a week. The client signs with your competitor. Since their estimate was wrong, your competitor goes on to extend the deadline twice in three weeks. Even though your estimate was accurate, your client’s estimate got them paid.

I’ve been in countless scenarios like this. Sometimes people outright pressure us into shortening our estimations, and sometimes the voice in our head pushes us to. Either way, giving accurate estimates is both technically hard and socially challenging ¹.

Two Types of Estimators

In response to this hard problem, we become systematic underestimators or systematic overestimators.

Underestimators fail to give enough buffer. This strategy has two key benefits. First, it signals (unrealistically) high performance. Like our virtue-signalling teammate, we can underestimate ahead of time, then point at concrete surprises for our eventual underperformance. And like our overpromising competitor, we can underestimate during a bid and do whatever we want after the contract is signed. Second, tight estimates demand efficiency. Underestimators set deadlines that they and their teams must work hard to meet. Underestimation works well when the costs of going over-budget are small. But when those costs are large, underestimations lead to disasters. On the whole, underestimators systematically run the risk of being burnt out, past-deadline, and over-budget.

Overestimators are instead biased towards large buffers. Extreme overestimators might send you articles titled “Estimations are a Scam”, or claim that estimations are simply tools for worker exploitation. Overestimation works when the costs of buffer are low and the costs of going over budget are high. But overestimators are constantly taxed by Parkinson’s Law. Parkinson’s Law is the pattern where projects fill the time and resources they’re allocated, instead of the time and resources they need ². Rather than pushing towards peak performance, overestimators systematically move at a bored, leisurely pace. Overestimators are also demotivating. Rather than inspiring the team to reach competitive goals, they disparage those who do. So while underestimators run the risk of their teams burning out, overestimators run the risk of their teams shutting down.

This line between underestimators and overestimators forms a classic “spectrum problem”. A spectrum problem occurs when we oversimplify the solution to a given tradeoff. In this case, we’re splitting the spectrum of estimation strategies in half. Underestimators fall on one side of the line, and overestimators fall on the other. With many spectrum problems, the better solution is to cut the spectrum into three pieces, choosing the middle strategy between extremes. In doing so, we acknowledge the costs and rewards of both sides, maximizing the upsides and minimizing the downsides systematically.

Playing Single-Pointed Darts

Imagine a game of darts with very simple rules. I throw my dart first, and you only score by hitting that same exact dart-sized point. This game is very simple, but so difficult that nobody would play it. To make darts playable, we specify scoring ranges. These same ranges are missing from our everyday estimations.

The most common estimate sounds something like: “I’ll have it ready by 5pm.” But this is just like playing single-point darts! Imagine that this estimate is perfectly precise: the project can’t be ready one second before or one second after 5pm. While impressive, there’s zero room for error. If we end up sick, or if the project ends up more complex than expected, our estimate becomes instantly wrong.

To account for surprises, we can add buffer. But adding buffer only shifts the dart board over a few inches. “I’ll have it ready by 5pm tomorrow” faces all the same problems as the first estimation. Two days of surprises still makes me wrong. We’re still playing single-pointed darts.

It’s a losing game. And yet we see it played again and again, day after day, project after project.

Playing Darts with Ranges

To enjoy this game of darts, we need to change the rules. Rather than giving a single-pointed estimate, we give two points: one for the best case, and one for the worst case. These two points create a range of targets to hit, just like the game of darts we know and love.

To demonstrate, I can give an example from my personal life. My girlfriend is very punctual, and I’m not. She’s a classic overestimator, giving as much buffer as we can afford. And I’m a classic underestimator, giving the most optimistic estimates. So whenever we have somewhere to be, and she asks me “what time should we be ready?”… it’s a classic estimation problem!

Recently, I’ve started giving her two answers. The first answer is “the green time”. The green time tells us when we should leave so that we’re pleasantly early. The second answer is “the red time”. The red time is the last minute we can leave, without definitely being late. If we can be ready by the green time without stress or shortcuts, that’s perfect. Once we pass the green time, we go into the “yellow zone.” The yellow zone is the buffer between early and late. There’s no need to take shortcuts or change plans yet, but we’re getting close. Once we approach the red line, we start discussing the need to take shortcuts, or to start telling others that we may be a little late.

Having a green time, a yellow zone, and a red time transforms our game of single-pointed darts into a proper dartboard, with zones of success. The green time makes my girlfriend happy: she can be ready early without worry. The red time makes me happy: I can fill my buffer time up with other activities. And the yellow zone is a signal for both of us to get focused or to start taking shortcuts ³.

Ranges make uncertainty explicit. Adding buffer doesn’t just shift the dartboard a few inches, it expands our range of accuracy. Larger buffers signal more uncertainty and require less precision, while smaller buffers signal more confidence and require more precision. The expression of certainty and confidence isn’t possible to communicate with a single-pointed estimate.

By estimating with two numbers instead of one, a richness of new information and strategies become available to us.

Simple Changes

Using two numbers instead of one, these so-called “confidence intervals” aren’t complicated. Then why are they so absent from everyday life?

Although nothing prevented us from discovering them earlier, the first mention of confidence intervals in scientific literature wasn’t until 1937. And it wasn’t until the 1980s that they were required in scientific journals. So if it took forty years for the most knowledgeable people to apply a simple solution towards the most urgent problems, it’s not surprising that it’s taken the rest of us at least as long. That said, the goal of this essay is to speed up that process.

We can move towards accurate estimations by practicing two simple rules. The first: When giving an estimate, state the best- and worst-case answers. The second: Whenever getting an estimate, ask for the best- and worst-case answers.

“It’ll take two weeks” should raise a red flag. A better answer sounds like: “in the best case, one week; in the worst case, three”. Boom! Now we have a range of days to deliver the project, instead of one. Not only do we have a picture of the optimal scenario, we have an idea for the hidden risks lurking in the future.

As a bonus, we’re capable of using the “green line, yellow zone, red line” imagery to guide our actions ⁴. The green line is how we strive for the best and signal our competence. The red line is our insurance policy in case of surprises. The yellow zone is where time is critical, and we need to coordinate in case our plans need to change.

In a world of uncertainty, this extra byte of information is enough to create more accurate thought and action.

Footnotes

1. Though questionable, accurate estimations are worth the effort. First of all, accurate estimations build trust. After a few false promises, people catch on. Like compound interest, commitments founded on years of accrued trust are worth much more than short-term deals based on deceptive estimates. Secondly, inaccurate estimations kick off a vicious cycle of degrading quality. Underestimations lead to insufficient resources, insufficient resources lead to shortcuts, shortcuts lead to poor quality, and poor quality leads to greater consumption of resources. These teams work harder and harder to do less and less. Inversely, overestimations lead to too many resources, too many resources lead to inefficiency, inefficiency leads to overestimations. These teams take longer and longer to do less and less. ↩

2. Parkinson’s Law is the insight that projects tend to consume the resources given to them, be it time, money, or people. Fear of Parkinson’s Law motivates people to justify tight deadlines. They claim that without tight deadlines, people tend to get lazy or unfocused. But if Parkinson’s Law was the whole story, we’d never see projects go over budget! But since we see projects go over deadline and budget all the time, adjusting deadlines can’t be the only factor. Instead, keeping deadlines as tight as possible leads to burnout, in addition to all the other costs of underestimating. Though useful for timeboxing projects in general, Parkinson’s Law should not be used as the sole guiding principle for estimations. ↩

3. In professional projects, the same concepts apply a bit differently. The green time is the earliest a project can be ready, satisfying the underestimators; and the red time would be the latest a project can be ready, satisfying the overestimators. Underestimators can still push for the green time, while overestimators still find peace of mind in knowing the red time exists. ↩

4. “One week, give or take two days” in a common way to convey a similar amount of information. I focused on the “best case / worst case” framing, because it opens the door to the green / yellow / red style of project management. That said, two answers are always better than one, so if adding a “give or take” is an easier way to make this transition in how we estimate, I’m all for it. ↩