Julie Starling

The Illusion of Certainty

12 August 2023

In the Matrix, the main character Neo gets the choice of a blue pill or a red pill.

If he takes the blue pill, he returns to his life and he gets to believe whatever he wants to believe. It is the choice of putting his head in the sand and being ignorant to what is really going on.

However, if he takes the red pill he faces an uncertain future but the truth of what is really going on. He will learn an uncomfortable truth but also gives him the opportunity to take action and influence the future.

When it comes to software delivery, I believe we have this same choice with the data we use.

We can take the blue pill spend time looking for certainty working to deadlines that are unachievable and we can believe the illusion of ‘the estimated date’

...or we can take the red pill and use data that represents the truth… which is that software delivery is uncertain, and trying to get to certainty is a wasteful activity.

If I’ve got your attention, take the red pill and keep reading to discover how we can avoid the illusion of certainty.

As I talked about in ‘What is “Late” Anyway?’ the data we use for software delivery in can be thought of in two ways:-

  1. Data that allows you to take meaningful action for teams and customers
  2. Data that gives you the illusion of being able to do that.

So what data allows us to take meaningful action for teams and customers? Data that represents the reality of the situation you are in…what you know now, what is likely to happen based on what we know today and how this has changed since yesterday.

Take the Red Pill, Use Real Data

When we think of weather or economic forecasting we accept that the answers change based on new information and changes, understanding this is the same for software delivery is key.

Understanding what the weather will be next week is a difficult problem, understanding how long it will take to build a solution to a problem is exactly the same.

A probabilistic forecast uses past and present data to make a prediction about the future.

What does it look like?

A probabilistic forecast always has a probability and a range. It helps us answer the question, “when will I get it?” or “how much will I get by a date?”.

How Much Will I Get

There is an 85% chance of delivering 29 or more work items by the 31st August.

When Will I Get It

There is a 75% chance of finishing 10 work items in 15 days or less.

In the above 2 examples of a probabilistic forecast the % represents the ‘probability’ part of the forecast. Then the ‘range’ is represented either by the number of work items or the date, depending on the question being asked (how much or when).

Continuous Forecasting

The really important part of probabilistic forecasting is that we don’t just do it once, we don’t do it quarterly, we are doing it continuously so we can track changes in the trends based off new information. It’s also crucial that we are having conversations about the data regularly. This is what allows us to manage expectations, take action and influence the outcomes of the future.

For example, we might have been forecasting for the delivery of 10 work items, but as we’ve learnt more we may have uncovered complexity and that’s now 15 items. Once we run a forecast after this new information the answer to ‘when will we get it’ is likely to change. By recognising this and the impact it allows us to have sensible conversations at the earliest possible opportunity. Are we ok that the forecasted date is later than it was before? If not, what are we going to do about it? This might feel uncomfortable, but actually this would have happened if you were forecasting or not, but now you know the instant something has change and you get the earliest possible opportunity to take action.

A lot of the time the change in information isn’t as obvious as going from 10 work items to 15, there are a lot of micro-events that happen daily that can impact when we will deliver work, and quite often they aren’t significant for us to realise the impact they are having so we often don’t know until it’s too late that our delivery isn’t ‘going to plan’. Continuous forecasting gets around this because we aren’t waiting for someone to tell us something has changed, the data does that for us and signals to us as early as possible.

How Is This Any Different?

When we estimate we are essentially guessing with the information we have at the time and the experience under our belts. It takes people time to give estimates, this context switches people away from work and takes time to do. We also have to ‘notice’ an event to trigger a re-estimate.

So I might sound like I’m contradicting myself, I’m saying we shouldn’t use estimates because they are inaccurate and take time, but we should probabilistically forecast and we should do it continuously!

Well, the difference is that probabilistic forecasts can be run in seconds, and because they use past and present data, they don’t require a team to context switch away from the work in order to generate.

Also…estimating is really hard right? We have to guess how long something is going to take without really understanding how we are going to execute the solution or what we are going to uncover on the way. Additionally, we can sometimes forget we need to estimate in elapsed time not effort. We have no idea how much we are going to be expected to context switch, the meetings we’ll need to attend or unexpected absences. No wonder estimates are so often wrong! When we use past data to forecast however, those scenarios have played out in the past and get factored into the forecast, so we don’t have to try and ‘add a buffer’ to account for it or forget about it all together.

Also, our stakeholders REALLY don’t care about story points, they just want to know when they are getting their value.

Resist the Blue Pill

When it gets uncomfortable as forecasts change over time, we need to remember we’ve not lost anything by forecasting, we’ve actually gained actionable information. The events that are happening which are impacting the forecast would be happening anyway, but without forecasting it’s likely you wouldn’t have known about it until too late. By time you did uncover the deadline was going to be met, it’s possible you wouldn’t understand what caused you to be in this position either, sometimes it’s a series of micro events that lead us down a different path to what is expected.

Remember, certainty is an illusion, take the red pill and take control of your deliveries!

“How to Probabilistically Forecast” coming in my next post, but I’d recommend “When Will It Be Done” by Daniel Vacanti to get started on your probabilistic forecasting journey.