Should You Risk It?

Navigating uncertainty

This post is inspired by a series on my personal blog if you want to dig deeper.

How much uncertainty is too much when you are faced with an important decision?

The answer depends on the size of the stakes, your tolerance for risk, and recognizing what you can control versus what you cannot.

While we can’t eliminate all the unknowns and the uncertainties they bring, we can weigh their likely impact and manage how we deal with them. The best way to do that in my experience, is to answer these three deceptively simple questions:

• How will uncertainty change my decision?

• Will it help to learn more?

• Is it worth the effort?

Let’s explore each of them in turn.

How will it change my decision?

Sometimes figuring out the unknown is not going to change the decision you make. Sometimes it will, but not in the way you expect.

For example, uncertainty in choosing lottery numbers will have no effect, while uncertainty in road conditions will drive you to check GPS before you depart. In business, uncertainty in the demand forecast may lead you to question if you have the correct data and models in your forecast.

In my consulting work I have seen many examples where a company’s data team spends time and money to answer a question asked by the business, only to have the answer ignored while the business does just what it originally intended.

If the unknown really doesn’t matter and it’s not likely to change your decision, then do not waste resources on it. The corollary, if specific parts of the uncertainty drive your questions, that is where you should focus your efforts.

Will learning more help?

Sometimes you can reduce or manage uncertainty by learning more about it, so it’s worth investing some time and effort to investigate.

Suppose the uncertainty lies in defining what outcome you really want. My meta-problem method evaluates alternatives because that’s the best way to reduce your uncertainty about what you want (without actually making a decision and feeling regret after).

You may find that the best option is to do nothing.

In any modern company, a forecast that doesn’t incorporate changing products and marketing promotions is pretty much guaranteed to be informational at best. In my analytics work the thing we’re usually learning more about is the past – we’re putting in extra time and effort to pull in those marketing campaigns, to see when we had a price increase, to filter out the “special orders” that aren’t part of our regular process.

So, how do you recognize that the type of uncertainty you are facing is worth learning more about? It’s usually because you have a model of how the uncertainty might change your decision (if I knew tomorrow’s lottery numbers, I’d buy that ticket). Then you ask yourself how you could get that additional information.

It is worth remembering that some types of uncertainty are inherently unlearnable. The lottery is a good example: you cannot educate yourself to a win.

Engaging a subject matter expert can help speed up the learning process. They know the questions to ask, and they know the implications of the answer. They figure out all the learnable uncertainty much faster and are left just with the remaining unlearnable portion.

Is it worth the effort?

The balance between the degree of uncertainty and the amount of effort it takes to reduce it is a key consideration. Of the many uncertainties you could tackle, which ones are worth reducing given the time, money and effort you will have to put in? How will the quality of your decision improve, as a function of that investment?

Start by asking yourself if there is something else out there that works better than the action you’re contemplating. For example it’s a waste of resources to buy food explicitly to donate to a food bank – donating money goes much further because of their buying power.

In business, I’m often surprised by how little attention people pay to Return on Investment (ROI). They often want to focus more on predicting the future (“can we bring in stock market data”), than on understanding the past (“we had a promotion, so of course we had higher sales for a bit and then they dropped down”). I view the former as a waste of resources because predicting the future is very hard, but creating a system that can react to it is often much easier.

Equally, buying a data set that claims to predict the future when you don’t even know how to use your own data is a waste of resources. If you have a robust system in place already, then it might be worth an experiment to see if that third party data can make your models better.

When the weakest link in the chain of decisions is very weak, you almost always want to strengthen that one. If you have lots of similar links, maybe you can improve all of them with one design change and that will have the biggest impact.

It pays off to figure out how the unknowns and decisions are interconnected. Are the risks all in series, or do some run in parallel (in which case make it a priority to learn which path you’re on).

You now have a model for recognizing what you should try to learn so you can reduce the unknowns that will have the biggest impact on the decision, for the lowest investment.

---

Are you certain it’s a sure thing?

Quotes on uncertainty

“As far as the laws of mathematics refer to reality, they are not certain; and as far as they are certain, they do not refer to reality.”― Albert Einstein

“We demand rigidly defined areas of doubt and uncertainty!”― Douglas Adams, The Hitchhiker's Guide to the Galaxy

“Il n'est pas certain que tout soit incertain.(Translation: It is not certain that everything is uncertain.)”― Blaise Pascal, Pascal's Pensees