13 steps to improve your forecasts

5 March 2023

Long-term readers of my blog probably know that I have a strong interest in decision-making. Behaviour finance has made people aware of how human biases lead to poor decisions in investing. Typical biases that lead to poor results are now well-known.

These biases include loss aversion (sell winners too early while keeping losers for too long), overconfidence (70% of drivers believe they are above-average drivers, which leads to excessive risk-taking and too fast decisions), short-term focus and recency bias (selling during the crisis and buying during boom times), confirmation bias (seeking facts that prove your point rather than looking how you can wrong), anchoring ('the stock was above $100 just a year ago, it cannot trade that low for long'), authority bias (we rarely question grey-haired experts in suits and glasses), narrative bias (we can be easily convinced by a coherent story rather than by a series of random, unrelated facts) and a few more.

However, only some incorporate this knowledge in their investment process. Perhaps, this is not surprising given that the top minds in this field often have been more interested in uncovering flaws in our judgement rather than in correcting them. As Noble prize winner, Daniel Kahneman once said, being aware of biases should improve your decisions.

In a different interview, Kahneman surprisingly admitted that his studies might help organisations but have little impact on individuals. Here is an excerpt:

In a series of lectures during 1992-95, the investment legend Charlie Munger famously summarised 25 biases that affect human judgements. Many investors refer to his summary as the source for better decisions.

However, more than just knowing them is required to apply this knowledge in real life. Moreover, what practical steps you should take to achieve better investment results is not entirely apparent.

A few years back, I started thinking that besides the 'right' principles and tools to succeed in investing, you also have to know how we use them in the uncertain world. A simple example of this would be a business that has been growing at 25% per year for the past 30 years (with above-average margins and return on capital). Then for the first time, the company misses its guidance and lowers its outlook for the upcoming year. The stock, trading at 27x PE, drops 20%, but as the next-year earnings estimate has also been reduced, its valuation multiple is down only to 25x PE.

How do you incorporate the new information? Is the bad news an inevitable bump on a long and ultimately successful road, or is it a sign of more significant troubles ahead? Is waiting to see how the business performs in the next few quarters the right strategy, or is it better to exit the position immediately? How would your plan change if, three months later, the company announced that its CEO, who had been running the business for the past 20 years, decided to resign? And to complicate things further, the stock is down another 30%. It is now trading at 17.5x earnings.

How investors deal in such a situation (what factors they look at, what weight they assign to them, and how they arrive at a conclusion) sets good investors apart from exceptional ones, that 5% who can beat the index.

There is no simple winning formula in investing. But is there a winning approach for dealing with uncertainty and making better decisions?

I have summarised a few practical steps that you can take to improve your decision-making. These steps are for individuals. There are separate points for organisations which I have left out for now. The list is not final, but I hope you can find something useful too. It has definitely helped me.

Suppose you need to decide if Company A is a good investment. As part of your analysis, you want to estimate the possible operating margin in Year 5 and apply an industry-average valuation multiple. The company has been in business for 15 years, reporting a 20% EBIT margin for the past ten years when it was a public entity, while industry peers have delivered only a 10% margin. What margin should you use for Year 5?

If you follow the best practice of Superfocasters, your first step is to determine the Base rate.

The base rate is the average indicator for similar events or objects. So, in our case, it is an operating margin for companies in the same sector. It is also called taking an Outside view. Instead of looking into specific details of the situation, you look at similar cases from the past to determine the likely outcome.

For example, if someone asks you how likely Harry and Meghan will divorce, your first step should be to look at the divorce statistics. Most people start thinking about how strong the feelings of Harry and Meghan are, how their relationship has been evolving and where it is now. The Outside view would suggest that, on average, 45% of first marriages end up in divorce (the rate is even higher for second marriages). So the starting estimate should be around 50% (royal couples may even have a higher rate).

Returning to our example, we can use a 10% margin, the average for the peer group, as our first step.

The second step should be to analyse the individual circumstances of our company (the Inside view). We can look into their operations in more detail to understand if and how they differ from competitors. At this stage, it is worth looking at things like the supply of raw materials, geographic location, marketing strategy, who the customers are, whether they are willing to pay a premium for their products etc. It is also worth paying attention to the management team. Anything about them that makes you more confident in the company's maintaining its super margins in the medium term?

Let's say that several unique qualities make you think the company can achieve higher margins in the future than what its peers currently earn (10%). How should you incorporate this into your estimate?

There are two ways to incorporate specific factors into your final estimate.

The first option is to use the Bayes formula:

P (A | B) = P (A) x P (B | A) / P (B)

A stands for the parameter we are trying to estimate (operating margin of 20% in Year 5).

B stands for additional condition or evidence, for example, market share. Let's say Company A has been a market leader with a 30% market share.

P (A | B) denotes the probability of Company A earning a 20% operating margin, given that its market share is 30%. In other words, if we take all companies with a 30% share, how many of them would earn a 20% margin?

The last parameter, P (B), represents the probability of having a 30% market share (how common is the evidence). How many companies enjoy a 30% market share among all existing companies?

In a more straightforward example, let's say you must decide if a bald man with glasses is a scientist. Many people mistakenly assign a high probability because of a stereotype about scientists' appearance. And it may well be true. It is reasonable to expect one out of 4 male scientists to be bald and wears glasses.

However, it is an answer to a different question. This high probability shows how many scientists are bald and wear glasses (P (B |A), not whether a man you see in the street is a scientist.

So, in this example, the probability of a bald man wearing glasses is P (B). It shows how common the evidence is. Let's say 30% of all men living on our planet are bald or losing their hair, while about 64% of adults wear glasses (surprising results, but this is what I found using Google, maybe I should have used ChapGPT instead). The bald men wearing glasses should then account for about 19% of all men (approximately one out of five men you meet in the street is bald and with glasses).

The last parameter we need to answer the original question is to estimate what proportion of men are scientists (regardless of their looks) - the Base rate. Google search comes with the answer that there are 8.8 million scientists, of which 59% are men (5.2m). Assuming a 4 billion male population, of which 2.8 billion are adults, the probability of finding scientists among them is just 0.2% (!).

So using the Bayes formula, the probability that a bald man wearing glasses is a scientist is just 0.26% (0.2% x 25% / 19%). In other words, less than 1 out of 100. Or, in a group of 10,000 men, only 26 are likely to be scientists.

Going back to our question on the operating margin of the company we study, let's assume the following values for the parameters we need (I did some Google search to come up with these assumptions):

P (A), the probability of having a 20% operating margin equals 20% (the share of businesses that earn that much).

P (B), the probability of having a 30% market share is 13%.

P (B | A), the share of companies with a 20% operating margin among those with a 30% market share is 40%.

In this case, we can estimate P (A | B) as 20% x 40% / 13% = 61.5%, which is relatively high. The initially low Base rate (20%) is boosted by the low probability of the evidence (companies with a 30% market share - 13%) and a reasonably strong relationship between the market share and operating margin (40% of companies with a 30% market share enjoy a 20% operating margin). To be conservative, instead of using 20%, I would assume something between 15-17% margin (still above a typical 10% operating margin), but not 20%, given that it is not a 100% probability.

Daniel Kahneman proposed the second way of incorporating additional information into your calculations in his latest book, Noise.

It is also a three-step process that starts with an intuitive guess (A), followed by an estimation of a mean for similar cases (B, or Base rate) and, finally, by estimating the value of the additional information that you have (C). The last point is the correlation between the evidence and the point you are trying to predict. Importantly, it is rare for social sciences to have C above 50%, while many correlations that we view as meaningful are at 20%.

The formula that Kahneman proposes to use is the following:

Final estimate = B + (A - B) x C

If you initially thought that a 20% margin could be achieved with a 75% probability (A), you would have to correct it by the Base rate (B) 20%) and the value of the additional information (C) (market leading position with a 30% share). Using the guidance from Kahneman that 0.5 is extraordinarily high and 0.2 is meaningful, we can probably use something like 0.3.

In this case, the final probability should equal 36.5%, derived as 20% + (75% - 20%) x 0.3.

There is quite a bit of a difference between the results of the two methods. In the first case, you can safely assume the margin is close to 20%, while in the other, you should be more conservative, aiming closer to the Base rate of 10% (perhaps, 12-13%).

I think both methods have their strengths and weaknesses. The first method uses more precise estimates. However, it is often impossible to get quality data. Also, I only considered one additional parameter (market share), which may not be the most helpful for predicting future margins. The second method would always push you closer to the Base rate, forcing you to pay little attention to the individual characteristics of a company or a situation.

I advise using Kahneman's formula when the data is hard to find while using the Bayes formula in all other cases knowing its limitations.

It is essential to put the critical arguments for making a specific decision on paper. It helps minimise the hindsight bias ('of course, I knew it from the start'). As a result, you will suffer less from the overconfidence problem. Besides, you may have much less conviction or miss specific vital facts to reach a conclusion. On the other hand, if your thesis is well thought through and is based on deep research, the next time the situation looks completely different (a great company makes a profit warning), you avoid panicking. In fact, you can take advantage of a lower price by buying more.

A pre-mortem is even more critical than a thesis since we subconsciously look for facts that prove our point when writing a thesis. We often cut corners and ignore counterarguments to make the story more coherent. By noting how a decision has failed (before making such a decision), we focus on points we have not considered. In investing, writing a thesis about why a stock is an excellent short could be a good exercise. If it is easy, you probably should not consider buying the stock.

As part of writing a pre-mortem, it is worth looking for counterarguments. Listening to people with opposite views is generally beneficial, even though it may sometimes be unpleasant. It doesn't mean you have to agree, but at least you should understand where the difference comes from and what makes you confident in your point of view.

This piece of advice comes from Kahneman's book Noise. There are many examples when the same person makes a different estimate the second time. The most notorious are sommeliers, trained wine experts, who are supposed to rate wine. The same expert often views the exact wine differently at different times. This method follows the logic that a crowd forecast is, on average, better than an individual forecast (because mistakes in the crowd tend to cancel each other out, like a too-pessimistic and a too-optimistic view would produce a more balanced assessment if combined).

Kahneman advises asking an independent person (not influenced by your views) for a second opinion. But if it is impossible, his advice is to make the second estimate yourself and take the average of your two estimates. I have tried it a couple of times with the general knowledge test, and 60% of the time, the mean of my two answers was better than the original estimate.

The advantage of not going with the first estimate immediately also has to do with the difference between System 1 and System 2 (to borrow Kahneman and Tversky's definitions). To remind, System 1 is more intuitive, effortless thinking developed thousands of years ago (a sound in the bush is quickly interpreted as the signal of an approaching animal and makes us run away when it could be just a wind blowing through the trees). Often, we come up with the most prominent and easy answer, unwilling to put more effort into thinking deeper. So by deferring your immediate decision, you reduce the risk of acting impulsively and give your System 2 a chance.

There is a vast body of evidence that crowd predictions are often more correct than individual assessments. But an essential condition for that is the group members have to make estimates independent of each other. Suppose one member with authority status makes his estimate first and invites others to share their views. In that case, it will be hard for junior members to disagree with the boss, and the final result would match the original estimate.

This idea suggests that financial markets (crowd forecasts), more often than not, make sound predictions about future returns. However, markets become less efficient when investors stop thinking independently, and a new mania emerges.

Most people are not sensitive to the time scope of the event they predict. They often assign a similar probability when asked: "What is the probability that a country detonates a nuclear device before 31 July 2023?" and "What is the probability that a country detonates a nuclear device before 31 December 2026". While the probability is very low in both cases, it is definitely higher in the second case.

Moreover, often people don't even consider the time horizon. By adding the time reference to the question, you can get an utterly different answer, but few people pay attention to that. Consider these two questions: "Will Vodafone go out of business?" and "Will Vodafone go out of business in the next two years?". Will you assign the same probability to the event in both questions?

The practical implication is that investors overestimate the probability of events happening in the near term and underestimate their likelihood in the long term. In reality, things don't change much in the short term. Tomorrow is most likely going to be the same as today. However, things can be very different in ten or even five years.

Rare experts consistently make better forecasts. One such group, Superforecasters, beat the best intelligence analysts who had access to classified information during the competition in a tournament described in the book by Wharton professor Philip Tetlock.

Much has been said about the exceptional track record of a small group of "Superinvestors" and the benefits of copying their ideas. It pays off more to look for smaller names that "Superinvestors" ignore because their portfolios are too big, and they have to invest in larger, more liquid stocks. But creating a list of ideas based on portfolios of investors with the best track record and then looking for the best one or two ideas in that list is much better than searching a universe of tens of thousands of stocks listed globally.

Many experts fall victim to their own success, focusing more on media attention than actual results. Such experts tend to stick to their views, emphasising the logic of their arguments more than the end result. They fear being viewed as unprofessional and untrustworthy if they change their opinions. It is essential to fight your ego. If you don't put too much at stake and make your prediction public, it may be easier to change your views as new information becomes available.

Stanley Druckenmiller, a long-time associate of George Soros and a founder of Duquesne Capital, emphasises the importance of remaining open-minded and ready to change his mind quickly. He is famous for having a motto: "strong convictions, held lightly".

Humans tend to lack consistency in their judgement. The same information can be interpreted differently by the same person depending on his most recent experience, recent conversation (which can lead to anchoring, for example), state of mind (when you are in a bad mood, you tend to be more sceptical and assign a lower probability of success).

Rules are not perfect and often do not work. But because they lead to a more consistent result, they tend to produce better outcomes, on average, than if an individual decides without following any rules.

A checklist is a must for investors. I think Warren Buffett also uses it, but he may not call it that way. Some of the questions on his list include the number of years the business has been operating (he excludes everything with a short history), levels of profitability, debt, sustainability of competitive advantage, and quality of management. Checklists cannot replace human judgement, but they can help reduce the universe of stocks to focus on. Occasionally, they may remind you of an important element you may have forgotten about.

Speaking of rules, Morgan Housel, the author of The Psychology of Money, shared an exciting story about a little-known investor in his latest article.

Studies suggest humans are bad at assigning precise values/probabilities. People use different scales. Differentiating between 60% and 70% probability is challenging for most. Ranking events is easier (what is more likely to happen rather than what is the likelihood of this event?). For that reason, if you consider a company to invest in, it is helpful to have a peer group to compare that business and assess its relative attractiveness. It may explain why many successful investors are generalists. Also, coming up with a precise fair value for a stock may not be as helpful as figuring out the one company that stands out from the rest. Naturally, the more companies you can compare it with, the more reliable your ultimate conclusion will be.

As mentioned before, this is not the final list. There are a few issues I am thinking about.

Outliers. Including base rate into your probability assessment will make you almost automatically reject the next Google or Amazon. Based on statistics for startups, most fail, and less than 0.1% turn into a trillion-dollar business. However, as shown in various studies, stock returns are negatively skewed, meaning the average stock market return (7-10%) is disproportionately influenced by a small group of outliers who deliver (20%+ returns). A typical company, on the other hand, achieves returns closer to 2%.

Lindy effect. I have not found an actual mathematical proof of this concept, although a few mathematicians refer to it. The Lindy effect suggests that a non-perishable thing will live as many years in the future, as it has lived until today. For example, the best way to forecast how long a book will remain popular is to check how many years ago it was published. A book published just a year ago is, thus, unlikely to stay popular in five or even two years, whereas stories by Agatha Christie or Arthur Conan Doyle will remain popular for at least another 100-140 years.

The Lindy effect favours stocks of old companies like Coca-Cola. The risk of their early death (and permanent capital loss) is low

More on Base rates. How big should the data set be? Should we give more weight to more recent events (e.g. in the past century, there were only four pandemics, but in the first 21 years of the new century, we have already experienced four pandemics, which implies about 5x higher probability)?

I also think one has to be extremely cautious when dealing with rare events (low base rates) that can have enormous consequences (e.g. Russia's invasion of Ukraine). While many Superforecasters assigned a low probability of this event based on the past data, the consequences of such action were dramatic. The great financial crisis (2008) was also statistically a low-probability event, yet it had a massive impact on asset prices around the world.

To tackle such risks, I have started to avoid levered companies and try to keep about 10% of my portfolio in cash. A higher cash level will drag portfolio returns lower during a strong market, I can underperform in 9 out of 10 years. However, when things go not as expected, the extra cash should not only protect the portfolio from steep losses but also provide the necessary dry powder to buy great businesses at bargain prices.

Such an approach is hard to implement in asset management as most clients will eventually leave after several years of underperformance, but it should work much better at building wealth over the next few decades and beyond.

Non-normal distribution. Forecasting has many more unresolved questions, including events that don't follow the normal distribution. Applying base rates to predict such events in the future is wrong.

Understanding what is priced in the share price. Making a correct forecast is hard, but even if you manage to be correct, you cannot fully benefit from your accurate forecast unless you know what is priced into a share price. You will likely be right if you bet that Coca-Cola is a great, high-return business that will continue selling its products in the XXI century. But how many other investors think the same, and is the stock price already reflecting that? How much money will you make from this forecast?

In this note, I have shared 13 steps to improve your forecasts. They are:

1. Start with the base rate (Outside view)

2. Consider individual circumstances (Inside view)

3. Use Bayes or Kahneman formula to adjust your views

4. Write a thesis

5. Write a pre-mortem

6. Seek counterarguments

7. Don't rush. Make the second judgement, and take the average of the two.

8. Consider crowd forecasts as long as they are independent

9. Pay attention to how much time is left until the event you are forecasting

10. Find your superforecasters

11. Be ready to change, don’t make your forecasts public

12. Use checklists and rules

13. Rank, not rate

Did you find this article useful? If you want to read my next article right when it comes out, please subscribe to my email list.