Library / Behavioural Finance
Date of review: October 2021
Book author: Nate Silver
Вook published: 2012
Book author: Nate Silver
Вook published: 2012
The book can be divided into two parts.
The first part of the book deals with our flaws in analysing information and making predictions. Unsurprisingly, the list of poor predictions by humans is quite long. Anyone interested in this book probably understands the issues, so more evidence seems to be not so necessary as long as you understand that our judgement is far from perfect.
Hence, the second part of the book is much more useful, in my view, as it provides a solution to the problem. The solution offered by Nate Silver is to use the Bayesian formula. It took me a while to recall the formula from my old classes on statistics. But unfortunately, I had never thought of applying this formula to making predictions in real-life situations, so almost all of the second part of the book was extremely useful to me.
The first part of the book deals with our flaws in analysing information and making predictions. Unsurprisingly, the list of poor predictions by humans is quite long. Anyone interested in this book probably understands the issues, so more evidence seems to be not so necessary as long as you understand that our judgement is far from perfect.
Hence, the second part of the book is much more useful, in my view, as it provides a solution to the problem. The solution offered by Nate Silver is to use the Bayesian formula. It took me a while to recall the formula from my old classes on statistics. But unfortunately, I had never thought of applying this formula to making predictions in real-life situations, so almost all of the second part of the book was extremely useful to me.
Bayesian formula
The English mathematician Bayes worked out a formula in the XVIII century to estimate the new probability of an event if you receive certain evidence. I would quote the author's example directly from the book as it is so vivid that understanding complex math becomes quite easy:
"Suppose you are living with a partner and come home from a business trip to discover a strange pair of underwear in your dresser drawer. You will probably ask yourself: what is the probability that your partner is cheating on you? … Bayes's theorem, believe it or not, can give you an answer to this sort of question - provided that you know (or are willing to estimate) three quantities:
"Suppose you are living with a partner and come home from a business trip to discover a strange pair of underwear in your dresser drawer. You will probably ask yourself: what is the probability that your partner is cheating on you? … Bayes's theorem, believe it or not, can give you an answer to this sort of question - provided that you know (or are willing to estimate) three quantities:
1
You need to estimate that the probability of the underwear's appearing as a condition of the hypothesis being true - that is, you are being cheated upon (Y).
2
You need to estimate the probability of the underwear's appearing conditional on the hypothesis being false. If he isn't cheating, are there some innocent explanations for how they got there (Z)?
3
Most important, you need what the Bayesians call a prior probability (or simply a prior). What is the probability you would have assigned to him cheating on you before you found the underwear (X)?
To estimate the probability that your partner is cheating, you should use the following formula:
Revised probability once you found the underwear = XY / (XY + Z (1 - X)).
The author points out that the original probability (or base rate - how many observations have been made of a particular object / event in the total population) matters a lot for the new prediction. If X is extremely low, then new evidence can hardly increase the probability significantly. Stocks, on the other hand, often tend to overreact to new evidence because the market ignores the base rate.
"One of the nice characteristics of the Bayesian perspective is that, inexplicitly, acknowledging that we have prior beliefs that affect how we interpret new evidence, it provides for a very good description of how we react to the changes in our world…If you hold there is a 100 percent probability that God exists, or a 0 percent probability, then under Bayes's theorem, no amount of evidence could persuade you otherwise".
Nate Silver also discusses making investment decisions on a stock market a few times in this book. Unfortunately, his verdict is that it is very hard to outperform the market without having some unique edge.
"It is not so much how good your predictions are in an absolute sense that matter but how good they are relative to the competition…Beating the stock market requires outpredicting teams of investors in fancy suits with MBAs from Ivy League schools who are paid seven-figure salaries and who have state-of-the-art computer systems at their disposals…It can require a lot of extra effort to beat the competition".
I think it is important to understand the playing field and try to focus on less crowded spots like small caps, small markets, rare / unpopular sectors and so on. I don't think that an overall high competition in the market should stop you from investing in individual stocks altogether.
"If you have strong, analytical skills that might be applicable in a number disciplines, it is very much worth considering the strength of the competition. It is often possible to make a profit by being pretty good at prediction in fields where the competition succumbs to poor incentives, bad habits, or blind adherence to tradition - or because you have better data or technology than they do. It is much harder to be very good in fields where everyone else is getting the basics right - and you may be fooling yourself if you think you have much of an edge".
Revised probability once you found the underwear = XY / (XY + Z (1 - X)).
The author points out that the original probability (or base rate - how many observations have been made of a particular object / event in the total population) matters a lot for the new prediction. If X is extremely low, then new evidence can hardly increase the probability significantly. Stocks, on the other hand, often tend to overreact to new evidence because the market ignores the base rate.
"One of the nice characteristics of the Bayesian perspective is that, inexplicitly, acknowledging that we have prior beliefs that affect how we interpret new evidence, it provides for a very good description of how we react to the changes in our world…If you hold there is a 100 percent probability that God exists, or a 0 percent probability, then under Bayes's theorem, no amount of evidence could persuade you otherwise".
Nate Silver also discusses making investment decisions on a stock market a few times in this book. Unfortunately, his verdict is that it is very hard to outperform the market without having some unique edge.
"It is not so much how good your predictions are in an absolute sense that matter but how good they are relative to the competition…Beating the stock market requires outpredicting teams of investors in fancy suits with MBAs from Ivy League schools who are paid seven-figure salaries and who have state-of-the-art computer systems at their disposals…It can require a lot of extra effort to beat the competition".
I think it is important to understand the playing field and try to focus on less crowded spots like small caps, small markets, rare / unpopular sectors and so on. I don't think that an overall high competition in the market should stop you from investing in individual stocks altogether.
"If you have strong, analytical skills that might be applicable in a number disciplines, it is very much worth considering the strength of the competition. It is often possible to make a profit by being pretty good at prediction in fields where the competition succumbs to poor incentives, bad habits, or blind adherence to tradition - or because you have better data or technology than they do. It is much harder to be very good in fields where everyone else is getting the basics right - and you may be fooling yourself if you think you have much of an edge".
Other ideas discussed in the book
- Importance of continuous updates of your beliefs to move closer to the truth as opposed to seeking one single moment when you discover the full. The reason such a method of incremental steps is preferred has to do with our understanding of reality which is an approximation of the truth.
- It is worth paying attention to a market consensus as it is often correct. However, there are cases when it is not the case and the further you move away from consensus, the stronger your evidence has to be. "This attitude…will serve you very well most of the time. It implies that although you might occasionally be able to beat markets, it is not something you should count on doing every day; that is a sure sign of overconfidence".
- There is another thought related to the previous idea about the challenges of running an asset management business based on active asset management. It is hard to justify charging your fees if you are not actively buying and selling stocks. "…markets are usually very right but occasionally very wrong. This, incidentally, is another reason why bubbles are hard to pop in the real world. There might be a terrific opportunity to short a bubble or long a panic once every fifteen or twenty years when one comes along in your asset class. But it's very hard to make a steady career out of that, doing nothing for years at a time".
- The book also refers to the work of 'giants' in this field including Kahneman, Tversky, Tetlock and others. One message concerning Kahneman is about mistakes we make using mental shortcuts (heuristics) which implies that using special tools, checklist and following rules is very important in making decisions. Another important message is about the difference in the quality of prediction and quality of result suggesting not to judge the former by the latter. This idea was discussed in more detail by Annie Duke in several of her books.
- The idea that I read about in Philip Tetlock's book about group forecasts is also discussed by Nate Silver. He points out that an average forecast is better than most individual forecasts as long as they were made independently. However, he also says that there could be instances when one individual forecast is better than the average so it may pay off to follow one particular forecaster (Think Warren Buffett in investing?).
Final Thoughts
To conclude, I think the biggest value from reading Nate Silver's book is the introduction of Bayesian theory and the ways to apply this theory in practice. It looks extremely useful in analysing stocks and I am really surprised it is not used more widely. I do not mean a narrow prediction of where the share price will be in one year, but rather analysing probabilities for various scenarios, assessing what assumptions are priced in already, whether a stock has overreacted to new information and so on.
I searched for more resources on the Bayesian theory and did some tests to be sure I had fully grasped the concept. It still looks quite complex and is worth practising this method more.
I would not say this is a must-read book. If you have mastered the Bayesian method, are familiar with the works of Tetlock and Kahneman - then probably this book will add little new to you. Having a very poor understanding of the Bayesian method, I was very glad to have read the book.
I searched for more resources on the Bayesian theory and did some tests to be sure I had fully grasped the concept. It still looks quite complex and is worth practising this method more.
I would not say this is a must-read book. If you have mastered the Bayesian method, are familiar with the works of Tetlock and Kahneman - then probably this book will add little new to you. Having a very poor understanding of the Bayesian method, I was very glad to have read the book.