Superforecasting: The Art and Science of Prediction

A great book on the art of forecasting. You will be able to take the key concepts away and practice it on your daily life.

I found this a really interesting book. I was very sceptical at first and the idea of superforecasters sounded like an issue of survivorship bias, but the author did address this satisfactorily. For the most part the book just goes through various types of logical fallacies and how you can avoid these to make more accurate predictions about the future, so if you know a bit about probability and logical fallacies already you won't find it much new. But the story of the Good Judgment Project is very interesting and certainly worth knowing about.Certainly in this time of COVID-19, after reading this book you'll start noticing a lot of public figures fall into basic data interpretation mistakes, make predictions that turn out to be totally wrong, and then continue as normal anyway!

One simply cant put this book down. Phillips Tettlock and Alan Gardner have clearly taken a leaf out of Gladwellian type books in terms of entertaining content as opposed a more Kahneman type approach which bombards one with details. The book is not of course a detailed methodology of how to forecast but simply looks at a simple question. What makes a good forecaster? We are led through the authors thoughts' on this matter throughout the book by means of some high end anecdotes (Obama, General Petraeus etc) and some more simple retired fold who happen to be good at forecasting. We know that they are good at forecasting through the online forecasting tool developed by Tetlock and others which invited participants to simply make forecasts on a number of events. All for the grand allure of an amazon gift voucher! (Well more books eh?). Once the cream rises to the top of this forecasting competition, the book then delves into what makes a good forecaster, which are summarised in temp key points at the end of the book. Its all bout being logical, taking baby steps (joining the dots), keeping up to date with the latest events, making small incremental changes in your forecast and of course remember that are human and prone to errors and bias.Although the book was more an entertaining summary of the study and why old retirees were getting Brier scores better than high paid statisticians, I was perhaps expecting a bit more science in this pop-science book. However full marks for this book. I understand if I want the science I am sure I can find some papers by him online.

A fascinating book which will change your life (and work) if you are someone who is ever called on to make a forecast, or to make a decision which depends to any extent on future events. This is the art of forecasting based on hard edged research into the art of forecasting (and not limited to any particular field). It teaches techniques which can be used to improve your view of the future in any area at all, whether it be sports, weather, politics, science, or anything else. It's also a rattling good read. The book is co-authored by the academic researcher who has led the field in the study of what makes for good and bad forecasting, and by a specialist in accessible writing for the general public. The result is a rare combination of cutting-edge, authoritative original research, and easy-to-read self-help for the general reader.

I wanted to know about forecasting and more specifically how i might be able to use this book practically to take advantage of certain opportunities in the thematic investment space. There is an incredible amount of waffle whereby the author goes on and on about the GJP. The practical application of forecasting isn't really discussed. It's a pity as it had the potential to be an excellent book but instead reads like a puff piece for the GJP in the main. There are tiny useful tidbits but luckily I didn't pay full price so on that basis probably got my money's worth. If I had paid full price I would have felt somewhat cheated. Unsure as to why it's got so many 5* reviews.

"Experts are about as accurate as chimps when predicting the future". This tidbit, so often mentioned when discussing (or dismissing) expert opinion or predictions, originates from the research of Mr. Tetlock on Expert Political Judgement. A natural next step was figuring out if anyone could reliably answer questions about the not so distant future and the result was the Good Judgment Project. The main results are detailed in this book, there indeed exists a group of super-forecasters who manage to constantly out-predict the chimps and experts in the intelligence community. The book describes some of the characteristics of a super-forecaster. Not surprisingly they are, in general, good with numbers and ingest a lot of information. They tend to be slow thinkers, in the sense of Kahneman, and at least in some cases not as much affected by cognitive biases. Super-forecasters, however, are not super-human. Forecasting is a skill that can be learnt or improved.

Nem só de algoritmos e métodos matemáticos devem viver as predições. Confiar nos instinto dos especialistas (que seguem uma metodologia) é super importante para se realizar predições mais assertivas.

If You are wondering how often the pundits are right, you will discover almost never… Your feeling is right! It s science based

I really enjoyed this book a few years ago, and I have come back to offer a review based on my notes at the time and how the insights have settled for me over time. I took away many key concepts for successfully forecasting uncertain events and also some areas I noted for further exploration. Many of the following notes are structured from the authors' insight into the demonstrated practices of repeatedly successful forecasters.The book mentions repeatedly the importance of measurement for assessment and revising forecasts and programs. Many people simply don't create any metrics of anything when they make unverifiable and chronologically ambiguous declarations.The book emphasizes the importance of receiving this feedback on predictions that measurement allows, as there is a studied gap between confidence and skill in judgment. We have a tendency to be uninterested in accumulating counterfactuals, but we must know when we fail to learn from it. If forecasts are either not made or not quantified and ambiguous, we can't receive clear feedback, so the thought process that led to the forecasts can't be improved upon. Feedback, however, allows for the psychological trap of hindsight bias. This is that when we know the outcome, that knowledge of the outcome skews our perception of what we thought at the time of the prediction and before we knew the outcome.The main qualities for successful forecasting are being open-minded, careful, and undertaking self-critical thinking with focus, which is not effortless. Commitment to self-improvement is the strongest predictor of long-term performance in measured forecasting. This can basically be considered as equivalent to the popular concept of grit. Studies show that individuals with fixed mindsets do not pay attention to new information that could improve their future predictions. Similarly, forecasts tend to improve when more probabilistic thinking is embraced rather than fatalistic thinking in regards to the perspective that certain events are inevitable.A few interesting findings that the authors expand upon in more detail in the book: experience is important to have the tacit knowledge essential to the practice of forecasting, and that grit, or perseverance, towards making great forecasts is three times as important as intelligence.Practices to undertake when forecasting are to create a breakdown of components to the question that you can distinguish and scrutinize your assumptions; develop backwards thinking as answering the questions of what you would need to know to answer the question, and then making appropriate numerical estimations for those questions; practice developing an outside view, which is starting with an anchored view from past experience of others, at first downplaying the problem's uniqueness; explore other potential views regarding the question; and express all aspects and perspectives into a single number that can be manipulated and updated.Psychological traps to be aware of discussed in the book include confirmation bias, which is a willingness to seek out information that confirms your hypothesis and not seek out information that may contradict it, which is the opposite of discovering counterfactuals; belief perseverance, also known as cognitive dissonance, in which individuals can be incapable of updating their belief in the face of new evidence by rationalization in order to not have their belief upset; scope insensitivity, which is not properly factoring in an important aspect of applicability of scope, such as timeframe, properly into the forecast; and thought type replacement, which is replacing a hard question in analysis with a similar question that's not equivalent but which is much easier to answer.Researched qualities to strive for as a forecaster: cautious, humble, nondeterministic, actively open-minded, reflective, numerate, pragmatic, analytical, probabilistic, belief updaters, intuitive psychologists, growth mindset.The authors then delve into a bit of another practical perspective on forecasting, which involves teams. Psychological traps for teams include the known phenomenon known as groupthink, which is that small cohesive groups tend to unconsciously develop shared illusions and norms that are often biased in favor of the group, which interfere with critical thinking regarding objective reality. There is also a tendency for members of the group to leave the hard work of critical thinking to others on the team instead of sharing this work optimally, which when combined with groupthink, leads the group towards tending to feel a sense of completion upon reaching a level of agreement. One idea to keep in mind for management of a group is that the group's collective thinking can be described as a product of the communication of the group itself and not the sum of the thinking of the individual members of a group.There are some common perceived problems with forecasting, which receive attention in the book: the wrong side of maybe fallacy, which is the thinking that a forecast was bad because the forecast was greater than 50% but the event didn't occur, which can lead to forecasters not willing to be vulnerable with their forecasts; publishing forecasts for all to see, where research shows that public posting of forecasts, with one's name associated with the forecast, creates more open-mindedness and increased performance; and the fallacy that because many factors are unquantifiable due their real complexity, the use of numbers in forecasting is therefore not useful.Some concepts that I took note of for further research from the book were: Bayesian-based application for belief updating, which is basically a mathematical way of comparing how powerful your past belief was relative to some specific new information, chaos theory, game theory, Monte Carlo methods, and systematic intake of news media. These are concepts that I was particularly interested in from the book based on my own interests and that I have continued to explore. This book was very valuable for cohesively bringing together the above concepts in the context of a compelling story, based on the DARPA research project which was compellingly won by the author's team as a product of the research that led to this groundbreaking book.

Ich habe mehrere Interviews mit Phil Tetlock gelesen und mit das Buch mit in den Urlaub genommen. Es gibt einen überraschenden Einblick in das Good Judgement Project, das von der IARPA durchgeführt wurde und die Charakteristik guter Prognosen untersuchen sollte. Es konnte gezeigt werden, dass auch Laien in der Lage sind gute Prognosen zu treffen, wenn sie bestimmte Eigenschaften mitbringen und ein Training in Wahrscheinlichkeitsrechnung erhielten. Ein kleiner Teil konnte sogar 90 % aller Prognosen richtig treffen. Die amerikanischen Geheimdienste kamen im Schnitt auf 60 %. Wer die Erfahrungen aus Perspektive einer dieser so genannten Superforecaster erfahren will, dem empfehle ich zusätzlich das Buch „Sichere Prognosen in unsicheren Zeiten“ von Bruno Jahn.

At first I thought that was just another regurgitate of others work, but this was something additive to the works around statistics and risk.This well written but still insightful. I took away ideas which will help me and which I can use.