I have to admit that statistics have not received much attention in my formal education, which has been relatively wide but not scientific or mathematical. However, I have become increasingly interested in statistical thinking in general in recent years from my experience of teaching Critical Thinking. Most recently I have been reading Daniel Kahneman’s book Thinking, Fast and Slow. This is a very rich book, which I will write more about in general at some other point, but in the second section of the book Kahneman focuses on errors we make in statistical thinking from a psychological point of view. What is fascinating about this for me is the way that showing statistical thinking as a psychological process reveals much wider patterns that are not limited to thinking about statistics as such. They are of much wider interest than just guarding against misinterpretations of data by scientists (important though that may be), and need to engage us all.
Crucially, statistics tell us about what has been observed, not the interpretations that we draw from what has been observed about what is going on. “Data” is distinguishable from “theory”. Of course, all data is to some extent acquired within a framework of theory, because our beliefs direct us in our observations and lead us to collect, or to focus on, certain data. Nevertheless, the process of “statistical reasoning” is generally one of moving from observations to claims. That is why statistical reasoning is not just about statistics as such. It is more broadly inductive reasoning. A personal experience that one’s neighbour is often short-tempered does not necessarily have to be collected as a precise record of numbers of measured occasions when his blood pressure was higher than average. Whether or not it is strictly “statistical” in the sense of being turned into numerical or graphical data, that experience will form the basis of generalisations about him, and expectations for the future, which may be more or less justified.
For Middle Way Philosophy, this kind of reasoning is basically how we get all useful information, about both facts and values. To try to access absolute a priori information from any other source than experience is deluded. Inductive reasoning is incremental, not absolute, but central to my thesis is the claim that it is capable of objectivity – indeed this is where objectivity lies, rather than in the circularity of metaphysical reasoning. Kahneman’s work on the delusions that assail us in this area, then, are valuable because they cast light on how we can be more objective in our thinking.
One of Kahneman’s most striking examples is that of ‘Linda’ (ch.15). Linda is a fictitious example used to test statistical reasoning. We are told that she is a bright philosophy major who is very concerned with issues of discrimination and social justice. Then, we are asked, is it more likely that Linda is a bank teller, or a feminist bank teller? Most people answer the latter, even though it is impossible for it to be more likely that any given individual is a member of a subset (feminist bank tellers) than of the larger set of which it forms a part (bank tellers). Kahneman’s central point, repeated in many different instances, is that we rely much more on associative reasoning than on statistical reasoning, and that we take the easy path of association (because Linda seems to be the sort of person who might be a feminist) in preference to the hard effort of thinking out probabilities in relation to each other.
This single example may not bring out the full importance of such a point. Kahneman is talking about thinking based on stereotypes as opposed to thinking that tries to get to grips with the complexity of the conditions around us. The false assumptions we often make about others because of this have obvious social and political importance. In science, too, the effectiveness of investigation can be greatly impeded by these sorts of biases. Kahneman shows both that statistical or scientific training makes little difference to people’s habitual thinking, and that scientists often make mistakes (like relying on sample sizes that are too small) due to relying on inadequate (but easy) intuitions rather than making the effort to think through what is required.
Kahneman does also offer some advice on how we can correct this sort of bias in our thinking. One is not to ‘neglect the base rate’, that is, to think about overall probabilities first. Our immediate experiences or new information should then modify those initial probabilities. So, in the case of Linda, the chances of her being a bank teller are actually probably greater than her being a philosophy teacher – because there are a lot more bank tellers in the population than philosophy teachers. We thus have to start with that ‘base rate’ and modify it in relation to the specific information we have about Linda, rather than jumping to the conclusion that an improbable occupation is likely based only on associations.
This is in line with the approach to statistical inference in Bayes’ Theorem – a statistical formula used for modifying past probabilities in the light of new evidence. For a long time I have been rather resistant to Bayes’ Theorem – or rather specifically to the idea that new evidence should be used to modify our previous estimates of probability rather than that we should just estimate probability based on new evidence. However, Kahneman has helped me to understand that the basic thinking behind Bayes’ Theorem is useful: indeed it is very much in line with Middle Way Philosophy because it tries to integrate thinking based on experience at different times, rather than relying excessively on one set of estimates with its possible delusions. Again, the general approach is Bayes does not have to be formally statistical: we could use it more broadly to just consciously take account of past judgements when struck by new evidence. If my bad-tempered neighbour suddenly becomes charming, for example, I should not just be bowled over and forget his previous bad-temperedness too quickly. Rather I should modify my previous estimate of his general bad-temperedness by the new evidence of his possible pleasantness, and it should take a while before I switch from thinking him more likely to be bad-tempered to thinking him more likely to be pleasant.
However, I’m still not sure that Bayes’ Theorem should be applied to every possible example. For example, I still have a lot of resistance to the standard ‘correct’ answer to the Monty Hall Problem. Here a quizmaster first offers a contestant a choice between three doors, two of which have goats behind them and one a car. The contestant chooses one door, then the quizmaster opens another one to reveal a goat. The question is then whether it is in the contestant’s interests (assuming he wants a car rather than a goat!) to switch from his original choice. If you apply Bayesian reasoning, the contestant should modify his previous judgement of the probability of any given door having a car behind it (1 in 3) according to the new evidence (by which it is 1 in 2), resulting in a higher probability (2 in 3) if the contestant switches doors and only 1 in 3 is he sticks with the current door.
My anti-establishment thought (which may be wrong) is that this scenario is not an appropriate application of Bayesian thinking. The reason why Bayesian reasoning generally works is psychological: in a generally mysterious universe where we have little idea what is going on, it helps to overcome our over-attachment to current evidence and makes us take into account wider evidence from other times. In the case of the Monty Hall problem, however, I suspect that Bayesian reasoning is being applied as though it was a ‘natural’ law about the world itself rather than a generally effective strategy for making judgements about it. This is because there is nothing mysterious about the game show set-up. It is nothing like the case of Linda, where we are given only a few words of description make judgements about a complex person in a complex world. In the game show, the probabilities have been artificially engineered to be clear. At first, the probability of picking a car behind any given door was 1 in 3, and after one door has been opened, it is 1 in 2. There is no reason why we should modify our new judgement by a past judgement, because in this particular case the new judgement is clearly adequate in an entirely transparent and predictable constructed scenario.
For me this illustrates the obvious peril with statistical reasoning. If you get too concerned with the figures themselves and the ‘laws’ that you think apply to them, you forget that these figures are abstractions from experience, and that experience itself is the best and most objective indicator of anything. Statistical reasoning can be extremely helpful when it assists us in overcoming our delusions and getting closer to what is really going on, but if its procedures are inappropriately rigidified, or its limited relationship to a complex world is forgotten, then it can start being an impediment.