Dr David Hardman, Senior Research Analyst at the Rail Safety and Standards Board.
Many thanks to the attentive and engaged audience who passed up the opportunity to watch the Belgium vs. France World Cup semi-final in order to hear my presentation to the branch on 10th July (note: I am also a committee member for this branch). As no-one was allocated to write a blog for this event, our Chair (James Barr) asked if I would give a short summary here.
There have been numerous popular books about decision making over the last decade or so, perhaps spurred by the rising interest in ‘behavioural economics’ in the wake of the economic crash in 2008 (behavioural economics is essentially the application of psychological ideas within the realm of economics). The most notable of these is ‘Thinking: Fast and Slow’ by the Nobel laureate, Daniel Kahneman. The story of his academic partnership and friendship with the late Amos Tversky has been memorably told in Michael Lewis’s book ‘The Undoing Project’. My own textbook, ‘Judgment and Decision Making: Psychological Perspectives’, appeared in 2009 and would have been in preparation at the publisher just as the world economy was going into meltdown.
There are a number of strong general conclusions to be drawn from the JDM literature, of which I’ll mention three. Firstly, many studies have shown that people struggle to combine and compare multiple items of information when making judgments (despite any subjective feelings that they are doing so). Instead, they often fall back on simplifying heuristics. Kahneman has shown how people unconsciously replace difficult questions with easy ones, such as when they replace a question about probability with one about typicality.
A second conclusion is that people do not have stable preferences that are revealed by the choices they make, but rather they construct their preferences at the moment of choice. Because of this, people’s preferences can be influenced by the context of the choice. For example, in several studies Chris Hsee showed that two choice options were valued differently, depending on whether participants saw the two options together or just one of the options. In one such study, people who were able to compare two tubs of ice cream placed a higher value on a larger quantity scoop that didn’t reach the brim of a tall cup, as compared to a smaller quantity scoop that was overfilling the brim of a tiny cup. However, these valuations were reversed in two groups of people who only viewed one of the cups.
Thirdly, when making choices, people don’t choose between ‘things’; they choose between descriptions of things. In the well-known ‘Disease Problem’, participants are asked to choose between one of two medical programmes in response to a disease that is expected to take 600 lives. When the options are described in terms of the number of lives expected to be ‘saved’ people tend to opt for an option that will save 200 lives for sure, as opposed to a ‘risky’ option that has a one-third chance of saving everybody and a two-thirds chance of saving nobody; but when the options are described in terms of the number of people who are likely to die, participants more often choose the risky option.
The ‘framing effect’ observed in the disease problem was predicted and explained by Prospect Theory (Kahneman and Tversky, 1979). This theory says that people represent potential outcomes as gains or losses from a subjective reference point. People are especially sensitive to changes near the reference point and are more sensitive to perceived losses than to gains of an equivalent amount. Consequently, people tend to be risk averse when considering potential gains and risk-seeking when they consider potential losses.
In my talk I discussed some recent developments in the JDM field, including some earlier results that have come under question. The first of these concerned the ‘hot hand fallacy’. In American basketball there is a longstanding belief that players who have successfully placed the ball in the basket on successive attempts are more likely to be successful on their next attempt (as compared to players who have had a series of misses). These players are said to have the ‘hot hand’. In a much-cited 1985 paper, Gilovich, Vallone and Tversky analysed the performance nine top players over the course of a year and found that a sequence of hits was not more likely to be followed by another hit than was a sequence of misses. They labelled belief in the hot hand as a fallacy. However, a reanalysis by Miller and Sanjurjo, currently in press at Econometrica, shows that Gilovich and colleagues themselves made a conceptual error in their original analysis, and that the players they studied really did demonstrate a hot hand effect. The explanation for this is beyond the scope of this blog, except to say that the error in the original analysis lies in the fact that the authors identified target attempts on the basis of having already identified that they were preceded by a sequence of hits or misses. A relatively accessible account of their analysis can be found at The Conversation.
Another textbook bias that has come under scrutiny is that of unrealistic optimism, the phenomenon whereby people perceive that they are less likely to experience negative events than the average person (and more likely to experience positive events, although in fact the empirical evidence is weaker for positive events). Adam Harris at UCL, and his colleagues, has argued that the appearance of unrealistic optimism in laboratory studies is an artifact arising from the rarity of the events that are assessed and the restricted nature of the scale that participants are asked to use for their responses. Through a thought experiment he has shown that even fully rational participants can give the appearance of unrealistic optimism. In experiments that control for some of the shortcomings of earlier research, he has found no evidence for unrealistic optimism.
My talk also examined a couple of studies questioning how far Prospect Theory can go in explaining everyday behaviour. The original prospect theory paper referred to the ‘last race of the day effect’, identified in a 1956 paper, whereby race track bettors increasingly bet on longshots on the last race of the day. Kahneman and Tversky explained this in terms of loss aversion, whereby bettors who had lost money during the course of the day (the majority!) switched to longshots on the final race in an attempt to recoup their losses. However, some more recent research has failed to find the last-race-of-the-day effect, suggesting that either the original research suffered from sampling error or that savvy bettors have got wise to the effect and changed their behaviour. In a 2016 paper, Craig McKenzie ran three experiments designed as simulations of the race track, in which participants - not experienced bettors - placed bets on each of several rounds, always knowing when the last round was about to happen. McKenzie did find an increased tendency to bet on longshots in the final round, but - contrary to Prospect Theory - this effect was true for bettors who had previously made gains as well as the ones who had made losses. He concluded that most people just like to “go out with a bang”.
In a 1997 paper, Colin Camerer also called upon loss aversion as an explanation for the behaviour of New York cab drivers, who did not appear to structure their working hours so as to maximise their earnings. Through an analysis of their trip sheets, Camerer found that they tended to quit early on busy days and to work longer on days when there were fewer customers. He argued that the drivers set themselves a daily earnings target (their reference point, in Prospect Theory terminology), and quit once this target was met.
However, critics have suggested that this study failed to control for certain confounds. For instance, New York cab drivers often hold down more than one job. It could be that events such as public holidays and major sports events could influence demand in their other job, as well as the demand for cabs. Likewise, some drivers may prefer to attend a sporting event rather than drive their cab when it is on. Tess Stafford, in a 2015 paper, analysed a group of workers less susceptible to such confounds - the commercial spiny lobster fishermen of Florida. These workers require a license to work, typically own their own boat, and have to submit a trip sheet to the local conservation body. Moreover, variations in the availability of lobster are fairly predictable. The supply is higher earlier in the season and during the period of the new moon. Stafford found that these workers did not demonstrate the reference-dependence observed in the NYC cabbies; rather they did structure their work in such a way as to maximise their earnings.
The original research that was included in the 1979 Prospect Theory paper was of the pencil-and-paper type, where participants had to choose between different hypothetical options. Whilst this identified the behaviour described earlier - risk aversion for gains and risk-seeking for losses - it also found the reverse of this when very small probabilities were involved. Small probabilities were said to have a large impact on behaviour, as with people’s purchase of lottery tickets and insurance. Over the last 16 years, a series of studies have looked at what happens when people experience decision outcomes rather than simply considering them as hypothetical scenarios (Decisions from Experience vs. Decisions from Description).
In decisions from experience participants typically make a sequence of choices between one of two doors that appear on a computer screen. After each choice they learn what the outcome is. The outcomes for each door represent a distribution of outcomes that can be summarised as probabilities. In some studies, people receive a payoff with each choice. In other studies, people sample from each door without a payoff, but once they feel they have sampled enough then they make a choice that does deliver a real payoff. These studies show an opposite pattern to decisions from description in that people are less rather than more sensitive to small probabilities. The findings are not necessarily in contradiction to Prospect Theory, though, because it may be that some participants never actually sample the low probability outcome. There is some evidence that Prospect Theory does actually account for decisions, when the analysis is based on the outcomes that people experienced.
One area of JDM that has aroused some controversy is that relating to the role of unconscious processes, and especially work on unconscious priming. Chapter 4 in Kahneman’s book ‘Thinking: Fast and Slow’ discusses these studies. Yet a prominent failure to replicate one of the best-known priming studies (in which people walk more slowly after being exposed to stereotypes of older age) has led much of this work to be criticised. This blog article reports analyses that suggest most of the studies discussed in Kahneman’s fourth chapter are statistically under-powered. The problem with social priming studies relates to the ongoing replication crisis (so-called) within psychology. Chris Chambers has published a terrific book, ‘The Seven Deadly Sins of Psychology’, in which he forensically analyses the problems in much methodological practice that have given rise to many false findings being published. Chambers also proposes a number of remedies that researchers should apply (or be made to apply) in order to clean up their act.
Unconscious processes are also addressed in a provocative recent book, ‘The Mind is Flat’ by Nick Chater. Chater argues that, contrary to decades of psychoanalytic theory, the mind has no ‘hidden depths’ because it has no depths. That is not to say that there are no unconscious processes. Clearly there are, but these processes are not concealed versions of potentially conscious thoughts. Rather, our conscious thoughts arise out of these unconscious processes. For Chater, the brain is a sense-making machine, piecing together myriad sensory impressions and memory fragments to make sense of the world in the moment. Specifically, the brain has evolved to provide us with a stable picture of the world, but not to provide us with an account of its own workings in doing so.
Many of Chater’s examples come from visual perception, such as the fact that the erratic saccades of our eyes are completely missing from our seamless perceptions. Similarly, most of our visual receptors are specialised for black and white vision, yet we perceive the world in colour. This is simply the result of our brain’s sense-making activities. Some of the implications for JDM are discussed in Chater’s Chapter 6. For instance, in one extraordinary series of studies, Petter Johansson and his colleagues asked participants to choose the most attractive face from a pair, each of which was presented on cards. After making their choice, the experimenter used sleight-of-hand to present them with their non-preferred face whilst telling them this was the face they had chosen. Under questioning, most participants then had no difficulty in providing an explanation as to why they had chosen a face that they had not in fact chosen.
Chater’s view of the brain/mind is consistent with the preference construction notion, described earlier, according to which (contrary to classical economics) people construct their preferences at the moment of choice according to the context of that choice. Chater also presents evidence from work on attention, showing that people are, to all intents and purposes, only able to focus on one thing at a time. This fits with decades of JDM research showing that people struggle to integrate multiple items of information when making judgments and decisions, typically falling back on heuristics.
This, then, was where my talk finished. Questions from the audience at this point asked about a variety of things I hadn’t covered in my talk, including personality and individual differences, and workplace applications of JDM research. JDM is a wide-ranging area, and I suspect the audience for the talk also came from a variety of backgrounds (a show of hands revealed there were very few who were actually working in academia). Nonetheless, some of these topics are covered in my own textbook!
Dr David Hardman is author of the book: