How to improve your figures #7: Don’t use a third dimension

Most static figures show information in two dimensions (with a horisontal dimension and a vertical dimension). This works really well on the screen as well as on paper. However, once in a while you also see figures presenting figures with a third dimension (3D). It is not necessarily a problem adding a third dimension if you are actually using it to visualise additional data in a meaningful manner. If you have three continuous variables constituting a space, it can make sense to show how observations are positioned in this space. Take for example this figure generated from the interflex package in R:

However, unless you have a clear plan for how, and specifically why, you want to present your data in three dimensions, my recommendation is to stick to two dimensions. I will, as always, present a few examples from the peer-reviewed scientific literature. These are not necessarily bad figures, just figures where I believe not having the third dimension would significantly improve the data visualisation.

First, take a look at a bar chart from this article on the willingness to trade off civil liberties for security before and after the 7/7 London terrorist attack:

The third dimension here is not adding anything of value to the presentation of the data. On the contrary, it is difficult to see whether item 3 is greater than or equal to 3 on the average willingness to trade-off. As Schwabish (2014) describes in relation to the use of 3D charts: “the third dimension does not plot data values, but it does add clutter to the chart and, worse, it can distort the information.” This is definitely the case here.

Second, let us look at a figure from this article showing correlations between voting preferences and ten distinct values:

Again, the third dimension is not adding anything of value to the visualisation. It is only making it more difficult to compare the values. The good thing about the figure is that it shows the exact values. However, it would be much better to present the information in a simple bar chart like this one:

In the figure above, I show the values without the use of any third dimension. Notice how it is easier to identify the specific values (e.g. “Uni”) in the bar chart compared to the 3D chart, when we do not need to move our eyes in a three-dimensional space.

Third, let us look at an example from this article where we actually add additional information to the third dimension:

The figure shows the number of articles referring to domestic nuclear energy in four different countries from March 12, 2011 to April 10, 2011. However, it is very difficult to compare the numbers in the different countries, and if there are more articles in the UK and/or France than in Switzerland, we will not be able to see how many articles there are in the latter country. This is not a good visualisation. Instead, it would have been better with a simple line graph in two dimensions.

Again, there can be good reasons to visualise information in three dimensions, but unless you have very good reasons to do so, my recommendation is to keep it simple in 2D. In other words, in most cases, 2D > 3D.

25 interesting facts #8

176. Governments in countries that experienced SARS in 2003 were quicker to implement social distancing policies to combat COVID-19 in 2020 (Ru et al. 2021)

177. Political parties that suffer electoral setbacks are more likely to change name (Kim and Solt 2017)

178. People underestimate their compliments’ value to others (Boothby and Bohns 2021)

179. People prefer to experience events at the same time as others (Shaddy et al. 2020)

180. The online role-playing game World of Warcraft experienced a full-blown epidemic in 2005 (Lofgren and Fefferman 2007)

181. Dissertations contain more unsupported hypotheses than journal articles (O’Boyle Jr. et al. 2017)

182. Economists believe markets are more efficient than they are (Page and Siemroth 2020)

183. People overestimate the value of adding members to a team and underestimate coordination costs (Staats et al. 2012)

184. By 1980 in the U.S., the oil industry promoted false information about climate change (Franta 2021)

185. Men are more likely to wear beards in countries with lower health and higher economic disparity (Dixson and Lee 2020)

186. Monozygotic twins differ on average by 5.2 early developmental mutations (Jonsson et al. 2021)

187. Political information presented with humour is easier to remember (Coronel et al. 2021)

188. Nonreplicable publications are cited more than replicable ones (Serra-Garcia and Gneezy 2021)

189. Firms headquartered in more corrupt states in the United States provide higher payouts to their shareholders (Hossain et al. 2021)

190. While not many celebrities run for public office, those who do usually win (Knecht and Rosentrater 2021)

191. Humans organize their social groups in similar ways as their animal neighbours (Barsbai et al. 2021)

192. The Brexit referendum led to a short-term increase in hate crimes (Devine 2020, Piatkowska and Lantz 2021)

193. People are more likely to share information that is consistent with their political orientation (Ekstrom and Lai 2021)

194. Nouns slow down speech across structurally and culturally diverse languages (Seifart et al. 2018)

195. There is no evidence that citizens on the political right are especially likely to endorse false political information (Ryan and Aziz 2021)

196. Approximately one-quarter of bitcoin users were involved in illegal activity in the period from 2009 to 2017 (Foley et al. 2019)

197. People with foreign-sounding names are 10% less likely to get a response on a training session request from amateur football clubs in Europe (Gomez-Gonzalez et al. 2021)

198. While friendship jealousy may be negative to experience, its purpose is to help maintain friendships (Krems et al. 2021)

199. In China, police stations are more likely to be located within walking distance of foreign religious sites (Liu and Chang 2021)

200. Humblebragging is less effective than straightforward bragging (Sezer et al. 2018)


Previous posts: #7 #6 #5 #4 #3 #2 #1

How to improve your figures #6: Don’t use bar graphs to mislead

In a previous post, I argued that the y-axis can be misleading under certain conditions. One of these conditions is when using a bar graph with a non-zero starting point. In this post I will show that bar graphs can be misleading even when the y-axis is not misleading.

In brief, bar graphs do not only convey certain estimates or data summaries but also an idea about how the data is distributed. The point here is that our perception of data is shaped by the bar graph, and in particular that we are inclined to believe that the data is placed within the bar. For that reason, it is often better to replace the bar graph with an alternative such as a box plot. Here is a visual summary of one of the key points:

There is a name for the bias: the within-the-bar bias. Newman and Scholl (2012) showed that this bias is present: “(a) for graphs with and without error bars, (b) for bars that originated from both lower and upper axes, (c) for test points with equally extreme numeric labels, (d) both from memory (when the bar was no longer visible) and in online perception (while the bar was visible during the judgment), (e) both within and between subjects, and (f) in populations including college students, adults from the broader community, and online samples.” In other words, the bias is the norm rather than the exception in how we process bar charts.

Godau et al. (2016) found that people are more likely to underestimate the mean when data is presented in bar graphs. Interestingly, they did not find any evidence that the height of the bars affected the underestimation. There is even some disagreement about whether bar charts should include zero (e.g., Witt 2019). Most recently, however, Yang et al. (2021) have demonstrated how truncating a bar graph persistently (even when presented with an explicit warning) misleads readers.

This is an important issue to focus on. Weissgerber et al. (2015) looked at papers in top physiology journals and found that 85.6% of the papers used at least one bar graph. I have no reason to believe that these numbers should differ significantly from other fields using quantitative data. For that reason, we need to focus on the limitations of bar graphs and potential improvements.

A limitation with the bar graph is that different distributions of the data can give you the same bar graph. Consider this illustration from Weissgerber et al. (2015) on how different distributions of the data (with different issues such as outliers and unequal n) can give you the same bar graph:

Accordingly, bar graphs will often not provide sufficient information on what the data actually looks like and can even give you a biased perception of what the data looks like (partially explained by the within-the-bar bias). The solution is to show more of the data in your visualisations.

Ho et al. (2019) provide one illustrative example on how to do this when you want to examine the difference between two groups. Here is their evolution of two-group data graphics (from panel a, the traditional bar graph, to panel e, an estimation graphic showing the mean difference with 95% confidence intervals as well):

From panel a to panel b, you can see how we address some of the within-the-bar bias, and further show how the data points are actually distributed when looking at panel c. This is just one example of how we can improve the bar graph to show more of the data, and often the right choice of visualisation will depend upon what message you will need to convey and how much data you will have to show.

That being said, there are some general recommendations that will make it more likely that you create a good visualisation. Specifically, Weissgerber et al. (2019) provide seven recommendations where I find four of them relevant in this context (read the paper for the full list as well as the rationale for each):

  1. Replace bar graphs with figures that show the data distribution
  2. Consider adding dots to box plots
  3. Use symmetric jittering in dot plots to make all data points visible
  4. Use semi-transparency or show gradients to make overlapping points visible in scatter plots and flow-cytometry figures

Bar graphs are great, and definitely better than pie charts, but do consider how you can improve them in order to show what your data actually looks like beyond the bar.

Visualizing climate change with stripes

Climate change is abstract. We do not personally experience climate change in our day-to-day activities (although cimate change is detectable from any single day of weather at global scale, cf. Sippel 2020), and if we are to understand climate change, data – and in particular data visualisation – is crucial. I have recently been reading some literature on the relevance of visualisations and uncertainty in relation to climate change. There has, for example, been some work on the role of visual imagery on public attitudes towards climate change (e.g., Bolsen et al. 2019 and van der Linden et al. 2014) and how uncertainty may make people more likely to accept climate scientists’ predictions (see Howe et al. 2019).

Scientific evidence and data is not enough and we need to consider the best possible ways to visualise climate change. One of the most popular visualisations is the iconic #ShowYourStripes figure that shows global average annual temperatures from 1850 to 2019:

I believe it is a good visualisation but I have a few objections. First and foremost, I like numbers and I do not like how simplified the presentation is. What exactly are the numbers we are looking at here? Should I be concerned? If the dark blue is -0.01°C and the dark red is 0.01°C, is the story here one of change or stability? What is the average temperature in the sample and how much variation is there? Call me old-fashioned, but I don’t think a data visualisation is working if you are simply saying that something is increasing over time.

Interestingly, you can also download the figure with labels, but this provides no information on what values the colours are showing – only the time dimension:

The lack of a meaningful legend is an issue here. It would not make the visualisation more complicated but only help better understand the changes.

Second, I am not convinced that the tool is actually good if you want to show your stripes (and that is what we are being told to do afterall). How useful is the visualisation when we go beyond the global averages? To illustrate my concern, here is the visualisation I got for Denmark:

Sure, there is a story to tell, but I begin to miss certain details. Again, what are the values I am looking at? How much variation is there? And most importantly, how much uncertainty is there over time?

Third, I do not like the extreme colour scale used to illustrate the changes from 1850 (or 1901) to 2019. We know that the temperatures are going to increase in the future and the visualisation can give the false impression that we are already at a peak. I know this is not the message that the figure wants to convey, but people might look at the figure and conclude that we have seen the worst of what is to come.

It is not a bad visualisation. However, it is definitely not the best. You can check out the Vis for Future project from 2019 and find a lot of other great visualisations (the ‘Warming Stripes’ was one of the winners). I can also recommend a lot of the work by Neil Kaye, e.g. this and this. A recent example of a good visualisation is this visualisation from the National Oceanic and Atmospheric Administration on annual temperatures compared to the 20th-century average in the United States (notice how the legend is making it easier to see what we are actually looking at):

Climate change is abstract, but good visualisations with labels can help us better understand the global changes in our climate.

Potpourri: Statistics #75

Introducing pewmethods: An R package for working with survey data
Exploring survey data with the pewmethods R package
Weighting survey data with the pewmethods R package
Analyzing international survey data with the pewmethods R package
autumn: Fast, Modern, and Tidy Raking
Data science for economists
Papers about Causal Inference and Language
Yale Applied Empirical Methods PHD Course
Spreadsheet Munging Strategies
Visual Vocabulary: Designing with data
What can we learn from a country’s diplomatic gifts?
Map, Walk, Pivot
The Epidemiologist R Handbook
Machine learning with {tidymodels}
Choose your own tidymodels adventure
Applied Spatial Statistics with R
ggplot: the placing and order of aesthetics matters
Introduction to Functional Data Analysis with R
Visualizing Distributions with Raincloud Plots with ggplot2
A Chat with Andrew on MLOps: From Model-centric to Data-centric AI
ISLR tidymodels Labs
Plotting maps with ggplot2
R instructions for our research projects
A gentle introduction to deep learning in R using Keras
Everything You Always Wanted to Know About ANOVA
Replication Materials for “The Flying Bomb and the Actuary” (Shaw and Shaw, 2019)
Colors and emotions in data visualization
Rookie R mistakes
10 Tips to Customize Text Color, Font, Size in ggplot2 with element_text()
Writing unit tests in R
The Good, the Bad and the Ugly: how to visualize Machine Learning data
A curated list of APIs, open data and ML/AI projects on climate change
R for SEO
Using Geospatial Data in R
Good Data Scientist, Bad Data Scientist
The Evolution of a ggplot (Ep. 1)
Do Wide and Deep Networks Learn the Same Things?


Previous posts: #1 #2 #3 #4 #5 #6 #7 #8 #9 #10 #11 #12 #13 #14 #15 #16 #17 #18 #19 #20 #21 #22 #23 #24 #25 #26 #27 #28 #29 #30 #31 #32 #33 #34 #35 #36 #37 #38 #39 #40 #41 #42 #43 #44 #45 #46 #47 #48 #49 #50 #51 #52 #53 #54 #55 #56 #57 #58 #59 #60 #61 #62 #63 #64 #65 #66 #67 #68 #69 #70 #71 #72 #73 #74

Noise

The world is full of noise. This is not a novel insight. Luckily, the power of statistical models to predict human behaviour is limited, and if a model is able to predict all relevant variation in an outcome of interest, we should be concerned about overfitting and other potential problems. There are good reasons why R² is rarely anywhere near 1 in regression outputs and why regularization in machine learning is such a hot topic.

Noise is part of the error term in our statistical models. A good model is not trying to explain everything about the world (and not have any noise at all). Unsurprisingly, within the domain of behavioural economics, the idea of noise as a core component of human behaviour has been known for decades. Humans are not (fully predictable) machines, partially because there are limits to how we process information. The concept of bounded rationality, introduced by Herbert Simon, is one good example of how we can understand the existence of noise.

The psychologists Daniel Kahneman and Amos Tversky improved our understanding of human behaviour by convincingly demonstrating that a lot of what goes into the error term, i.e., what we treat as noise, is not in fact noise but predictable behaviour rooted in cognitive biases. Here is how the economist Richard Thaler describes this historical development in his book, Misbehaving: The Making of Behavioural Economics:

I believe many economists ignored [Herbert] Simon because it was too easy to brush aside bounded rationality as a “true but unimportant” concept. Economists were fine with the idea that their models were imprecise and that the predictions of those models would contain error. In the statistical models used by economists, this is handled simply by adding what is called an “error” term to the equation. Suppose you try to predict the height that a child will reach at adulthood using the height of both parents as predictors. This model will do a decent job since tall parents tend to have tall children, but the model will not be perfectly accurate, which is what the error term is meant to capture. And as long as the errors are random—that is, the model’s predictions are too high or too low with equal frequency—then all is well. The errors cancel each other out. This was economists’ reasoning to justify why the errors produced by bounded rationality could safely be ignored. Back to the fully rational model! Kahneman and Tversky were waving a big red flag that said these errors were not random. Ask people whether there are more gun deaths caused by homicide or suicide in the U.S., and most will guess homicide, but in fact there are almost twice as many gun deaths by suicide than homicides. This is a predictable error.

Over the years, thousands of studies have tried to remove noise from the models and give the predictable errors unique names, often in the form of specific cognitive biases. Today, behavioural economics is a thriving research agenda uncovering too many predictable errors (biases) to cover in a blog post (or a book for that matter!). In his book from 2011, Thinking, Fast and Slow, Daniel Kahneman summarised a lot of the research within the field (alas, a lot of this research relied on noisy data and, accordingly, does not hold up well).

Three years prior to the publication of Thinking, Fast and Slow, the book Nudge: Improving Decisions about Health, Wealth, and Happiness, by Richard Thaler and Cass Sunstein, brought behavioural economics to the policy-makers toolbox. The basic principle is that traditional economic models of human behaviour will not be sufficient if we are to reach policy objectives. In other words, there is too much noise generated by traditional models that behavioural economics can help us understand – and even take advantage of if we are to improve decisions about … health, wealth, and happiness.

However, despite decades of research on minimising the error term, i.e., shed light on the predictable errors in human judgments and decisions, the fact remains that there is still a lot of noise in human judgments. But maybe we focus too much on what we can predict and too little on noise? This is the topic of the new book, Noise: A Flaw in Human Judgment, by Daniel Kahneman, Olivier Sibony and Cass Sunstein. In other words, the book is not trying to simply understand biases, but this thing called noise. Or, as the authors introduce the focus of the book: “The topic of bias has been discussed in thousands of scientific articles and dozens of popular books, few of which even mention the issue of noise. This book is our attempt to redress the balance.”

Is the book successful in redressing this balance? Maybe, but I am not convinced. Here is my main issue with Noise: the book is trying to both argue that noise is rarely acknowledged and we know very little about noise (and, for that reason, we need new concepts to study noise) and summarise decades of research on noise (including research done by the authors themselves decades ago). The book is both trying to set a new research agenda with actionable insights (what Sunstein did with Nudge) and introduce the reader to decades of research (what Kahneman did with Thinking, Fast and Slow). You can’t have your cake and eat it too.

In reading the book you will learn about different types of noise, in particular system noise, level noise, pattern noise and occasion noise. Despite the graphical presentation of the sum of squares in various chapters, and especially in chapter 17 on the components of noise, it is difficult to follow the reasoning throughout the book. I am left with the feeling that some additional conceptual work is needed in order for the concepts to fully work. Alas, upon reading the book, the added value of the concepts remains limited. Also, at one place in the book, it is described that system noise can be broken into three components of noise (level, pattern, and occasion), whereas the authors later write “System noise can be broken down into level noise and pattern noise.” I believe the authors could have done a much better job by providing more conceptual clarity, for example with tables and additional useful examples to better demonstrate and discuss the usefulness and relevance of the concepts.

In particular, I would like to see the authors use existing concepts in the scientific literature instead of pretending that they are the first to come up with concepts to think about reliability. I can understand if the authors don’t want to rely on too many concepts, but an Appendix listing the relevant and related concepts would be great. Instead, bhe book begins by introducing the well-known distinction between bias (validity) and noise (reliability) without using mentioning reliability. (Surprisingly, the book is doing what it can not to mention the word reliability more than necessary.) Here the core distinction is also introduced: bias is systematic deviation and noise is random scatter.

The first chapter introduces a key theme of the book, namely that “wherever there is judgment, there is noise—and more of it than you think”. The authors then set out to understand noise in various topics, such as medicine, child custody decisions, forecasts, asylum decisions, personnel decisions, bail decisions, forensic science, and grant patents. However, the book is not interested in random scatter, but to explain systematic components to noise. That is, to actually focus more on biases than noise. Here is how chapter 1 begins:

Suppose that someone has been convicted of a crime—shoplifting, possession of heroin, assault, or armed robbery. What is the sentence likely to be?
The answer should not depend on the particular judge to whom the case happens to be assigned, on whether it is hot or cold outside, or on whether a local sports team won the day before.

This could have been the introduction to a lot of recent pop psychology books with one-word book titles. Again, the problem is that we are not really interested in noise here, but rather to show that there is less noise (by adding variables on judge characteristics, outdoor temperature, sport results, etc.). Accordingly, the book is more about showing that what we believe is noise is not actually noise. This is the noise paradox. Once you can explain random scatter, it is no longer random. Once we can explain noise, there is less noise. What the book is doing with a lot of examples is to say that there is less noise than we think there is (whereas the authors interpret this as there is more noise than we think). In sum, there are contradictory forces at play. On the one hand, the authors want to explain noise (i.e., say that we can understand and predict noise). On the other hand, the authors want to say that there is more noise than we expect (cf. “wherever there is judgment, there is noise—and more of it than you think”).

This focus on showing how “noise” matters, or how small and irrelevant factors matter, also brings some low-quality research into the book (see, for example, here and here). To be fair, the authors also rely on a lot of high-quality research, but they use low-quality research to make some extreme statements. Accordingly, the book is worst when it provides conclusions such as “If you are suffering political persecution in your home country and want asylum elsewhere, you should hope and maybe even pray that your hearing falls on a cool day.” Again, this has more to do with focusing on noise in data than providing actionable insights on how to deal with noise. This is also how you end up with coverage about the book such as “Why You Shouldn’t Buy Bitcoin When You’re Hungry“. In my view, such articles add more noise than signal to our understanding of human behaviour. And it is not doing a good job in redressing the balance.

The problem with the definition of noise in the book (or rather the lack hereof) is that every difference we can imagine can be understood as noise. Anywhere there is variation, there is noise. For example, because all countries in the world did not respond to the COVID-19 crisis in a similar manner, we can conclude that there is noise. Or, as the authors formulate it: “For another exercise in counterfactual thinking, consider how different countries and regions responded to the COVID-19 crisis. Even when the virus hit them roughly at the same time and in a similar manner, there were wide differences in responses. This variation provides clear evidence of noise in different countries’ decision making.” This is not the best way to think about noise, nor a good example of noise in decision making. (Also, apparently, there is so much noise in the world, yet here is the cognitive bias that makes you panic about a pandemic.)

It is old news that people do not respond to treatments (events, news, pandemics, Sundays, hunger, books, etc.) in a similar way and it is not surprising that all countries did not respond with the exact same policies to the COVID-19 pandemic. Even scientific studies differ in the results they find and we now have several meta-analytic techniques we can rely on to understand how effects vary over different persons, locations, treatments, and outcomes. However, if Noise is the first book you ever read, you will have the impression that very little work has been done to understand such differences in treatment effects.

The statement that noise can be any variation also means that the book is dealing with different topics that should not necessarily be put together if we are to reduce the noise in our thinking. The authors write that noise “is variability in judgments that should be identical.” This means that noise is both treatment heterogeneity (“If every respondent makes the same mistake, there is no noise.”) and measurement error (“Can we agree on an anchor case that will serve as a reference point on the scale?”). However, the authors do not convincingly show why we should use the same concept to consider both treatment heterogeneity and measurement error. On the contrary, it seems like the authors rely on a catch-all definition of noise solely for the purpose of bringing different points and arguments together within the same book. This also makes the book very difficult to review. There is simply too much going on.

Overall, I would have liked to see the authors focus more systematically on how little we can actually explain (instead of how much we can explain). For all the examples they provide throughout the book, to what extent are we actually able to predict behaviour? At some point the authors argue that correlations of about .2 are quite common in human affairs, but I find such interpretations too optimistic based on what we know from the replication crisis within psychology (and related fields). Speaking of correlations, I have also seen some discussions on how the book misunderstands how the absence of a correlation is not evidence of no causation (Rachael Meager made this point on Twitter, and see the discussion on Andrew Gelman’s blog where Daniel Kahneman also responds).

And speaking of “speaking of”. Each chapter ends with a few “Speaking of …” bullet points to capture the key points relevant to the chapter in question. However, these are often symptomatic for the fact that the genereal insights provided throughout the chapters are trivial at best and confusing at worst. Consider these randomly selected examples: “To fight noise, they first have to admit that it exists.”, “Wherever there is judgment, there is noise—and that includes reading fingerprints”, “Before we start discussing this decision, let’s designate a decision observer” and “We have kept good decision hygiene in this decision process; chances are the decision is as good as it can be”.

Ironically, the book is best in part 6 when the authors discuss the seven objections to reducing or eliminating noise: 1) it is not always cheap to reduce noise, 2) reducing noise can introduce bias, 3) eliminating noise can reduce the feeling of respect and dignity, 4) noise is a prerequisite for moral and political evolution, 5) noise makes it difficult for people to exploit a system, 6) noise introduce uncertainty that can be used in combination with biases to improve social outcomes, and 7) reducing noise might lead to less creativity. These are not original insights but there are some good discussions and it is great to see the book devote attention to the benefits of noise as well, especially to provide balance to the naive optimism provided throughout the book when it comes to noise reducing strategies such as algorithms.

Overall, if you care about noise (as you should), I will not recommend this book. Instead, if you do want a book with noise in the title, consider Nate Silver’s The Signal and the Noise (that also focuses on the work by Philip E. Tetlock, provides references to Moneyball, etc.) and Nassim Nicholas Taleb’s Fooled by Randomness. The latter provides a much better introduction to noise, uncertainty, and randomness. Recent books that deal with some of the same topics and ideas are Dark Data: Why What You Don’t Know Matters (I provided a few thoughts on the book here) and Radical Uncertainty: Decision-Making Beyond the Numbers.

Philip E. Tetlock describes Noise as “A masterful achievement and a landmark in the field of psychology” and Cass Sunstein said in an interview that: “Unlike bias, noise isn’t intuitive, which is why we think we’ve discovered a new continent”. This might very well be the case but why not acknowledge previous research on noise? If you are truly discovering a new continent, you need to provide a stronger case for this continent being new. For example, why not mention the work on noise traders in financial markets (inspired by Fischer Black, and also described in Thaler’s book Misbehaving). When not dealing with all of this research, Andrew Gelman is correct with his comment: “But then I realized that Sunstein kinda is like Columbus, in that he’s an ignorant guy who sails off to a faraway land, a country that’s already full of people, and then he goes back and declares he’s discovered the place.”

This is not to say that there isn’t a lot of new land to discover. There is a lot of potential in a book about noise. Noise is all around us and we are not good at dealing with noise. How, for example, do we use different strategies to deal with different types of noise? And what about when people do not agree with each other about what is noise and what is not? (We might call this “noise in noise”.) What is noise for some people can be a signal for other people. In opinion polls, there is a lot of noise that journalists use to create engaging narratives – but political scientists (at least those not yet turned pundits) see nothing but random error. (We used such noise to study how small differences within the margin of error affected the coverage of opinion polls.)

Noise is a book closer to being an appendix to Thinking, Fast and Slow and Nudge than an original piece of work that improves our understanding of human behaviour, and in particular the role of noise. Again, there is a lot of potential in providing a book-length examination of noise in human judgment and decision making. And while there are definitely good things to say about Noise, is a lost opportunity to provide such an examination. In sum, this is a fairly lengthy book with a low signal-to-noise ratio for people interested in the scientific study of noise/cognitive biases/measurement error/treatment heterogeneity/reliability.

There is a lot of noise in Noise.

Which party do you think is most likely to agree?

In a new poll, JL Partners surveyed more than 2,000 respondents to understand how the public perceives the Labour Party and the Conservative Party on a series of different “woke” topics. The Daily Mail uses the poll to conclude that “Sir Keir Starmer’s Labour Party is out of touch with public opinion”. Is that what the poll finds? No.

The question wording was “For each of the following issues, please indicate which of the main two parties is more likely to agree with it.” and the ideas include statements such as “Statues of historical figures being taken down” and “Shorter sentences for criminals”. The figure below shows the results together with information on the percentage of the public supporting these ideas.

The Campaign For Common Sense, which has nothing to do with common sense, concluded that “the Conservatives can confidently take a stand on these issues knowing that they are more in tune with the British public.” However, it is not possible to use the data in the survey to draw such a conclusion.

The problem is that we cannot use the question to understand 1) whether any of the parties actually support the ideas (both parties might reject them) or 2) whether both parties actually support the ideas (both parties might support them).

To understand why, consider if the idea had been “An income tax rate of 75%”. Most people would say that Labour would be most likely to agree with this idea, but that is not the same as Labour would support such an idea. And most voters would most likely say, if asked, that Labour is not currently supporting such an idea. For that reason, it is not possible to use the answers in the survey to say anything meaningful about whether any of the parties are more in tune with the British public in relation to the specific ideas.

If the Campaign For Common Sense was interested in showing what party was most in tune with the British public, they would need to ask questions about whether the public believes that the parties support certain ideas – not about which party is most likely to agree.

New article in Electoral Studies: Populist parties in European Parliament elections

In the June issue of Electoral Studies, you will find an article I’ve written together with Mattia Zulianello. In the article, we introduce a comparative dataset on left, right and valence populist parties in European Parliament elections from 1979 to 2019. Here is the abstract:

Despite the increasing interest in populism, there is a lack of comparative and longterm evidence on the electoral performance of populist parties. We address this gap by using a novel dataset covering 92 populist parties in the European Parliament elections from 1979 to 2019. Specifically, we provide aggregate data on the electoral performance of all populist parties as well as the three ideational varieties of populism, i.e. right-wing, left-wing and valence populist parties. We show that there is significant variation both across countries as well as between the ideational varieties of populism. Most notably, while the success of left-wing and valence populists is concentrated in specific areas, right-wing populist parties have consolidated as key players in the vast majority of EU countries.

You can find the article here. I also recommend that you check out this great Twitter thread. You can find the data on GitHub and the Harvard Dataverse.