Here is a collection of statistical issues and misunderstandings you often will encounter in empirical research. My plan is to add more examples in the future.

Issue |
Description |
Source |

Absence of evidence fallacy | No evidence for a finding should not be interpreted as there is evidence of its absence. | Altman and Bland (1995) |

Berkson’s paradox | When conditioning on a variable creates a spurious correlation (i.e., collider bias, the conditioning on a collider). | Berkson (1946) |

Cronbach’s alpha | People often misunderstand the coefficient. There is not a particular level of alpha that is desired or adequate. | Hoekstra et al. (2018) |

Garbage can regression | Adding too many independent variables to your regression model (i.e. a kitchen-sink approach). | Achen (2004) |

Garden of forking paths | When researchers conduct multiple analyses but only end up reporting a subset of these (data-dependent analysis). | Gelman and Loken (2014) |

Moderation vs. Mediation | A moderator is a variable that affects the direction and/or strength of the relation between two variables – not the same as mediation. | Baron and Kenny (1986) |

Multivariate vs. Multivariable | A multivariate model is a model with multiple dependent variables. |
Mustillo et al. (2018) |

p-value as a probability | The p-value is not the probability that the null hypothesis is true. |
Greenland et al. (2016) |

Prosecutor’s fallacy | Incorrectly assuming that Pr(A|B) = Pr(B|A). | Westreich et al. (2014) |

Simpson’s paradox | A trend in the data can disappear or reverse when looking at subgroups in the data. | Simpson (1951) |

Spurious correlation | When two variables correlate but are not casually related. | Simon (1954) |

Statistical power | The importance of having sufficient data to estimate the effect size of interest. | Cohen (1992) |

### Recommended readings

Kennedy, P. E. 2002. Sinning in the Basement: What are the Rules? The Ten Commandments of Applied Econometrics. *Journal of Economic Surveys* 16(4): 569-589.

Makin, T. R., and J. O. de Xivry. 2019. Science Forum: Ten common statistical mistakes to watch out for when writing or reviewing a manuscript. *eLife* 8:e48175.

Motulsky, H. J. 2014. Common Misconceptions about Data Analysis and Statistics. *Journal of Pharmacology and Experimental Therapeutics* 351(1): 200-205.

Schrodt, P. A. 2014. Seven deadly sins of contemporary quantitative political analysis. *Journal of Peace Research* 51(2): 287-300.