I have a new short paper titled Problems with the Big Five assessment in the World Values Survey in Personality and Individual Differences (co-authored with Steven Ludeke). In the paper, we examine basic psychometric properties of the Big Five personality traits included in Wave 6 of the World Values Survey. The abstract:

Publicly-available data from the World Values Survey (WVS) is an extremely valuable resource for social scientists, serving as the basis for thousands of research publications. The most recent assessment (Wave 6) was the first to assess Big Five personality traits, and this data has already been used in published research. In the present paper, we show for the first time that the Big Five data from WVS Wave 6 is extremely problematic: items from the same trait correlate negatively with each other as often as not, occasionally to truly extreme degrees. Particular caution is warranted for any future research aiming to use this data, as we do not identify any straightforward solution to the data’s challenges.

In Figure 2 in the paper, also presented below, we show the distribution of item-item correlations for the Big Five personality traits in all countries. Ideally, the item-item correlations would be positive and strong. However, in most cases, the correlations are weak and/or going in the wrong direction.

There are multiple potential problems with the data and, alas, we are unable to identify a single issue explaining why there is a problem and thus provide guidelines on how to correct it. Noteworthy, some blog posts examine the data in further detail and discuss issues such as coding errors, translation errors and acquiescence bias, e.g. Rene Bekkers blog post, Hunting Game: Targeting the Big Five, and Florian Brühlmanns blog post, Can we trust Big Five data from the WVS?.

Reproducibility material for the paper can be retrieved at GitHub and the Harvard Dataverse.