Problems with the Big Five assessment in the World Values Survey

I have a new short paper titled Problems with the Big Five assessment in the World Values Survey in Personality and Individual Differences (co-authored with Steven Ludeke). In the paper, we examine basic psychometric properties of the Big Five personality traits included in Wave 6 of the World Values Survey. The abstract:

Publicly-available data from the World Values Survey (WVS) is an extremely valuable resource for social scientists, serving as the basis for thousands of research publications. The most recent assessment (Wave 6) was the first to assess Big Five personality traits, and this data has already been used in published research. In the present paper, we show for the first time that the Big Five data from WVS Wave 6 is extremely problematic: items from the same trait correlate negatively with each other as often as not, occasionally to truly extreme degrees. Particular caution is warranted for any future research aiming to use this data, as we do not identify any straightforward solution to the data’s challenges.

In Figure 2 in the paper, also presented below, we show the distribution of item-item correlations for the Big Five personality traits in all countries. Ideally, the item-item correlations would be positive and strong. However, in most cases, the correlations are weak and/or going in the wrong direction.

There are multiple potential problems with the data and, alas, we are unable to identify a single issue explaining why there is a problem and thus provide guidelines on how to correct it. Noteworthy, some blog posts examine the data in further detail and discuss issues such as coding errors, translation errors and acquiescence bias, e.g. Rene Bekkers blog post, Hunting Game: Targeting the Big Five, and Florian Brühlmanns blog post, Can we trust Big Five data from the WVS?.

Reproducibility material for the paper can be retrieved at GitHub and the Harvard Dataverse.

Potpourri: Statistik #37

UK government using R to modernize reporting of official statistics
Seeing Theory: A visual introduction to probability and statistics
Excel is threatening the quality of research data — Data Packages are here to help
Strange R Things
Social Media and Big Data Research
Announcing ggraph: A grammar of graphics for relational data
rtweet: Collecting Twitter Data
A glance at R-bloggers Twitter feed
adventures in garbage millennial confirmation bias

Selektion på den afhængige variabel: Parforhold

TV 2 skriver, at halvdelen af alle ægteskaber ender i opløsning, men at det ikke afspejler sig i danskernes vurderinger af deres parforhold. Dette fremstilles som noget tilnærmelsesvist paradoksalt, hvorfor der konkluderes: “Derfor er det også overraskende for eksperter, at danskerne er tilfredse i deres parforhold.”

Det interessante er selvfølgelig, at det på ingen måde er overraskende, at danskerne er tilfredse i deres forhold. Hvorfor? Fordi de personer, der ikke er tilfredse i deres forhold, i de fleste tilfælde vælger ikke at være i et forhold længere.

Artiklen hos TV 2 bygger på en meningsmåling som Megafon har foretaget for TV 2, hvor 90 procent af de adspurgte svarer, at de føler sig glade og tilfredse i deres parforhold. Den interessante historie burde om nogen være, at 10 procent af de adspurgte er i et forhold, de ikke er glade for eller tilfredse med.