One of the most popular studies on the Social Science Research Network (SSRN) is a study published in Economic Inquiry in 2015. The abstract is viewed more than half a million times and the study is downloaded ~80,000 times. These are impressive numbers. The study is in top 10 of the all time total number of downloads on the SSRN.
Why do people share and like this study? Because there is a certain irony to the main finding, namely that the more money people spend on an engagement ring, the less likely it is that their marriage will last. Unsurprisingly, this study was also picked up by the popular press (e.g. here, here and here).
The title of the study is ‘“A diamond is forever” and other fairy tales: The relationship between wedding expenses and marriage duration‘. Here is the full abstract of the study:
In this study, we evaluate the association between wedding spending and marriage duration using data from a survey of more than 3,000 ever-married persons in the United States. Controlling for a number of demographic and relationship characteristics, we find evidence that marriage duration is inversely associated with spending on the engagement ring and wedding ceremony.
With the impressive number of downloads of the study, I am surprised that I haven’t seen more critical reflections on the study. I decided to take a closer look at the study and I am now even more surprised by all the attention that the study has received. Let’s try to understand what the study is about.
The basic idea of the study is to compare whether people with a more expensive engagement ring are more likely to get divorced (the outcome of the study is marital dissolution). I applaud the authors for taking on such a difficult task as this is one of the question we cannot study in a randomised experiment. Specifically, we cannot sample a lot of people that are about to get married and assign half of the sample to a cheap engagement ring and the other half to an expensive engagement ring.
More importantly, when we study a question like these, we need to be very much concerned about whether people who are divorced and have an expensive engagement ring are more likely to take part in a survey for money than people who are still married with an expensive engagement ring.
The sample composition is from what I can see one of the primary limitation with the study. Specifically, the study is using a sample of people from mTurk which is not a nationally representative sample. On the contrary, the study is relying on a lot of participants that were offered $0.50–$0.75 to answer approximately 40 questions. This sample is not sufficient to say anything meaningful about the impact of an expensive engagement ring on marriage duration. Importantly, the authors are aware of this limitation on page 1920 in the study: “Of course, for our purposes, a large national probability sample would be preferable.” Of course.
There is a much more severe problem with the study that, even if we assume that the design is strong, there is no reason to believe that people with a more expensive engagement ring are more likely to get divorced.
In brief, the authors conduct a survival analysis (a Cox proportional hazard model). Take a look at this description from the paper (page 1922) and see if you can get a sense of what some of the issues are:
In bivariate regressions, having no engagement ring and having wedding expenses below $1,000 are each associated with increases in the hazard of divorce, while spending $8,000 or more on an engagement ring is associated with a decrease in the hazard of divorce. This appears to be consistent with the relationship between wedding expenses and marriage duration posited by wedding industry advertising. However, the picture changes in multivariate regressions. Spending between $2,000 and $4,000 on an engagement ring is significantly associated with an increase in the hazard of divorce among the sample of men. Specifically, in the sample of men, spending between $2,000 and $4,000 on an engagement ring is associated with a 1.3 times greater hazard of divorce as compared with spending between $500 and $2,000.
There is a lot of stuff going on in the text above. The first problem is that bivariate models actually suggest the opposite of what the authors argue in the paper. The consistent finding in a bivariate model is that people engaged with no engagement ring are significantly more likely to be divorced. And people with an expensive engagement ring are more likely to stay married.
So what about the multiple regressions? There are two issues here. First, and least problematic, we are dealing with a strong portion of regression soup. Here is a list of the covariates in the model: age, marriage age, gender, race/ethnicity, education, employment, household income, region of residence, religious attendance, respondent-spouse differences (age difference, race difference, education difference), children with spouse, knew spouse very well, length of time dated before proposal, feelings and attitudes at time of proposal, had a honeymoon, wedding attendance, and total wedding expenses. One problem here is that some of these variables can induce post-treatment bias and I see no reason to be much more confident in the impact of an expensive engagement ring in the multiple regressions compared to the bivariate regression.
Second, even if we assume that nothing of the above is problematic, there is no pattern in the multiple regressions suggesting that people with a more expensive engagement ring are more likely to get a divorce. Take a look at the regression coefficients for engagement ring expenses in Table 2 (to keep it simple, I do not show all the other variables in the models):
It is a little difficult to understand the coefficients as the reference category is “$500 to $2,000”. In the bivariate model we can see, as stated above, that the more expensive the proposer’s engagement ring, the more likely that the respondent is still married. If we look at the multiple models for all respondents, we see no significant effects. In other words, no indication at all that “marriage duration is inversely associated with spending on the engagement ring” (as pointed out in the abstract).
Let’s take another look at what the authors say they find with the table above in mind: “in the sample of men, spending between $2,000 and $4,000 on an engagement ring is associated with a 1.3 times greater hazard of divorce as compared with spending between $500 and $2,000”. In the sample of men. Of course! Between $2,000 and $4,000. Of course! There is one statistically significant regression coefficient and nine statistically non-significant (not counting the reference category and the coefficients for “Don’t know”). Again, if anything, the models suggest that having spend more money on an engagement ring correlates with a longer marriage – not shorter.
Marriage duration is not inversely associated with spending on the engagement ring. This is just yet another social science fairy tale.