Here is a strange study: When You Can’t Tube… Impact of a Major YouTube Outage on Rapes. And here is the abstract for the study:
On Tuesday, October 16, 2018, YouTube experienced a major and rare global service outage. Using high-frequency crime data from the United States, we document an important increase in rapes in the 24-hour period following the outage. We investigate various potential underlying channels that may link the YouTube outage to the subsequent observed increase in rapes. The overall evidence only supports the hypothesis that the increase in rapes was driven by an increase in pornography viewing.
I find the actual mechanism improbable, i.e. that people – when YouTube is down – will be more likely to engage in certain criminal activities. For that reason, I decided to actually read the paper.
The first reason for calling the study strange is that I do not remember any stories about ‘a major and rare service outage’. Of course, this is most likely just an event that I did not hear about and that happened at a time when I did not use YouTube. It happens now and then that major services are down, including YouTube (together with other Google services) back in December.
Interestingly, when I looked at the paper I could not find any explicit information about how long the actual outage being studied was. The only information on the length of outages is in footnote 8: “Other minor outages were on June 16, 2017 at 10 a.m. for 2 hours; November 12, 2018 at 5 p.m. for an hour; November 18, 2018 at 7 p.m. for half an hour. Note that these outages did not occur at night.”. I guess a few hours of downtime is a minor outage? Well, apparently not.
When I looked into what media outlets reported in relation to the ‘major and rare global service outage’, I was surprised. For example, as reported by Business Insider: “YouTube abruptly went down for around an hour on Tuesday evening”. Around an hour? Well, no surprise I don’t remember it. And no surprise the length of the outage is not mentioned in the paper. I guess the key point is that this outage occurred at night.
There is, however, information that the outage occurred at night between 9 and 11pm: “On Tuesday, October 16, 2018, between 9 and 11p.m. Eastern time, YouTube experienced a major and rare global service outage.” I guess this was the length of the outage, i.e. from 9pm to 11pm. But why then describe another outage of 2 hours as a minor outage!? And why focus on this specific outage?
This is a red flag to me, i.e. when researchers have several events they can explore. And then add other outcomes and time to the mix, or as they write in the paper: “We find that other crimes and offenses (including drug, alcohol, and traffic) were not affected by the outage. We also report that the observed increase in rapes did not occur in the 2-hour period during the outage, but in the 22-hour period after YouTube service had been restored.” This is a major red flag.
What if the researchers found that drug related crimes increased within the first six hours of the outage? Would that be in line with the theory as well? To be fair, the researchers do develop a theoretical framework with the mandatory reference to Becker 1968, but did they actually develop this framework prior to looking at the data? Maybe, but I have been around the block long enough to be suspicious – especially when you save the theoretical framework to the appendix and use individual-level descriptions such as “the agent rapes if the utility from raping is greater than zero” to explain a coefficient estimated with noisy aggregate-level data.
Wait, why am I not impressed by the results? Take a look at Figure 4 in the paper with the placebo tests:
For these placebo tests to be convincing, we would like to see that the coefficients were randomly distributed around zero and the coefficient for day with the outage (0 on the x-axis) would stand out. This is not the case here. On the contrary, we see effects similar to the main result on other so-called placebo days. This is not convincing and there is, in my view, no systematic evidence showing that the YouTube outage caused an increase in the number of rapes.
In sum, the theory and empirics are too weak to convincingly demonstrate that we should expect (or even predict) an increase in the number of rapes when YouTube is down.