The Impact of Statewide Stay-at-Home Orders: Estimating the Heterogeneous Effects Using GPS Data from Mobile Devices

Show the Community
,
#1

Hi all: Glad to share with you and welcome to comment on two fresh reports by the Center for Research on the Wisconsin Economy (CROWE), where @Noah_Williams_UW_Madison is the director and I am an economist. Both use the SafeGraph data.

In the first report, we attempt to answer a question asked by many, including @Ryan_Fox_Squire_SafeGraph: Are stay-at-home orders effective? Different from many like @Marc_Painter_SLU who pooled all orders together for an average effect, we estimate the impact of individual orders using the synthetic control method. The key finding is that not all orders are equally effective. For example, we find the order in Michigan increased the fraction of devices at home all day significantly by about 5.5 percentage points, while the estimate for Ohio is small and insignificant. This is striking because the orders in the two neighboring states were issued in a short span of less than 24 hours and went into effect on the same day. However, it’s consistent with other evidence from UI claims.
https://crowe.wisc.edu/wp-content/uploads/sites/313/2020/04/StayAtHome.pdf

In the second report, we use the estimate on Wisconsin from the first one as well as other estimates @Noah_Williams_UW_Madison has been working on, many of which are based on data from this consortium, to estimate the economic cost of the Safer at Home order in Wisconsin. Combining the economic factors with health factors, we provide indices that policymakers can use for a phased-in regional relaxation of social distancing guidelines.

#2

This is very impressive @Junjie_Guo_UW_Madison. Can you elaborate on how the synthetic control method works? Does it produce different results than using a regression approach like @Marc_Painter_SLU if every state had its own model?

#3

@Ryan_Fox_Squire_SafeGraph Synthetic control methods (SCM) take heterogeneity across units very seriously. For example, if you want to estimate the impact of Wisconsin’s Safer at Home order, instead of assuming that, conditional on observables, states without a stay-at-home order are comparable with Wisconsin both individually and thus on average, like regressions do. SCM would say no, that may not be case: Iowa and Wyoming may not be equally comparable with Wisconsin in the absence of the order. So instead of comparing Wisconsin with the simple average of controls like regressions do, SCM would weight the controls properly so that the weighted average (the synthetic control) mimics the behavior of Wisconsin before the order, and then use the differences after the order to estimate the impact. One advantage of SCM is transparency, because you could see visually and statistically whether the synthetic control is doing a good job mimicking the behavior of Wisconsin before the order. It also allows for the possibility that you may not find a good synthetic control, which could happen if the treated unit is in some ways an outlier (like California and New York in this pandemic), which is a good thing because we should caution against any estimate involving an outlier. Because regressions allow for extrapolation, you would always get an estimate there, even if it’s a bad one. SCM does have its disadvantages, for example, in terms of inference. Because inherently you are only working with one treated unit, traditional methods based on standard errors wouldn’t work. The following review paper is an excellent resource for any interested in SCM. https://economics.mit.edu/files/17847

#4

Thank you for taking the time to explain this, as well as sharing the source for the review paper (which I have skimmed at the moment). I now have a better understanding of your paper about reopening as well.