Hey guys, got a question about the sampling. Suppose I want to use the home_panel data to make a year-to-year comparison. There’s 100 devices in a major metro neighborhood, COVID happens, there’s migration, and then there’s 50 devices at the same time next year. Does the change in counts signal a change in population, or a change in SafeGraph’s sampling? How could one tell the difference?
This topic was automatically generated from Slack. You can find the original thread here.
Good question. It could be both, so it’s a bit tricky to disaggregate the effect of each.
One thing you could do is compare the proportion of devices in the neighborhood to the MSA or county or state. At least then, you could see whether devices dropped across the board (probably more indicative of sampling) or whether devices dropped at a faster rate for this neighborhood (probably indicative of at least some change in population).
Gotcha, so I need to compare the ratios at all levels for me to make a sufficient comparison between the two?
So if there’s a downward trend in the state level counts, I’d need to disentangle changes at the county or CBG level?
Not at all levels per se, but those are some of the larger levels I’d consider to feel confident
Probably county makes the most sense, if you want to pick one.
Copy that, what about at the census group level?
Could we estimate a change in population using that?
I might start with something like county first since that’s large enough that our panel samples that well. I am not sure about anything in between CBG and county, as per this analysis
My guess is that if it’s relatively easy for you to compare to multiple geographic levels (census tract, county, state), and you notice a change in devices relative to the higher levels each time, then I’d feel more confident saying that there’s a population change
Understood, thanks Jeff! You guys answer so quickly, it’s amazing
Thanks @Jeff_Ho_SafeGraph and @Alexander_Audet_American_Enterprise_Institute ! To prevent any further questions from being overlooked, I’ll go ahead and close this thread out. If you have any more questions or follow-up questions, we’re always here to help! Just be sure to make a new post to help, as we aren’t monitoring old threads at this time. Thanks!