First one started, last one finished

This news is a few weeks old, but I managed to bag another fun (at least to me) publication. It started as a footnote in my MS thesis in the Fall 2014 semester at Purdue University, with my eventual PhD advisor, Jim Binkley. Finding this paper a home took us a while, but now it can rest easy. Its title: "The Chow Test with Time Series-Cross Section Data". It was the first journal article I ever helped write, let alone submit to a journal. While there's hopefully more to come, it's my most recent.

If you have a regression model using nested data over groups and subgroups (e.g. industries & firms, counties & states) you may want to test for the presence of group effects. Often this is done with a Chow Test to examine differences in slopes for the groups' regression lines. However, if each datapoint has multiple observations (e.g. each county in the states in your sample has 5 years' worth of data), then this method may inadvertently detect subgroup differences and think they're group differences. The more observations per individual unit, and the stronger the individual or subgroup differences, the higher the rate of false statistically significant outcomes.

What is the remedy? We figured the maintained hypothesis of the Chow Test is that the individual effects are, on average, zero and therefore it's okay to use the F-distribution table in the back of a statistics textbook. But if those effects don't actually "wash out", then you need to use another distribution. Which one? Make one up! Well, sort of: simulate an F-distribution that includes these  problematic effects giving us misleading results. If they're not really a problem, then you'll get something close to the back-of-the-textbook distribution. But if they are, then you need to look at the percentiles (critical values) of the pseudo-F distribution to decide whether or not to reject your null hypothesis.

How did we come across this? Like I said, it was during my MS thesis writing. About a week before my defense, we got some results that were by all accounts, wrong. They disagreed with the theory, they contradicted the model estimates, and they were totally counterintuitive. What happened? We were using the Chow Test, and our first reaction was, "well, congrats, we just broke the Chow Test. Also, you defend next week." We tabled it until after the defense, but over Winter Break, Jim discovered what was really going on. It appeared what we mistook for retailer differences in milk pricing behavior was, in fact, individual grocery store price heterogeneity, reinforced by each store having 70+ months' worth of data. The solution was to make a new test/table of critical F-values, so we made one and it worked. Then we made up some data and some models, found the Chow Test would overstate the significance of the simulated results, but our procedure worked again. And it only took us 8 years to publish all of this!

If you're so inclined, take a look at the paper. There is some math (sorry) but we do try to state in plain English what is actually going on. We even have the grocery example included.

Popular posts from this blog

Specialty Crop Hobby Update: Pawpaws

Will Farmer's Markets Benefit from the Food Tariffs? ---PART 1---

Will Farmer's Markets Benefit from the Food Tariffs? ---PART 2---