I remember my office-mate going down a crazy rabbit hole trying to figure out exactly what the right standard error calculations should be for his job market paper. He probably spent a month obsessing over it and never did come to a definitive conclusion.

Probably no surprise, but each of the half dozen different approaches told pretty much the same story.

Undisciplined

Oh yeah I’m not really criticizing robustness checks.

If anything, the people who trust too much in the formal statistics deserve more criticism (imo). And I’m mainly referring to social science here as well.

SimpleStacker

science

When Data Is Missing, Scientists Guess. Then Guess Again

0xbitcoiner

In my advisor's defense, she doesn't care that much about econometric nit-picking. Her preference is very much to find highly defensible natural experiments and then do a simple regression analysis.

I actually hadn't even connected this practice to robustness checks in my mind, because it comes up towards the beginning of the process. It always just seemed like her attitude was "Why don't you see if it's a problem before you spend a bunch of time worrying about it?"