Why I don’t wish that I had never learned about the replication crisis

Doing good science is really hard. It takes a lot of effort and time, and the latter is critical for early-career researchers who have a very limited amount of time to show their productivity to their next potential employer. Doing bad science is easier: It takes less time to plan an experiment, one needs less data, and if one doesn’t get good results immediately (one hardly ever does), the amount of time needed to make the results look good is still less than the amount of time needed to plan and run a well-designed and well-powered study.

Doing good science is frustrating at times. It can make you wonder if it’s even worth it. Wouldn’t life be easier if we were able to continue running our underpowered studies and publish them in masses, without having a bad conscience? I often wonder about this. But the grass always looks greener from the other side, so it’s worth taking a critical look at the BC (before-crisis) times before deciding whether the good old days really were that good.

I learned about the replication crisis gradually, starting in the second year of my PhD, and came to realise its relevance for my own work towards the end of my PhD. During my PhD, I conducted a number of psycholinguistic experiments. I knew about statistical power, in theory – it was that thingy that we were told we should calculate before we start an experiment, but nobody really does that, anyway. I collected as many participants as was the norm in my field. And then the fun started: I remember sleepless nights, followed by a 5-am trip to the lab because I’d thought of yet another analysis that I could try. Frustration when also that analysis didn’t work. Doubts about the data, the experiment, my own competence, and why was I even doing a PhD? Why was I unable to find a simple effect that others had found and published? Once, I was very close to calling the university which gave me my Bachelor of Science degree, and asking them to take it back: What kind of scientist can’t even replicate a stupid effect?

No, I don’t wish that I had never learned about the replication crisis. Doing good science is frustrating at times, but much more satisfying. I know where to start, even if it takes time to get to the stage when I have something to show. I can stand up for what I think is right, and sometimes, I even feel that I can make a difference in improving the system.