Open Science: Best Practices in Empirical Research

The goal of the module is to sensitize the students to the possibly sensitive issues during the research process that have contributed to the current replication crisis in empirical research. Specifically, it will equip the students with knowledge of the questionable research practices and other problems of the publication system that have fostered to the replication crisis (e.g., p-hacking, HARKing, underpowered studies, publication bias). Even more importantly, the students will gain practical skills—including knowledge and first-hand experience in setting up a preregistered study and implementing practices of Open Science (e.g., open data, open analysis code)—and become familiar with approaches in data analysis (e.g., Bayesian statistics) that promise greater robustness in statistical inference.

A replication crisis has shaken the empirical sciences, revealing that in several disciplines (e.g., experimental economics, psychology) the results of key findings are difficult to reproduce. This replication crisis raises important questions about current research and publication practices. In this module, we trace the history of the replication crisis and discuss possible causes of the apparently limited replicability of past research. Further, we will get to know various methodological reforms and developments toward more reliable, robust, and transparent empirical research (e.g., preregistration, Open Science practices, replication research, Bayesian data analysis).

In short presentations, students will present empirical investigations and analyses that have shaped the recent discussion on the replicability of behavioral research. In group discussions, the students will analyze seminal empirical articles and discuss methods for improving the robustness, replicability, and transparency of empirical research. In small-group exercises, students will get hands-on experience with drafting a preregistration document and preparing a repository for making data and analysis code publicly available.