Our preregistered direct replication study of Hare et al. (2009, Science) on the neural background of self-control was recently published! Did we replicate the original findings? Yes and No… – a summary of the project:
Background
Self-control is of vital importance to successful, healthy lives. What mechanisms let us prioritize long-term goals over short-term rewards? Hare et al. were among the first to provide empirical evidence on the neural correlates of self-control. The original study has been used extensively for hypothesis building and interpretation of findings in tons of studies. It is important that the findings these studies build on are replicated, thereby contributing to the development of neurobiologically underpinned models of human behavior. The aim of the project was to directly replicate the original study and to assess if the results of the original study are robust across different analytic strategies. We replicated experimental procedures and analysis pipelines of the original study as closely as possible. Similar as in the original study, participants rated the healthiness and tastiness of food items and decided about eating the foods while undergoing fMRI.
Findings
Our replication data provide support for the notion that decisions are associated with a value signal in ventromedial prefrontal cortex, which integrates relevant choice attributes to inform a final decision. Like the original study, we found that areas in the medial prefrontal cortex (MPFC) encoded "goal value" (people's decisions about eating each food item, Strong no - Strong yes). Activity extracted from MPFC nicely distinguishes between individual decision levels like Yes vs Strong Yes, but effect sizes strongly depend on analysis pipelines, e.g. whether you have a localizer task to identify key MPFC voxels for each individual.
What about self-control? Like Hare et al., we identified self-controllers/SC and non-self-controllers/NSC, based on their success at resisting unhealthy food they liked or accepting healthy food they disliked. The original and replication samples of SC and NSC participants behaved substantially similar. Like Hare et al., we found that SC, but not NSC participants show encoding of healthiness in MPFC. However, we do not find significant group differences when these groups are directly compared (not reported in the original study).
We did not find strong support for the hypothesized role of left dorsolateral prefrontal cortex (DLPFC) in self-control. One of the most important findings of the original study was the involvement of DLPFC in self-control (in terms of average activity and connectivity with MPFC): In the original study, higher DLPFC activity was found in SC compared to NSC participants during successful self-control. We only find brain areas with lower activation in this contrast, even at lenient statistical thresholds. Although whole-brain analysis did not show DLPFC involvement in self-control, we did replicate the original results when we extracted activity from DLPFC voxels that are meta-analytically associated with self-control (http://neurosynth.org). How come? We show that these replicated results are produced by analytical choices. Using the replicated analysis pipeline, we also produce "evidence" that white matter voxels are associated with self-control. Applying an approach more standard today, we do not find DLPFC involvement. We do find evidence of direct negative functional connectivity between DLPFC & MPFC, which was expected but not found in this form in the original study. However, we conclude that mixed findings warrant more investigation of DLPFC-MPFC connectivity during self-control.
Impact on theory
The study of Hare conceptualizes self-control as a value-based decision but in line with traditional dual-system models it still posits that there are dual motives and that the future part is “special”: integrating longer-term considerations into the value system, that is, changing the weight of long-term attributes, requires involvement from control-related areas (i.e., the DLPFC). The replication results are more in line with the proposition of self-control as a simple form of value-based decision-making (Berkman et al., 2017). Within this conceptualization of self-control, there is nothing special about long-term goals: attributes related to short- and long-term goals treated similar in this equation though the relative weights may be different.
Importance of replication studies
Direct replication is a corner stone of science; falsification is a fundamental principle in science. Although growing concern about the reproducibility of scientific findings (the “replication crisis”) has led to several replication projects in fields like social psychology and biomedical research, replication remains uncommon in neuroscience. This is partly due to the high monetary and time-investment costs of fMRI studies. I am are grateful for the opportunity to run this careful replication, funded by the replication program of the Dutch Science Foundation – NWO Science, together with a great team consisting of Christin Scholz, Hang-Yee Chan, Russell Poldrack, Ale Smidts and Denise de Ridder.
This project has been one of the most enjoyable, challenging, and educational ones of my career as a researcher. The original study was published in the year that I started my PhD and this study, as well as its legacy, has been an inspiration for my own research. I think replication studies should become more “mainstream” (Zwaan, 2018) – a failed replication does not need to mean the conclusion of the original study is incorrect. Often, multiple replications are needed to find out which findings are sufficiently solid to build on.
Een belangrijke laatste les die we hieruit kunnen trekken is dat je niet klaar bent als de app klaar is en in de store staat. Een succesvolle implementatie kan bereikt worden als er voortdurent geëvalueerd wordt op basis van de resultaten, de communicatie over de app en het ontwerp van de app aangepast worden.
Resources
The paper is published open access in Human Brain Mapping: Scholz, C., Chan, H-Y., Poldrack, R.A., De Ridder, D.T.D., Smidts, A., Van der Laan, L.N. (2022) Can we have a second serving? A preregistered direct replication study on the neurobiological mechanisms underlying self-control. Human Brain Mapping (epub ahead of print). https://onlinelibrary.wiley.com/doi/10.1002/hbm.26065
The pre-registration and the stimulus the stimulus set: https://osf.io/qzyxm
The dataset of the project can be downloaded here: https://openneuro.org/datasets/ds002643