Not All Effects Are Indispensable: Psychological Science Requires Verifiable Lines of Reasoning for Whether an Effect Matters

Abstract

To help move researchers away from heuristically dismissing “small” effects as unimportant, recent articles have revisited arguments to defend why seemingly small effect sizes in psychological science matter. One argument is based on the idea that an observed effect size may increase in impact when generalized to a new context because of processes of accumulation over time or application to large populations. However, the field is now in danger of heuristically accepting all effects as potentially important. We aim to encourage researchers to think thoroughly about the various mechanisms that may both amplify and counteract the importance of an observed effect size. Researchers should draw on the multiple amplifying and counteracting mechanisms that are likely to simultaneously apply to the effect when that effect is being generalized to a new and likely more dynamic context. In this way, researchers should aim to transparently provide verifiable lines of reasoning to justify their claims about an effect’s importance or unimportance. This transparency can help move psychological science toward a more rigorous assessment of when psychological findings matter for the contexts that researchers want to generalize to.

Publication DOI: https://doi.org/10.1177/17456916221091565
Divisions: College of Health & Life Sciences > Applied Health Research Group
College of Health & Life Sciences > Aston Institute of Health & Neurodevelopment (AIHN)
College of Health & Life Sciences
College of Health & Life Sciences > School of Psychology
Additional Information: (c) Sage, 2022. This manuscript has been accepted for publication in Perspectives on Psychological Science. The final publication is available via Sage at [https://doi.org/10.1177/17456916221091565]
Uncontrolled Keywords: effect size,practical significance,benchmarks,evaluation
Publication ISSN: 1745-6916
Full Text Link:
Related URLs: https://journal ... 456916221091565 (Publisher URL)
PURE Output Type: Article
Published Date: 2022-08-22
Published Online Date: 2022-08-22
Accepted Date: 2022-03-01
Authors: Anvari, Farid
Kievit, Rogier
Lakens, Daniël
Pennington, Charlotte R. (ORCID Profile 0000-0002-5259-642X)
Przybylski, Andrew K.
Tiokhin, Leo
Wiernik, Brenton M.
Orben, Amy

Export / Share Citation


Statistics

Additional statistics for this record