Jones, Andrew, Earnest, Jessica, Adam, Martyna, Clarke, Ross, Yates, Jack and Pennington, Charlotte Rebecca (2022). Careless responding in crowdsourced alcohol research: a systematic review and meta-analysis of practices and prevalence. Experimental and Clinical Psychopharmacology, 30 (4), pp. 381-399.
Abstract
Crowdsourcing—the process of using the internet to outsource research participation to “workers”—has considerable benefits, enabling research to be conducted quickly, efficiently, and responsively, diversifying participant recruitment, and allowing access to hard-to-reach samples. One of the biggest threats to this method of online data collection however is the prevalence of careless responders who can significantly affect data quality. The aims of this preregistered systematic review and meta-analysis were: (a) to examine the prevalence of screening for careless responding in crowdsourced alcohol-related studies; (b) to examine the pooled prevalence of careless responding; and (c) to identify any potential moderators of careless responding across studies. Our review identified 96 eligible studies (˜126,130 participants), of which 51 utilized at least one measure of careless responding, 53.2%, 95% CI [42.7%–63.3%]; ˜75,334 participants. Of these, 48 reported the number of participants identified by careless responding method(s) and the pooled prevalence rate was ˜11.7%, 95% CI [7.6%–16.5%]. Studies using the MTurk platform identified more careless responders compared to other platforms, and the number of careless response items was positively associated with prevalence rates. The most common measure of careless responding was an attention check question, followed by implausible response times. We suggest that researchers plan for such attrition when crowdsourcing participants and provide practical recommendations for handling and reporting careless responding in alcohol research.
Publication DOI: | https://doi.org/10.1037/pha0000546 |
---|---|
Divisions: | College of Health & Life Sciences College of Health & Life Sciences > School of Psychology College of Health & Life Sciences > Aston Institute of Health & Neurodevelopment (AIHN) Aston University (General) |
Additional Information: | © 2021 APA. Jones, A., Earnest, J., Adam, M., Clarke, R., Yates, J., & Pennington, C. R. (2022). Careless responding in crowdsourced alcohol research: A systematic review and meta-analysis of practices and prevalence. Experimental and Clinical Psychopharmacology. Advance online publication. https://doi.org/10.1037/pha0000546 |
Uncontrolled Keywords: | Alcohol use research,Careless responding,Crowdsourcing,Insufficient effort responding,Meta-analysis,Pharmacology,Psychiatry and Mental health,Pharmacology (medical) |
Publication ISSN: | 1936-2293 |
Last Modified: | 03 Dec 2024 08:18 |
Date Deposited: | 15 Dec 2021 15:52 |
Full Text Link: |
https://psyarxiv.com/rs9xh/ |
Related URLs: |
https://osf.io/p837n/
(Related URL) https://doi.apa ... 37%2Fpha0000546 (Publisher URL) http://www.scop ... tnerID=8YFLogxK (Scopus URL) |
PURE Output Type: | Article |
Published Date: | 2022 |
Published Online Date: | 2022-02-07 |
Accepted Date: | 2021-12-13 |
Authors: |
Jones, Andrew
Earnest, Jessica Adam, Martyna Clarke, Ross Yates, Jack Pennington, Charlotte Rebecca ( 0000-0002-5259-642X) |