Test-retest reliability for common tasks in vision science

Abstract

Research in perception and attention has typically sought to evaluate cognitive mechanisms according to the average response to a manipulation. Recently, there has been a shift toward appreciating the value of individual differences and the insight gained by exploring the impacts of between-participant variation on human cognition. However, a recent study suggests that many robust, well-established cognitive control tasks suffer from surprisingly low levels of test-retest reliability (Hedge et al., 2018b). We tested a large sample of undergraduate students (n = 160) in two sessions (separated by 1–3 weeks) on four commonly used tasks in vision science. We implemented measures that spanned a range of perceptual and attentional processes, including motion coherence (MoCo), useful field of view (UFOV), multiple-object tracking (MOT), and visual working memory (VWM). Intraclass correlations ranged from good to poor suggesting that some task measures are more suitable for assessing individual differences than others. VWM capacity (ICC = 0.77), MoCo threshold (ICC = 0.60), UFOV middle accuracy (ICC = 0.60) and UFOV outer accuracy (ICC = 0.74) showed good-to-excellent reliability. Other measures, namely the maximum number of items tracked in MOT (ICC = 0.41) and UFOV number accuracy (ICC = 0.48), showed moderate reliability; the MOT threshold (ICC = 0.36) and UFOV inner accuracy (ICC = 0.30) showed poor reliability. In this paper, we present these results alongside a summary of reliabilities estimated previously for other vision science tasks. We then offer useful recommendations for evaluating test-retest reliability when considering a task for use in evaluating individual differences.

Publication DOI: https://doi.org/10.31234/osf.io/7fm2z
Divisions: College of Health & Life Sciences > Applied Health Research Group
College of Health & Life Sciences > Aston Institute of Health & Neurodevelopment (AIHN)
College of Health & Life Sciences
College of Health & Life Sciences > School of Psychology
Additional Information: Copyright 2022 The Authors. This work is licensed under a Creative Commons Attribution 4.0 International License.
Uncontrolled Keywords: Sensory Systems,Ophthalmology
Publication ISSN: 1534-7362
Full Text Link:
Related URLs: https://jov.arv ... ticleid=2783521 (Publisher URL)
https://osf.io/gtusw/ (Related URL)
PURE Output Type: Article
Published Date: 2022-07-29
Accepted Date: 2022-05-24
Authors: Clark, Kait
Birch-Hurst, Kayley
Pennington, Charlotte R. (ORCID Profile 0000-0002-5259-642X)
Petrie, Austin C. P.
Lee, Joshua T.
Hedge, Craig (ORCID Profile 0000-0001-6145-3319)

Download

[img]

Version: Draft Version

Access Restriction: Restricted to Repository staff only

License: Creative Commons Attribution


[img]

Version: Published Version

License: Creative Commons Attribution

| Preview

Export / Share Citation


Statistics

Additional statistics for this record