The mosaic test:benchmarking colour-based image retrieval systems using image mosaics

Plant, William, Lumsden, Jo and Nabney, Ian (2011). The mosaic test:benchmarking colour-based image retrieval systems using image mosaics. IN: Proceedings of 1st European Workshop on Human-Computer Interaction and Information Retrieval (EuroHCIR). CEUR workshop proceedings . CEUR-WS.org.

Abstract

Evaluation and benchmarking in content-based image retrieval has always been a somewhat neglected research area, making it difficult to judge the efficacy of many presented approaches. In this paper we investigate the issue of benchmarking for colour-based image retrieval systems, which enable users to retrieve images from a database based on lowlevel colour content alone. We argue that current image retrieval evaluation methods are not suited to benchmarking colour-based image retrieval systems, due in main to not allowing users to reflect upon the suitability of retrieved images within the context of a creative project and their reliance on highly subjective ground-truths. As a solution to these issues, the research presented here introduces the Mosaic Test for evaluating colour-based image retrieval systems, in which test-users are asked to create an image mosaic of a predetermined target image, using the colour-based image retrieval system that is being evaluated. We report on our findings from a user study which suggests that the Mosaic Test overcomes the major drawbacks associated with existing image retrieval evaluation methods, by enabling users to reflect upon image selections and automatically measuring image relevance in a way that correlates with the perception of many human assessors. We therefore propose that the Mosaic Test be adopted as a standardised benchmark for evaluating and comparing colour-based image retrieval systems.

Divisions: Engineering & Applied Sciences > Computer science
Engineering & Applied Sciences > Computer science research group
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Event Title: 1st European Workshop on Human-Computer Interaction and Information Retrieval
Event Type: Other
Event Dates: 2011-07-04
Uncontrolled Keywords: benchmarking,content-based image retrieval,image databases,image mosaic,performance evaluation,Computer Science(all)
Full Text Link: http://ceur-ws. ... -763/paper6.pdf
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
Published Date: 2011
Authors: Plant, William
Lumsden, Jo ( 0000-0002-8637-7647)
Nabney, Ian ( 0000-0003-1513-993X)

Download

[img]

Version: Published Version


Export / Share Citation


Statistics

Additional statistics for this record