Principles and Guidelines for Evaluating Social Robot Navigation Algorithms

Abstract

A major challenge to deploying robots widely is navigation in human-populated environments, commonly referred to as social robot navigation. While the field of social navigation has advanced tremendously in recent years, the fair evaluation of algorithms that tackle social navigation remains hard because it involves not just robotic agents moving in static environments but also dynamic human agents and their perceptions of the appropriateness of robot behavior. In contrast, clear, repeatable, and accessible benchmarks have accelerated progress in fields like computer vision, natural language processing and traditional robot navigation by enabling researchers to fairly compare algorithms, revealing limitations of existing solutions and illuminating promising new directions. We believe the same approach can benefit social navigation. In this paper, we pave the road towards common, widely accessible, and repeatable benchmarking criteria to evaluate social robot navigation. Our contributions include (a) a definition of a socially navigating robot as one that respects the principles of safety, comfort, legibility, politeness, social competency, agent understanding, proactivity, and responsiveness to context, (b) guidelines for the use of metrics, development of scenarios, benchmarks, datasets, and simulators to evaluate social navigation, and (c) a design of a social navigation metrics framework to make it easier to compare results from different simulators, robots and datasets.

Publication DOI: https://doi.org/10.1145/3700599
Divisions: College of Engineering & Physical Sciences > School of Computer Science and Digital Technologies > Applied AI & Robotics
College of Engineering & Physical Sciences > School of Computer Science and Digital Technologies
College of Engineering & Physical Sciences
Aston University (General)
Additional Information: Copyright © 2025 held by the owner/author(s). This work is licensed under a Creative Commons Attribution International 4.0 License.
Uncontrolled Keywords: social navigation,Artificial Intelligence,Human-Computer Interaction
Publication ISSN: 2573-9522
Last Modified: 06 Mar 2025 08:11
Date Deposited: 17 Jan 2025 12:32
Full Text Link:
Related URLs: https://dl.acm. ... 10.1145/3700599 (Publisher URL)
PURE Output Type: Article
Published Date: 2025-02-20
Published Online Date: 2024-12-27
Accepted Date: 2024-06-20
Submitted Date: 2023-11-27
Authors: Francis, Anthony
Perez D'arpino, Claudia
Li, Chengshu
Xia, Fei
Alahi, Alexandre
Alami, Rachid
Bera, Aniket
Biswas, Abhijat
Biswas, Joydeep
Chandra, Rohan
Chiang, Hao-Tien Lewis
Everett, Michael
Ha, Sehoon
Hart, Justin
How, Jonathan P.
Karnan, Haresh
Lee, Tsang-Wei Edward
Manso, Luis J. (ORCID Profile 0000-0003-2616-1120)
Mirsky, Reuth
Pirk, Soren
Singamaneni, Phani Teja
Stone, Peter
Taylor, Ada V.
Trautman, Peter
Tsoi, Nathan
Vazquez, Marynel
Xiao, Xuesu
Xu, Peng
Yokoyama, Naoki
Toshev, Alexander
Martin-Martin, Roberto

Download

[img]

Version: Accepted Version

Access Restriction: Restricted to Repository staff only

License: ["licenses_description_unspecified" not defined]


[img]

Version: Published Version

License: Creative Commons Attribution


Export / Share Citation


Statistics

Additional statistics for this record