Hand-Object Interaction: From Human Demonstrations to Robot Manipulation


Human-object interaction is of great relevance for robots to operate in human environments. However, state-of-the-art robotic hands are far from replicating humans skills. It is, therefore, essential to study how humans use their hands to develop similar robotic capabilities. This article presents a deep dive into hand-object interaction and human demonstrations, highlighting the main challenges in this research area and suggesting desirable future developments. To this extent, the article presents a general definition of the hand-object interaction problem together with a concise review for each of the main subproblems involved, namely: sensing, perception, and learning. Furthermore, the article discusses the interplay between these subproblems and describes how their interaction in learning from demonstration contributes to the success of robot manipulation. In this way, the article provides a broad overview of the interdisciplinary approaches necessary for a robotic system to learn new manipulation skills by observing human behavior in the real world.

Publication DOI: https://doi.org/10.3389/frobt.2021.714023
Divisions: College of Engineering & Physical Sciences
College of Engineering & Physical Sciences > Aston Institute of Urban Technology and the Environment (ASTUTE)
College of Engineering & Physical Sciences > Systems analytics research institute (SARI)
Additional Information: Copyright © 2021 Carfì, Patten, Kuang, Hammoud, Alameh, Maiettini, Weinberg, Faria, Mastrogiovanni, Alenyà, Natale, Perdereau, Vincze and Billard. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Funding: This work is supported by the CHIST-ERA (2014-2020) project InDex and received funding from the Italian Ministry of Education and Research (MIUR), Austrian Science Fund (FWF) under grant agreement No. I3969-N30, Engineering and Physical Sciences Research Council (EPSRC-UK) with reference EP/S032355/1 and Agence Nationale de la Recherche (ANR) under grant agreement No. ANR-18-CHR3-0004. This work is also supported by the ERA-net CHIST-ERA project BURG (PCIN2019-103447).
Uncontrolled Keywords: Robotics and AI,hand-object interaction,learning from demonstration,imitation learning,transfer learning,grasping,manipulation,anthropomorphic hands,data extraction
Publication ISSN: 2296-9144
Last Modified: 14 May 2024 17:06
Date Deposited: 15 Oct 2021 07:11
Full Text Link:
Related URLs: https://www.fro ... 021.714023/full (Publisher URL)
PURE Output Type: Article
Published Date: 2021-10-01
Accepted Date: 2021-09-14
Submitted Date: 2021-05-24
Authors: Carfì, Alessandro
Patten, Timothy
Kuang, Yingyi
Hammoud, Ali
Alameh, Mohamad
Maiettini, Elisa
Weinberg, Abraham Itzhak
Faria, Diego (ORCID Profile 0000-0002-2771-1713)
Mastrogiovanni, Fulvio
Alenyà, Guillem
Natale, Lorenzo
Perdereau, Véronique
Vincze, Markus
Billard, Aude



Version: Published Version

License: Creative Commons Attribution

| Preview

Export / Share Citation


Additional statistics for this record