Multi-scale pedestrian intent prediction using 3D joint information as spatio-temporal representation


There has been a rise of use of Autonomous Vehicles on public roads. With the predicted rise of road traffic accidents over the coming years, these vehicles must be capable of safely operate in the public domain. The field of pedestrian detection has significantly advanced in the last decade, providing high-level accuracy, with some technique reaching near-human level accuracy. However, there remains further work required for pedestrian intent prediction to reach human-level performance. One of the challenges facing current pedestrian intent predictors are the varying scales of pedestrians, particularly smaller pedestrians. This is because smaller pedestrians can blend into the background, making them difficult to detect, track or apply pose estimations techniques. Therefore, in this work, we present a novel intent prediction approach for multi-scale pedestrians using 2D pose estimation and a Long Short-term memory (LSTM) architecture. The pose estimator predicts keypoints for the pedestrian along the video frames. Based on the accumulation of these keypoints along the frames, spatio-temporal data is generated. This spatio-temporal data is fed to the LSTM for classifying the crossing behaviour of the pedestrians. We evaluate the performance of the proposed techniques on the popular Joint Attention in Autonomous Driving (JAAD) dataset and the new larger-scale Pedestrian Intention Estimation (PIE) dataset. Using data generalisation techniques, we show that the proposed technique outperformed the state-of-the-art techniques by up to 7%, reaching up to 94% accuracy while maintaining a comparable run-time of 6.1 ms.

Publication DOI:
Divisions: College of Business and Social Sciences > Aston Business School
College of Business and Social Sciences > Aston Business School > Operations & Information Management
Additional Information: Copyright © 2023 The Author(s). Published by Elsevier Ltd. This is an open access article under the CC BY license (
Uncontrolled Keywords: Intent prediction,LSTM,Pedestrian detection,Pose estimation,Tracking,Engineering(all),Computer Science Applications,Artificial Intelligence
Publication ISSN: 1873-6793
Last Modified: 20 Jun 2024 07:25
Date Deposited: 27 Apr 2023 13:44
Full Text Link:
Related URLs: http://www.scop ... tnerID=8YFLogxK (Scopus URL)
PURE Output Type: Article
Published Date: 2023-09-01
Published Online Date: 2023-04-13
Accepted Date: 2023-04-06
Authors: Ahmed, Sarfraz
Al-Bazi, Ammar (ORCID Profile 0000-0002-5057-4171)
Saha, Chitta
Rajbhandari, Sujan
Huda, M. Nazmul



Version: Published Version

License: Creative Commons Attribution

| Preview

Export / Share Citation


Additional statistics for this record