Xie, Qian, Cheng, Ta-Ying, Dai, Zhuangzhuang, Tran, Vu, Trigoni, Niki and Markham, Andrew (2023). Illumination-Aware Hallucination-Based Domain Adaptation for Thermal Pedestrian Detection. IEEE Transactions on Intelligent Transportation Systems ,
Abstract
Thermal imagery is emerging as a viable candidate for 24-7, all-weather pedestrian detection owning to thermal sensors’ robust performance for pedestrian detection under different weather and illumination conditions. Despite the promising results obtained from combining visible (RGB) and thermal cameras in multi-spectral fusion techniques, the complex synchronization requirements, including alignment and calibration of sensors, impede their deployment in real-world scenarios. In this paper, we introduce a novel approach for domain adaptation to enhance the performance of pedestrian detection based solely on thermal images. Our proposed approach involves several stages. Firstly, we use both thermal and visible images as input during the training phase. Secondly, we leverage a thermal-to-visible hallucination network to generate feature maps that are similar to those generated by the visible branch. Finally, we design a transformer-based multi-modal fusion module to integrate the hallucinated visible and thermal information more effectively. The thermal-to-visible hallucination network acts as domain adaptation, allowing us to obtain pseudo-visual and thermal features using solely thermal input. Based on the experimental results, it is observed the mean average precision (mAP) increases by 4.72% and the miss rate decreases by 7.56% on the KAIST dataset when compared to the baseline model.
Publication DOI: | https://doi.org/10.1109/tits.2023.3307167 |
---|---|
Divisions: | College of Engineering & Physical Sciences College of Engineering & Physical Sciences > School of Computer Science and Digital Technologies > Applied AI & Robotics College of Engineering & Physical Sciences > School of Computer Science and Digital Technologies College of Engineering & Physical Sciences > Aston Centre for Artifical Intelligence Research and Application College of Engineering & Physical Sciences > Smart and Sustainable Manufacturing |
Additional Information: | Copyright © 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. Funding: This work was supported by the Engineering and Physical Sciences Research Council (EPSRC) Program “ACE-OPS: From Autonomy to Cognitive assistance in Emergency OPerationS” under Grant EP/S030832/1. |
Uncontrolled Keywords: | Pedestrian detection,modality hallucination,thermal image,transformed-based fusion,Mechanical Engineering,Automotive Engineering,Computer Science Applications |
Publication ISSN: | 1558-0016 |
Last Modified: | 07 Oct 2024 07:51 |
Date Deposited: | 13 Sep 2023 10:02 |
Full Text Link: | |
Related URLs: |
https://ieeexpl ... cument/10238358
(Publisher URL) http://www.scop ... tnerID=8YFLogxK (Scopus URL) |
PURE Output Type: | Article |
Published Date: | 2023-09-01 |
Published Online Date: | 2023-09-01 |
Accepted Date: | 2023-08-04 |
Authors: |
Xie, Qian
Cheng, Ta-Ying Dai, Zhuangzhuang ( 0000-0002-6098-115X) Tran, Vu Trigoni, Niki Markham, Andrew |