A Deep Learning Approach to Automatic Caption Generation for News Images

Batra, Vishwash, He, Yulan and Vogiatzis, George (2019). A Deep Learning Approach to Automatic Caption Generation for News Images. IN: The 11th International Conference on Language Resources and Evaluation (LREC). European Language Resources Association (ELRA).

Abstract

Automatic caption generation of images has gained significant interest. It gives rise to a lot of interesting image-related applications. For example, it could help in image/video retrieval and management of vast amount of multimedia data available on the Internet. It could also help in development of tools that can aid visually impaired individuals in accessing multimedia content. In this paper, we particularly focus on news images and propose a methodology for automatically generating captions for news paper articles consisting of a text paragraph and an image. We propose several deep neural network architectures built upon Recurrent Neural Networks. Results on a BBC News dataset show that our proposed approach outperforms a traditional method based on Latent Dirichlet Allocation using both automatic evaluation based on BLEU scores and human evaluation.

Divisions: Engineering & Applied Sciences > Computer science
Engineering & Applied Sciences > Systems analytics research institute (SARI)
Engineering & Applied Sciences > Computer science research group
Additional Information: The LREC 2018 Proceedings are licensed under a Creative Commons Attribution-NonCommercial 4.0 International License
Event Title: The 11th International Conference on Language Resources and Evaluation (LREC)
Event Type: Other
Event Dates: 2018-05-07 - 2018-05-12
Uncontrolled Keywords: Recurrent Neural Networks,Image caption generation,Deep learning,Order Embedding
Published Date: 2019-01-01
Accepted Date: 2018-12-20
Authors: Batra, Vishwash
He, Yulan ( 0000-0003-3948-5845)
Vogiatzis, George

Download

Export / Share Citation


Statistics

Additional statistics for this record