BERT-Based Transformers for Early Detection of Mental Health Illnesses


This paper briefly describes our research groups' efforts in tackling Task 1 (Early Detection of Signs of Self-Harm), and Task 2 (Measuring the Severity of the Signs of Depression) from the CLEF eRisk Track. Core to how we approached these problems was the use of BERT-based classifiers which were trained specifically for each task. Our results on both tasks indicate that this approach delivers high performance across a series of measures, particularly for Task 1, where our submissions obtained the best performance for precision, F1, latency-weighted F1 and ERDE at 5 and 50. This work suggests that BERT-based classifiers, when trained appropriately, can accurately infer which social media users are at risk of self-harming, with precision up to 91.3% for Task 1. Given these promising results, it will be interesting to further refine the training regime, classifier and early detection scoring mechanism, as well as apply the same approach to other related tasks (e.g., anorexia, depression, suicide).

Publication DOI:
Divisions: ?? 50811700Jl ??
College of Business and Social Sciences > Aston Institute for Forensic Linguistics
Additional Information: Copyright © 2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0). Acknowledgements & Funding: The first author would like to thank the following funding bod- ies for their support: FEDER/Ministerio de Ciencia, Innovaci ́on y Universidades, Agencia Estatal de Investigaci ́on/Project (RTI2018-093336-B-C21), Conseller ́ıa de Educacion, Universidade e Formaci ́on Profesional and the European Regional DevelopmentFund (ERDF) (accreditation 2019–2022 ED431G-2019/04, ED431C 2018/29, ED431C2018/19). The second and third authors would like to thank the UKRI’s EPSRC Project Cumulative Revelations in Personal Data (Grant Number: EP/R033897/1) for theirsupport. They would also like to thank David Losada for arranging this collaboration.
Event Title: 12th International Conference of the CLEF Association
Event Type: Other
Event Location: Virtual Event
Event Dates: 2021-09-21 - 2021-09-24
Uncontrolled Keywords: Self-harm,Depression,Classification,Social media,Early detection,BERT,XLM-RoBERTa
ISBN: 9783030852504, 9783030852511
Last Modified: 21 May 2024 07:20
Date Deposited: 20 Dec 2022 15:46
Full Text Link:
Related URLs: https://link.sp ... -030-85251-1_15 (Publisher URL)
https://purepor ... 90-2cbed04df588 (Author URL)
PURE Output Type: Conference contribution
Published Date: 2021-09-14
Published Online Date: 2021-08-24
Authors: Martínez-Castaño, Rodrigo
Htait, Amal (ORCID Profile 0000-0003-4647-9996)
Azzopardi, Leif
Moshfeghi, Yashar

Export / Share Citation


Additional statistics for this record