Is Linguistic Inquiry and Word Count (LIWC) reliable, efficient, and effective for the analysis of large online datasets in forensic and security contexts?

Abstract

This article evaluates the reliability, efficiency, and effectiveness of Linguistic Inquiry and Word Count (LIWC; Boyd et al., 2022) for the analysis of a white nationalist forum. This is important because LIWC has been the computational tool of choice for scores of studies generally and many examining extremist content in a forensic or security context. Our purpose, therefore, is to understand whether LIWC can be depended upon for large-scale analyses; we initially examine this here using a small sample of posts from a set of just eight users and manually checking the program's automated codings of a subset of categories. Our results show that the LIWC coding cannot be relied upon – precision falls to as low as 49.6% and recall as low as 41.7% for some categories. It would be possible to engage in considerable manual correction of these results, but this undermines its purported efficiency for large datasets.