Catching Feelings: Aspect-Based Sentiment Analysis for Fanfiction Comments about Greek Myth

Abstract

The application of sentiment analysis in literary studies has been limited and often criticized, yet aspect-based sentiment analysis (ABSA) offers interesting applications in this domain because it addresses some limitations of traditional SA tools and provides a more detailed and context-sensitive analysis of sentiment. To investigate its usage in literary reception studies, we apply ABSA to a corpus of ±25,000 comments written by readers in response to fanfiction about Greek mythology on fanfiction website Archive of Our Own (AO3), one of the largest platforms for fanfiction in English. Our ABSA pipeline detects sentiment (positive/negative) associated with eight aspects of fanfiction stories (general evaluation, Greek mythology, character, character emotion, reading experience, writing style, events and storyworld, and non-specific sentiment). We explain the process of data collection and annotation and present a small inter-annotator agreement study (Pairwise Cohen’s 0.86 for aspects and 0.88 for sentiments). We develop, evaluate, and fine-tune a machine-learning pipeline for ABSA, tackling the aspect extraction and sentiment analysis tasks respectively. We obtain the best results using NuNER for aspect extraction (0.5 macro F1) and Twitter-roBERTa-sentiment for sentiment analysis (0.75 macro F1). Finally, we outline some avenues for future research and reflect on the generalizability of our method to other domains, especially to fanfiction from other fandoms and platforms but also other social media.