LLMs vs. Traditional Sentiment Tools in Psychology: An Evaluation on Belgian-Dutch Narratives

The evaluation of Dutch-specific Large Language Models (LLMs) against traditional sentiment analysis tools has revealed unexpected results. Conducted with approximately 25,000 spontaneous textual responses from 102 Dutch-speaking participants, the study aimed to assess how well these models could predict emotional valence in Flemish, a low-resource language variant. Contrary to expectations, the LLMs, including ChocoLlama-8B-Instruct, Reynaerde-7B-chat, and GEITje-7B-ultra, did not outperform the traditional tool Pattern, which demonstrated superior performance. This finding challenges the prevailing assumption that LLMs are inherently better suited for sentiment analysis tasks. The results underscore the complexity of capturing emotional nuances in real-world narratives and emphasize the necessity for developing culturally and linguistically tailored evaluation frameworks to enhance the effectiveness of sentiment analysis tools in diverse linguistic contexts.