ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance

Automated emotion detection systems are increasingly utilized in various fields, including mental health and hiring. However, these models often fail to accurately recognize emotional expressions in dialects like African American Vernacular English (AAVE) due to reliance on dominant cultural norms. A study analyzing 2.7 million tweets from Los Angeles found that emotion recognition models exhibited significantly higher false positive rates for anger in AAVE compared to General American English (GAE), highlighting the limitations of current emotion AI technologies.

The report on Spectral Neuro-Symbolic Reasoning II introduces enhancements to the existing framework, focusing on three key areas: transformer-based node merging to reduce redundancy, sentence-level entailment validation for improved edge quality, and alignment with external knowledge graphs to provide additional context. These modifications aim to enhance the fidelity of knowledge graphs while maintaining the spectral reasoning pipeline. Experimental results indicate accuracy gains of up to 3.8% across various benchmarks, including ProofWriter and CLUTRR.

This paper investigates the automated classification of exam questions and learning outcomes based on Bloom's Taxonomy. A dataset of 600 sentences was categorized into six cognitive levels: Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation. Various machine learning models, including traditional methods and large language models, were evaluated, with Support Vector Machines achieving the highest accuracy of 94%, while RNN models and BERT faced significant overfitting issues.

The article discusses a novel approach to in-context learning (ICL) for text classification, emphasizing the importance of selecting appropriate demonstrations. Traditional methods often prioritize semantic similarity, neglecting label distribution alignment, which can impact performance. The proposed method, TopK + Label Distribution Divergence (L2D), utilizes a fine-tuned BERT-like small language model to generate label distributions and assess their divergence. This dual focus aims to enhance the effectiveness of demonstration selection in large language models (LLMs).

Personal attacks have increasingly characterized U.S. presidential debates, influencing public perception during elections. This study presents a framework for analyzing such attacks using manual annotation of debate transcripts from the 2016, 2020, and 2024 election cycles. By leveraging advancements in deep learning, particularly BERT and large language models, the research aims to enhance the detection of harmful language in political discourse, providing valuable insights for journalists and the public.