Where Does Authorship Signal Emerge in Encoder-Based Language Models?

A recent study published on arXiv investigates the emergence of authorship signals in encoder-based language models, revealing that performance can vary significantly based on the scoring mechanism used, despite identical training conditions. The research employs mechanistic interpretability tools to analyze stylistic features across model layers, indicating that the scoring method influences where authorship signals are consolidated within the encoder.

This finding is crucial for the development of authorship attribution models, as it highlights the importance of scoring mechanisms in determining model effectiveness. Understanding these dynamics can lead to improved models that better capture authorship characteristics, enhancing applications in fields such as digital forensics and content creation.

The study contributes to ongoing discussions about the interpretability of large language models and their underlying mechanisms. It aligns with broader research themes exploring the reliability of model outputs, the impact of training dynamics, and the challenges of ensuring that models accurately reflect human-like reasoning and behavior.