arXiv:2511.09854v2 Announce Type: replace 
Abstract: Large language models (LLMs) have demonstrated impressive performance in text generation tasks; however, their embedding spaces often suffer from the isotropy problem, resulting in poor discrimination of domain-specific terminology, particularly in legal and financial contexts. This weakness in terminology-level representation can severely hinder downstream tasks such as legal judgment prediction or financial risk analysis, where subtle semantic distinctions are critical. To address this problem, we propose TermGPT, a multi-level contrastive fine-tuning framework designed for terminology adaptation. We first construct a sentence graph to capture semantic and structural relations, and generate semantically consistent yet discriminative positive and negative samples based on contextual and topological cues. We then devise a multi-level contrastive learning approach at both the sentence and token levels, enhancing global contextual understanding and fine-grained terminology discrimination. To support robust evaluation, we construct the first financial terminology dataset derived from official regulatory documents. Experiments show that TermGPT outperforms existing baselines in term discrimination tasks within the finance and legal domains.

تم اقتراح إطار جديد يسمى TermGPT لتحسين تكيف المصطلحات في المجالات القانونية والمالية من خلال الضبط المتباين متعدد المستويات. تتناول هذه الطريقة مشكلة التماثل في نماذج اللغة الكبيرة، والتي غالبًا ما تؤدي إلى تمثيل غير كافٍ للمصطلحات الخاصة بالمجال، وهو أمر حاسم لمهام مثل التنبؤ بالأحكام القانونية وتحليل المخاطر المالية.

Se ha propuesto un nuevo marco llamado TermGPT para mejorar la adaptación de la terminología en los dominios legal y financiero a través de un ajuste contrastivo de múltiples niveles. Este enfoque aborda el problema de isotropía en los grandes modelos de lenguaje, que a menudo conduce a una representación inadecuada de la terminología específica del dominio, crucial para tareas como la predicción de juicios legales y el análisis de riesgos financieros.

Un nouveau cadre nommé TermGPT a été proposé pour améliorer l'adaptation terminologique dans les domaines juridique et financier grâce à un ajustement contrastif à plusieurs niveaux. Cette approche vise à résoudre le problème d'isotropie des grands modèles de langage, qui conduit souvent à une représentation inadéquate de la terminologie spécifique au domaine, cruciale pour des tâches telles que la prédiction de jugements juridiques et l'analyse des risques financiers.

A new framework named TermGPT has been proposed to enhance terminology adaptation in the legal and financial domains through multi-level contrastive fine-tuning. This approach addresses the isotropy problem in large language models, which often leads to inadequate representation of domain-specific terminology, crucial for tasks like legal judgment prediction and financial risk analysis.

TermGPT: Multi-Level Contrastive Fine-Tuning for Terminology Adaptation in Legal and Financial Domain

One More Thing in AI – Your Shortcut to AI Mastery

TermGPT: Multi-Level Contrastive Fine-Tuning for Terminology Adaptation in Legal and Financial Domain

Was this article worth reading? Share it

One More Thing in AI

Airparser

Humanize AI

LucidQuery AI

ZeroGPT.org

GPTBox

Ready to build your own newsroom?