arXiv:2409.17892v3 Announce Type: replace 
Abstract: In this work, we introduce EMMA-500, a large-scale multilingual language model continue-trained on texts across 546 languages designed for enhanced multilingual performance, focusing on improving language coverage for low-resource languages. To facilitate continual pre-training, we compile the MaLA corpus, a comprehensive multilingual dataset enriched with curated datasets across diverse domains. Leveraging this corpus, we conduct extensive continual pre-training of the Llama 2 7B model, resulting in EMMA-500, which demonstrates robust performance across a wide collection of benchmarks, including a comprehensive set of multilingual tasks. Our results highlight the effectiveness of continual pre-training in expanding large language models' language capacity, particularly for underrepresented languages, demonstrating significant gains in cross-lingual transfer, task generalization, and language adaptability. We release the MaLA corpus, EMMA-500 model weights, scripts, and model generations.

تم تقديم EMMA-500 كنموذج لغوي متعدد اللغات على نطاق واسع، حيث تم استكمال تدريب نموذج Llama 2 7B على نصوص من 546 لغة، بهدف تحسين الأداء متعدد اللغات، خاصة للغات ذات الموارد المحدودة. يعتمد النموذج على مجموعة بيانات MaLA، وهي مجموعة شاملة تم تجميعها للتدريب المستمر.

Se ha presentado EMMA-500 como un modelo de lenguaje multilingüe a gran escala, continuando el entrenamiento del modelo Llama 2 7B con textos de 546 idiomas, con el objetivo de mejorar el rendimiento multilingüe, especialmente para idiomas de bajos recursos. El modelo se apoya en el corpus MaLA, un conjunto de datos integral compilado para el preentrenamiento continuo.

EMMA-500 a été introduit comme un modèle de langage multilingue à grande échelle, poursuivant l'entraînement du modèle Llama 2 7B sur des textes provenant de 546 langues, visant à améliorer les performances multilingues, en particulier pour les langues à faibles ressources. Le modèle est soutenu par le corpus MaLA, un ensemble de données complet compilé pour le pré-entraînement continu.

EMMA-500 has been introduced as a large-scale multilingual language model, continuing the training of the Llama 2 7B model on texts from 546 languages, aimed at enhancing multilingual performance, particularly for low-resource languages. The model is supported by the MaLA corpus, a comprehensive dataset compiled for continual pre-training.

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models

arXiv:2512.06812v1 Announce Type: new 
Abstract: Discharge Summaries are documents written by medical professionals that detail a patient's visit to a care facility. They contain a wealth of information crucial for patient care, and automating their generation could significantly reduce the effort required from healthcare professionals, minimize errors, and ensure that critical patient information is easily accessible and actionable. In this work, we explore the use of five Large Language Models on this task, from open-source models (Mistral, Llama 2) to proprietary systems (GPT-3, GPT-4, Gemini 1.5 Pro), leveraging MIMIC-III summaries and notes. We evaluate them using exact-match, soft-overlap, and reference-free metrics. Our results show that proprietary models, particularly Gemini with one-shot prompting, outperformed others, producing summaries with the highest similarity to the gold-standard ones. Open-source models, while promising, especially Mistral after fine-tuning, lagged in performance, often struggling with hallucinations and repeated information. Human evaluation by a clinical expert confirmed the practical utility of the summaries generated by proprietary models. Despite the challenges, such as hallucinations and missing information, the findings suggest that LLMs, especially proprietary models, are promising candidates for automatic discharge summary generation as long as data privacy is ensured.

أظهرت الأبحاث الحديثة إمكانية نماذج اللغة الكبيرة (LLMs) في أتمتة إنشاء ملخصات الخروج، وهي مستندات حيوية في رعاية المرضى. قامت الدراسة بتقييم خمسة نماذج، بما في ذلك أنظمة ملكية مثل GPT-4 وGemini 1.5 Pro، ووجدت أن Gemini، خاصة مع التحفيز الأحادي، أنتجت ملخصات كانت الأكثر تشابهًا مع المعايير الذهبية. يمكن أن يؤدي هذا التقدم إلى تقليل عبء العمل على المتخصصين في الرعاية الصحية وتحسين دقة معلومات المرضى.

Investigaciones recientes han demostrado el potencial de los Modelos de Lenguaje de Gran Escala (LLMs) en la automatización de la generación de resúmenes de alta de pacientes, documentos críticos en la atención médica. El estudio evaluó cinco modelos, incluidos sistemas propietarios como GPT-4 y Gemini 1.5 Pro, y encontró que Gemini, especialmente con un solo aviso, producía resúmenes más similares a los estándares de oro. Este avance podría reducir significativamente la carga de trabajo de los profesionales de la salud y mejorar la precisión de la información del paciente.

Des recherches récentes ont démontré le potentiel des modèles de langage de grande taille (LLMs) dans l'automatisation de la génération de résumés de sortie, qui sont des documents critiques dans les soins aux patients. L'étude a évalué cinq modèles, y compris des systèmes propriétaires comme GPT-4 et Gemini 1.5 Pro, et a constaté que Gemini, en particulier avec un prompt unique, produisait des résumés les plus similaires aux normes d'or. Cette avancée pourrait réduire considérablement la charge de travail des professionnels de la santé et améliorer l'exactitude des informations sur les patien…

Recent research has demonstrated the potential of Large Language Models (LLMs) in automating the generation of discharge summaries, which are critical documents in patient care. The study evaluated five models, including proprietary systems like GPT-4 and Gemini 1.5 Pro, and found that Gemini, particularly with one-shot prompting, produced summaries most similar to gold standards. This advancement could significantly reduce the workload of healthcare professionals and enhance the accuracy of patient information.

EMMA-500: Enhancing Massively Multilingual Adaptation of Large Language Models

Was this article worth reading? Share it

LucidQuery AI

Chattermate

FastML