arXiv:2511.17562v1 Announce Type: new 
Abstract: This paper introduces ChineseErrorCorrector3-4B, a unified model for Chinese spelling and grammatical error correction based on Qwen3-4B. The model demonstrates outstanding performance in general text correction tasks and achieves state-of-the-art results in both spelling correction (CSC) and grammatical correction (CGC). On several authoritative benchmark datasets -- including SIGHAN-2015, EC-LAW, MCSC, and NaCGEC -- the model's F1 and F0.5 scores significantly surpass existing publicly available models, ranking first in both spelling and grammatical error correction tasks.

يمثل تقديم ChineseErrorCorrector3-4B تقدمًا كبيرًا في مجال الذكاء الاصطناعي، وخاصة في تصحيح الأخطاء الإملائية والنحوية باللغة الصينية. لقد أظهر هذا النموذج الموحد، المستند إلى Qwen3-4B، أداءً استثنائيًا عبر مجموعة متنوعة من مجموعات البيانات المرجعية، بما في ذلك SIGHAN-2015 وEC-LAW، محققًا أعلى الدرجات في مهام تصحيح الأخطاء الإملائية والنحوية.

La introducción de ChineseErrorCorrector3-4B marca un avance significativo en el campo de la inteligencia artificial, específicamente en la corrección de errores ortográficos y gramaticales en chino. Este modelo unificado, basado en Qwen3-4B, ha demostrado un rendimiento excepcional en varios conjuntos de datos de referencia, incluidos SIGHAN-2015 y EC-LAW, logrando las mejores puntuaciones en tareas de corrección ortográfica y gramatical.

L'introduction de ChineseErrorCorrector3-4B représente une avancée significative dans le domaine de l'intelligence artificielle, en particulier dans la correction des erreurs d'orthographe et de grammaire en chinois. Ce modèle unifié, basé sur Qwen3-4B, a démontré des performances exceptionnelles sur divers ensembles de données de référence, y compris SIGHAN-2015 et EC-LAW, atteignant des scores de premier plan dans les tâches de correction orthographique et grammaticale.

The introduction of ChineseErrorCorrector3-4B marks a significant advancement in the field of artificial intelligence, specifically in the area of Chinese spelling and grammatical error correction. This unified model, based on Qwen3-4B, has demonstrated exceptional performance across various benchmark datasets, including SIGHAN-2015 and EC-LAW, achieving top scores in both spelling and grammatical correction tasks.

ChineseErrorCorrector3-4B: State-of-the-Art Chinese Spelling and Grammar Corrector

arXiv:2511.17127v1 Announce Type: new 
Abstract: We report on the first large-scale mixture-of-experts (MoE) pretraining study on pure AMD hardware, utilizing both MI300X GPUs with Pollara interconnect. We distill practical guidance for both systems and model design. On the systems side, we deliver a comprehensive cluster and networking characterization: microbenchmarks for all core collectives (all-reduce, reduce-scatter, all-gather, broadcast) across message sizes and GPU counts on Pollara. To our knowledge, this is the first at this scale. We further provide MI300X microbenchmarks on kernel sizing and memory bandwidth to inform model design. On the modeling side, we introduce and apply MI300X-aware transformer sizing rules for attention and MLP blocks and justify MoE widths that jointly optimize training throughput and inference latency. We describe our training stack in depth, including often-ignored utilities such as fault-tolerance and checkpoint-reshaping, as well as detailed information on our training recipe. We also provide a preview of our model architecture and base model - ZAYA1 (760M active, 8.3B total parameters MoE) - which will be further improved upon in forthcoming papers. ZAYA1-base achieves performance comparable to leading base models such as Qwen3-4B and Gemma3-12B at its scale and larger, and outperforms models including Llama-3-8B and OLMoE across reasoning, mathematics, and coding benchmarks. Together, these results demonstrate that the AMD hardware, network, and software stack are mature and optimized enough for competitive large-scale pretraining.

تم إجراء دراسة تدريب على نطاق واسع باستخدام مزيج من الخبراء (MoE) باستخدام أجهزة AMD النقية، وبالتحديد وحدات معالجة الرسوميات MI300X مع الاتصال Pollara. توفر هذه الدراسة إرشادات عملية حول تصميم الأنظمة والنماذج، بما في ذلك قياسات دقيقة شاملة لجمع البيانات الأساسية وقياسات MI300X الدقيقة لحجم النواة وعرض النطاق الترددي للذاكرة.

Se ha llevado a cabo un estudio de preentrenamiento a gran escala sobre una mezcla de expertos (MoE) utilizando hardware puro de AMD, específicamente GPUs MI300X con interconexión Pollara. Este estudio proporciona orientación práctica sobre el diseño de sistemas y modelos, incluidos microbenchmarks completos para colectivos centrales y microbenchmarks MI300X para el tamaño de núcleos y el ancho de banda de memoria.

Une étude de préentraînement à grande échelle sur un mélange d'experts (MoE) a été réalisée en utilisant du matériel AMD pur, en particulier des GPU MI300X avec interconnexion Pollara. Cette étude fournit des conseils pratiques sur la conception des systèmes et des modèles, y compris des microbenchmarks complets pour les collectifs de base et des microbenchmarks MI300X pour la taille des noyaux et la bande passante mémoire.

A large-scale mixture-of-experts (MoE) pretraining study has been conducted using pure AMD hardware, specifically MI300X GPUs with Pollara interconnect. This study provides practical guidance on system and model design, including comprehensive microbenchmarks for core collectives and MI300X microbenchmarks for kernel sizing and memory bandwidth.

ChineseErrorCorrector3-4B: State-of-the-Art Chinese Spelling and Grammar Corrector

Was this article worth reading? Share it

GPTBox

Grubby.AI

EduWriter