arXiv:2511.10628v2 Announce Type: replace 
Abstract: Large language models (LLMs) have demonstrated remarkable performance across a wide range of tasks, yet the majority of high-performing models remain closed-source or partially open, limiting transparency and reproducibility. In this work, we introduce Instella, a family of fully open three billion parameter language models trained entirely on openly available data and codebase. Powered by AMD Instinct MI300X GPUs, Instella is developed through large-scale pre-training, general-purpose instruction tuning, and alignment with human preferences. Despite using substantially fewer pre-training tokens than many contemporaries, Instella achieves state-of-the-art results among fully open models and is competitive with leading open-weight models of comparable size. We further release two specialized variants: Instella-Long, capable of handling context lengths up to 128K tokens, and Instella-Math, a reasoning-focused model enhanced through supervised fine-tuning and reinforcement learning on mathematical tasks. Together, these contributions establish Instella as a transparent, performant, and versatile alternative for the community, advancing the goal of open and reproducible language modeling research.

تقدم Instella عائلة من نماذج اللغة المفتوحة بالكامل مع ثلاثة مليارات معلمة، تم تدريبها على بيانات ورموز متاحة علنًا. باستخدام وحدات معالجة الرسوميات AMD Instinct MI300X، تحقق Instella أداءً رائدًا على الرغم من استخدام عدد أقل من رموز التدريب المسبق مقارنة بالعديد من المنافسين. تشمل النماذج نوعين متخصصين: Instella-Long، القادر على التعامل مع أطوال السياق تصل إلى 128K رمز، وInstella-Math، المصمم لتعزيز قدرات التفكير. يعالج هذا التطور الحاجة إلى الشفافية وقابلية التكرار في الذكاء الاصطناعي.

Instella presenta una familia de modelos de lenguaje completamente abiertos con tres mil millones de parámetros, entrenados con datos y código disponibles públicamente. Utilizando GPUs AMD Instinct MI300X, Instella logra un rendimiento de vanguardia a pesar de usar menos tokens de preentrenamiento que muchos competidores. Los modelos incluyen dos variantes especializadas: Instella-Long, capaz de manejar longitudes de contexto de hasta 128K tokens, e Instella-Math, diseñado para mejorar las capacidades de razonamiento. Este desarrollo aborda la necesidad de transparencia y reproducibilidad en l…

Instella présente une famille de modèles de langage entièrement ouverts avec trois milliards de paramètres, formés sur des données et un code disponibles publiquement. Utilisant des GPU AMD Instinct MI300X, Instella atteint des performances de pointe malgré l'utilisation de moins de tokens de pré-formation que de nombreux concurrents. Les modèles incluent deux variantes spécialisées : Instella-Long, capable de gérer des longueurs de contexte allant jusqu'à 128K tokens, et Instella-Math, conçu pour des capacités de raisonnement améliorées. Ce développement répond à la nécessité de transparence …

Instella introduces a family of fully open large language models (LLMs) with three billion parameters, trained on openly available data and code. Utilizing AMD Instinct MI300X GPUs, Instella achieves state-of-the-art performance despite using fewer pre-training tokens than many competitors. The models include two specialized variants: Instella-Long, which can handle context lengths up to 128K tokens, and Instella-Math, designed for enhanced reasoning capabilities. This development addresses the need for transparency and reproducibility in AI.

Instella: Fully Open Language Models with Stellar Performance

One More Thing in AI – Your Shortcut to AI Mastery

Instella: Fully Open Language Models with Stellar Performance

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Chattermate

Magicley AI

Dyad

MindStudio

Ready to build your own newsroom?