<A HREF="https://venturebeat.com/ai/bolmos-architecture-unlocks-efficient-byte-level-lm-training-without"><IMG VSPACE="4" HSPACE="4" BORDER="0" ALIGN="RIGHT" SRC="http://www.techmeme.com/251215/i42.jpg"></A>
<A HREF="http://www.techmeme.com/251215/p42#a251215p42" TITLE="Techmeme permalink"><IMG WIDTH=11 HEIGHT=12 SRC="http://www.techmeme.com/img/pml.png" STYLE="border:none;padding:0;margin:0;"></A> Emilia David / <A HREF="https://venturebeat.com/">VentureBeat</A>: 
<A HREF="https://venturebeat.com/ai/bolmos-architecture-unlocks-efficient-byte-level-lm-training-without">Allen Institute for AI launches Bolmo 7B and Bolmo 1B, claiming they are &ldquo;the first fully open byte-level language models&rdquo;, built on its Olmo 3 models</A>&nbsp; &mdash;&nbsp; Enterprises that want tokenizer-free multilingual models are increasingly turning to byte-level language models to reduce brittleness in noisy or low-resource text.

أطلق معهد ألين للذكاء الاصطناعي نموذج Bolmo 7B وBolmo 1B، والذي يدعي أنهما أول نماذج لغة على مستوى البايت مفتوحة بالكامل، مبنية على نماذج Olmo 3 الخاصة به. تهدف هذه النماذج إلى توفير قدرات متعددة اللغات بدون الحاجة إلى أدوات تقسيم الكلمات، مما يلبي الحاجة إلى معالجة لغوية أكثر قوة في بيئات نصية صاخبة أو ذات موارد منخفضة.

El Instituto Allen para la IA ha lanzado Bolmo 7B y Bolmo 1B, que afirma ser los primeros modelos de lenguaje de nivel byte completamente abiertos, construidos sobre sus modelos Olmo 3. Estos modelos buscan proporcionar capacidades multilingües sin tokenización, abordando la necesidad de un procesamiento del lenguaje más robusto en entornos textuales ruidosos o de bajos recursos.

L'Institut Allen pour l'IA a lancé Bolmo 7B et Bolmo 1B, qu'il affirme être les premiers modèles de langage byte-level entièrement ouverts, construits sur ses modèles Olmo 3. Ces modèles visent à fournir des capacités multilingues sans tokenizer, répondant au besoin de traitements linguistiques plus robustes dans des environnements textuels bruyants ou à faibles ressources.

The Allen Institute for AI has launched Bolmo 7B and Bolmo 1B, which it claims are the first fully open byte-level language models, built on its Olmo 3 models. These models aim to provide tokenizer-free multilingual capabilities, addressing the need for more robust language processing in noisy or low-resource text environments.

Allen Institute for AI launches Bolmo 7B and Bolmo 1B, claiming they are "the first fully open byte-level language models", built on its Olmo 3 models (Emilia David/VentureBeat)

Enterprises that want tokenizer-free multilingual models are increasingly turning to byte-level language models to reduce brittleness in noisy or low-resource text. To tap into that niche — and make it practical at scale — the Allen Institute of AI (Ai2) introduced Bolmo, a new family of models that leverage its <a href="https://venturebeat.com/ai/ai2s-olmo-3-family-challenges-qwen-and-llama-with-efficient-open-reasoning">Olmo 3 models</a> by “bytefiying” them and reusing their backbone and capabilities. The company launched two versions, Bolmo 7B and Bolmo 1B, which are “the first fully open byte-level language model,” <a href="https://allenai.org/blog/bolmo">according to Ai2</a>. The company said the two models performed competitively with — and in some cases surpassed — other byte-level and character-based models.<div></div>Byte-level language models operate directly on raw UTF-8 bytes, eliminating the need for a predefined vocabulary or tokenizer. This allows them to handle misspellings, rare languages, and unconventional text more reliably — key requirements for moderation, edge deployments, and multilingual applications.For enterprises deploying AI across multiple languages, noisy user inputs, or constrained environments, tokenizer-free models offer a way to reduce operational complexity. Ai2’s Bolmo is an attempt to make that approach practical at scale — without retraining from scratch.<h2>How Bolmo works and how it was built </h2>Ai2 said it trained the Bolmo models using its Dolma 3 data mix, which helped train its <a href="https://venturebeat.com/ai/ai2s-new-olmo-3-1-extends-reinforcement-learning-training-for-stronger">Olmo flagship models</a>, and some open code datasets and character-level data.The company said its goal “is to provide a reproducible, inspectable blueprint for byteifying strong subword language models in a way the community can adopt and extend.” To meet this goal, Ai2 will release its checkpoints, code, and <a href="https://www.datocms-assets.com/64837/1765814974-bolmo.pdf">a full paper</a> to help other organizations build byte-level models on top of its Olmo ecosystem. Since training a byte-level model completely from scratch can get expensive, Ai2 researchers instead chose an existing Olmo 3 7B checkpoint to byteify in two stages. In the first stage, Ai2 froze the Olmo 3 transformer so that they only train certain parts, such as the local encoder and decoder, the boundary predictor, and the language modeling head. This was designed to be “cheap and fast” and requires just 9.8 billion tokens. The next stage unfreezes the model and trains it with additional tokens. Ai2 said the byte-level approach allows Bolmo to avoid the vocabulary bottlenecks that limit traditional subword models.<h2>Strong performance among its peers</h2>Byte-level language models are not as mainstream as small language models or LLMs, but this is a growing field in research. <a href="https://venturebeat.com/ai/metas-new-blt-architecture-replaces-tokens-to-make-llms-more-efficient-and-versatile">Meta released its BLT architecture</a> research last year, aiming to offer a model that is robust, processes raw data, and doesn’t rely on fixed vocabularies. Other research models in this space <a href="https://arxiv.org/abs/2105.13626">include ByT5</a>, <a href="https://arxiv.org/abs/2410.20771">Stanford’s MrT5</a>, and <a href="https://huggingface.co/transformers/v4.12.5/model_doc/canine.html">Canine</a>.  Ai2 evaluated Bolmo using its evaluation suite, covering math, STEM reasoning, question answering, general knowledge, and code. Bolmo 7B showed strong performance, outperforming character-focused benchmarks like CUTE and EXECUTE, and also improving accuracy over the base LLM Olmo 3. Bolmo 7B outperformed models of comparable size in coding, math, multiple-choice QA, and character-level understanding. <h2>Why enterprises may choose byte-level models</h2>Enterprises find value in a hybrid model structure, using a mix of models and model sizes. Ai2 makes the case that organizations should also consider byte-level models not only for robustness and multilingual understanding, but because it “naturally plugs into an existing model ecosystem.”“A key advantage of the dynamic hierarchical setup is that compression becomes a toggleable knob,” the company said. For enterprises already running heterogeneous model stacks, Bolmo suggests that byte-level models may no longer be purely academic. By retrofitting a strong subword model rather than training from scratch, Ai2 is signaling a lower-risk path for organizations that want robustness without abandoning existing infrastructure.

أطلق معهد ألين للذكاء الاصطناعي (Ai2) نموذج بولمو، وهو عائلة جديدة من نماذج اللغة على مستوى البايت مصممة للعمل بدون مفردات أو أدوات تقسيم مسبقة، مما يعزز الكفاءة في تدريب النماذج متعددة اللغات. يتميز النموذجان، بولمو 7B و بولمو 1B، بأدائهما التنافسي مقارنة بالنماذج الحالية المعتمدة على البايتات والشخصيات.

El Instituto Allen de IA (Ai2) ha lanzado Bolmo, una nueva familia de modelos de lenguaje a nivel de bytes diseñados para operar sin vocabularios o tokenizadores predefinidos, mejorando así la eficiencia en el entrenamiento de modelos multilingües. Los modelos, Bolmo 7B y Bolmo 1B, se destacan por su rendimiento competitivo frente a modelos existentes basados en bytes y caracteres.

L'Institut Allen pour l'IA (Ai2) a lancé Bolmo, une nouvelle famille de modèles de langage au niveau des octets conçus pour fonctionner sans vocabulaires ou tokenizers prédéfinis, améliorant ainsi l'efficacité de l'entraînement des modèles multilingues. Les modèles, Bolmo 7B et Bolmo 1B, se distinguent par leurs performances compétitives par rapport aux modèles basés sur des octets et des caractères existants.

The Allen Institute of AI (Ai2) has launched Bolmo, a new family of byte-level language models designed to operate without predefined vocabularies or tokenizers, thus enhancing efficiency in multilingual model training. The models, Bolmo 7B and Bolmo 1B, are noted for their competitive performance against existing byte-level and character-based models.

Bolmo’s architecture unlocks efficient byte‑level LM training without sacrificing quality

أطلق معهد ألين للذكاء الاصطناعي Bolmo 7B وBolmo 1B، اللذان يُعتبران أول نماذج لغوية على مستوى البايت مفتوحة بالكامل. تم بناؤها على بنية Olmo 3، وتلغي هذه النماذج الحاجة إلى أدوات تقسيم الكلمات، مما يعزز قدرات المعالجة متعددة اللغات وكفاءة التدريب. تم تصميمها لتكون تنافسية في البيئات المليئة بالضوضاء أو ذات الموارد المنخفضة، مما يمثل تقدمًا كبيرًا في تكنولوجيا نماذج اللغة.

El Allen Institute for AI ha presentado Bolmo 7B y Bolmo 1B, considerados como los primeros modelos de lenguaje byte-level completamente abiertos. Construidos sobre la arquitectura Olmo 3, estos modelos eliminan la necesidad de tokenizadores, mejorando así las capacidades de procesamiento multilingüe y la eficiencia en el entrenamiento. Están diseñados para competir en entornos ruidosos o de bajos recursos, marcando un avance significativo en la tecnología de modelos de lenguaje.

L'Allen Institute for AI a lancé Bolmo 7B et Bolmo 1B, présentés comme les premiers modèles de langage byte-level entièrement ouverts. Basés sur l'architecture Olmo 3, ces modèles éliminent le besoin de tokenizers, améliorant ainsi les capacités de traitement multilingue et l'efficacité de l'entraînement. Ils sont conçus pour performer de manière compétitive dans des environnements bruyants ou à faibles ressources, marquant une avancée significative dans la technologie des modèles de langage.

Allen Institute for AI Unveils Bolmo 7B and 1B, Pioneering Open Byte-Level Language Models

Advertise Now

Own the #1 slot on Top Trending. Massive visibility when attention is highest.

Allen Institute for AI Unveils Bolmo 7B and 1B, Pioneering Open Byte-Level Language Models

Allen Institute for AI Unveils Bolmo 7B and 1B, Pioneering Open Byte-Level Language Models

Why World Pulse Now

Global Coverage

Emotional Lens

Trending Stories

Read Less, Know More

Multi-Language

Save for Later

Live Stats

Featured Products