Bolmo’s architecture unlocks efficient byte‑level LM training without sacrificing quality

The Allen Institute for AI has launched Bolmo 7B and Bolmo 1B, which it claims are the first fully open byte-level language models, built on its Olmo 3 models. These models aim to provide tokenizer-free multilingual capabilities, addressing the need for more robust language processing in noisy or low-resource text environments.

Tech giants, including Meta, are investing heavily in new data centers to support artificial intelligence initiatives, with a significant project underway in Eagle Mountain, Utah. These facilities can cost tens of billions to build, prompting companies to seek ways to mitigate financial risks associated with these investments.

In a recent debate, Yann LeCun, Meta's Chief AI Scientist, argued against the efficacy of Large Language Models (LLMs) in achieving human-like intelligence, stating that their predictive nature represents a dead end in the quest for Artificial General Intelligence (AGI). This discussion took place alongside DeepMind researcher Adam Brown, highlighting fundamental concerns regarding the limitations of current AI models.

A new framework called meta-statistical learning has been introduced, which utilizes supervised learning to optimize the design of statistical estimators. This approach addresses the challenges of crafting estimators with desirable properties, such as bias and variance, by employing permutation-invariant neural networks like Set Transformers to analyze entire datasets.