stable-pretraining-v1: Foundation Model Research Made Simple

A new study introduces the Interleaved Multi-Domain Identity Curriculum (IMIC), enabling models to perform object recognition, face recognition from varying image qualities, and person recognition in a unified embedding space without significant catastrophic forgetting. This approach was tested on foundation models DINOv3, CLIP, and EVA-02, demonstrating comparable performance to domain experts across all tasks.

The QiMeng-Kernel framework introduces a Macro-Thinking Micro-Coding paradigm aimed at enhancing the generation of high-performance GPU kernels for AI and scientific computing. This approach addresses the challenges of correctness and efficiency in existing LLM-based methods by decoupling optimization strategies from implementation details, thereby improving both aspects significantly.

A new benchmark called MMTU has been introduced, featuring over 28,000 questions across 25 real-world table tasks, aimed at enhancing the evaluation of large language models (LLMs) in table-based applications. This initiative addresses the current limitations in benchmarking table-related tasks, which have been largely overlooked compared to other NLP benchmarks.

The introduction of STAlloc, a new GPU memory allocator for deep learning frameworks, aims to enhance memory efficiency during large-scale model training by reducing fragmentation caused by existing online memory allocators that overlook tensor lifespans. This innovation is particularly relevant as the demand for large language models (LLMs) continues to grow, leading to increased GPU memory pressure and potential out-of-memory errors.

NNGPT has been introduced as an open-source framework that transforms large language models into self-improving AutoML engines, particularly for neural network development in computer vision. This framework enhances neural network datasets by generating new models, allowing for continuous fine-tuning through a closed-loop system of generation, assessment, and self-improvement.

A recent study introduces Concept-Aware Batch Sampling (CABS), a novel framework designed to enhance language-image pretraining by utilizing a dynamic, concept-based approach to data curation. This method builds on DataConcept, a dataset of 128 million annotated image-text pairs, allowing for more adaptive and efficient training processes in vision-language models.

A novel framework called the correlation adaptation prompt network (CAPNET) has been proposed to enhance long-tailed multi-label visual recognition, addressing the challenges posed by imbalanced class distributions in datasets. This approach leverages pre-trained vision-language models like CLIP to better model label correlations, aiming to improve performance on tail classes that are often neglected in traditional methods.

President Donald Trump announced the Genesis Mission on November 24, 2025, likening it to the Manhattan Project, aiming to revolutionize scientific research in the U.S. The initiative directs the Department of Energy to create a closed-loop AI experimentation platform that integrates national laboratories and government data for enhanced research capabilities.