Bee: A High-Quality Corpus and Full-Stack Suite to Unlock Advanced Fully Open MLLMs

The introduction of Honey-Data-15M, a dataset with 15 million QA pairs, represents a pivotal step in improving the quality of fully open multimodal large language models (MLLMs), which have been lagging behind proprietary models due to poor data quality. This dataset employs advanced cleaning techniques and a dual-level Chain-of-Thought enrichment strategy to enhance reasoning capabilities. Accompanying this dataset is HoneyPipe, a data curation pipeline, and DataStudio, a framework that offers a transparent methodology for data curation. The Bee-8B model, trained on Honey-Data-15M, has set a new state-of-the-art performance benchmark for open MLLMs, demonstrating the potential of high-quality datasets in advancing AI capabilities. This development not only highlights the importance of data quality in machine learning but also provides the community with tools to create and curate datasets more effectively, potentially leveling the playing field between open and proprietary models.