arXiv:2503.04763v2 Announce Type: replace-cross 
Abstract: In this work, we conduct an experiment using state-of-the-art LLMs to translate MiniF2F into Rocq. The translation task focuses on generating a Rocq theorem based on three sources: a natural language description, the Lean formalization, and the Isabelle formalization. We conducted our experiment in 3 stages of increasing complexity, from basic one-shot prompting to multi-turn conversations that incorporate feedback from unsuccessful attempts. At each stage, we perform multiple rounds of translation using increasingly advanced models: GPT-4o mini, Claude 3.5 Sonnet, o1 mini, and o1. We successfully translated 478 out of 488 theorems. The dataset is opensource: https://github.com/LLM4Rocq/miniF2F-rocq.

تم إجراء تجربة لترجمة MiniF2F إلى Rocq باستخدام نماذج لغوية متقدمة، حيث تم النجاح في ترجمة 478 من أصل 488 نظرية بناءً على أوصاف باللغة الطبيعية وتشكيلات رسمية من Lean وIsabelle. شمل هذا العملية ثلاث مراحل من التعقيد المتزايد، باستخدام نماذج مثل GPT-4o mini وClaude 3.5 Sonnet.

Se realizó un experimento para traducir MiniF2F a Rocq utilizando modelos de lenguaje avanzados, logrando traducir 478 de 488 teoremas basados en descripciones en lenguaje natural y formalizaciones de Lean e Isabelle. El proceso involucró tres etapas de complejidad creciente, utilizando modelos como GPT-4o mini y Claude 3.5 Sonnet.

Une expérience a été menée pour traduire MiniF2F en Rocq en utilisant des modèles de langage avancés, réussissant à traduire 478 des 488 théorèmes basés sur des descriptions en langage naturel et des formalismes de Lean et Isabelle. Le processus a impliqué trois étapes de complexité croissante, utilisant des modèles tels que GPT-4o mini et Claude 3.5 Sonnet.

An experiment was conducted to translate MiniF2F into Rocq using advanced language models, successfully translating 478 out of 488 theorems based on natural language descriptions and formalizations from Lean and Isabelle. The process involved three stages of increasing complexity, utilizing models such as GPT-4o mini and Claude 3.5 Sonnet.

MiniF2F in Rocq: Automatic Translation Between Proof Assistants -- A Case Study

Was this article worth reading? Share it

LucidQuery AI

Langfuse

OpenL Translator

AQ

Bocca

Doc2lang

Ready to build your own newsroom?