arXiv:2605.26947v1 Announce Type: new 
Abstract: Kazakh is underrepresented in resources for evaluating the safety behavior of large language models. We present KZ-SafetyPrompts, a Kazakh prompt dataset for safety evaluation across eleven categories covering common risk areas such as self-harm, violence, child exploitation, sexual content, racist content, radicalization, and regulated goods or illegal activities. The dataset contains 5,717 prompts written natively in Kazakh (Cyrillic), organized by category, with English translations for cross-lingual analysis. Prompts resemble realistic user queries, often in a teen or child style, and are phrased as intent prompts without procedural instructions. We document the writing protocol, labeling procedures (including borderline-case decision rules), and quality-control steps (schema standardization, completeness checks, and deduplication). We also align the categories with widely used safety taxonomies to support integration with existing evaluation pipelines. Baseline results with GPT-4o show an overall refusal rate of 28.2%, varying from 5.5% to 53.8% across categories, indicating that Kazakh prompts expose category-specific safety gaps not captured by English-only evaluation.

يمثل تقديم KZ-SafetyPrompts تقدمًا كبيرًا في تقييم سلوكيات الأمان في نماذج اللغة الكبيرة، وخاصة للغة الكازاخية التي كانت تاريخيًا تحت التمثيل في هذا المجال. يتكون هذا المجموع من 5,717 من المحفزات المصنفة إلى أحد عشر مجالًا من المخاطر، بما في ذلك إيذاء النفس والعنف، مما يوفر موردًا حيويًا للباحثين والمطورين.

La introducción de KZ-SafetyPrompts marca un avance significativo en la evaluación del comportamiento de seguridad de los grandes modelos de lenguaje, específicamente para el idioma kazajo, que ha estado históricamente subrepresentado en esta área. Este conjunto de datos consta de 5,717 indicaciones categorizadas en once áreas de riesgo, incluyendo autolesiones y violencia, proporcionando un recurso crucial para investigadores y desarrolladores.

L'introduction de KZ-SafetyPrompts représente une avancée significative dans l'évaluation des comportements de sécurité des grands modèles de langage, en particulier pour la langue kazakhe, qui a été historiquement sous-représentée dans ce domaine. Cet ensemble de données comprend 5 717 invites classées en onze domaines de risque, y compris l'automutilation et la violence, fournissant une ressource cruciale pour les chercheurs et les développeurs.

The introduction of KZ-SafetyPrompts marks a significant advancement in the evaluation of safety behaviors in large language models, specifically for the Kazakh language, which has been historically underrepresented in this area. This dataset consists of 5,717 prompts categorized into eleven risk areas, including self-harm and violence, providing a crucial resource for researchers and developers.