arXiv:2606.13275v1 Announce Type: new 
Abstract: This paper presents Custom ZeroCLIP, a retrieval-augmented vision-language framework for zero-shot captioning of Indonesian traditional garments. The dataset contains 3,800 expert-annotated images from all 38 Indonesian provinces. Using a province-level inductive zero-shot protocol, the model is trained on 24 seen provinces, validated on 6 seen provinces, and evaluated on 8 unseen provinces. The framework combines a frozen CLIP ViT-B/32 image encoder, a CLIP text encoder, a BERT text encoder, and an LSTM caption decoder. During inference, unseen-province labels and captions are unavailable, and retrieval uses only captions from training provinces. No unseen-province image, label, or caption is used during training, validation, or retrieval-bank construction. Custom ZeroCLIP achieves a CLIPScore of 0.8536, BLEU-4 of 0.3342, and METEOR of 0.4859, outperforming existing baselines. Ablation results show that retrieval improves cultural vocabulary recovery with a 19.3\% METEOR gain, while human evaluation confirms stronger cultural accuracy and fluency. The results demonstrate the effectiveness of retrieval-augmented domain adaptation for culturally grounded caption generation in low-resource heritage settings. The dataset is publicly available at https://github.com/AnugrahAidinYotolembah/Traditional-Indonesian-Clothing-Captioning-Dataset.

تم تطوير إطار جديد يسمى Custom ZeroCLIP لتوليد تسميات بدون تدريب مسبق للملابس التقليدية الإندونيسية، باستخدام مجموعة بيانات تحتوي على 3,800 صورة تم وضع علامات عليها من قبل خبراء من جميع المحافظات الإندونيسية الـ 38. يستخدم النموذج مزيجًا من التقنيات المتقدمة، بما في ذلك مشفر صورة CLIP ViT-B/32 المجمد ومشفر نص BERT، محققًا مقاييس أداء ملحوظة مثل CLIPScore بقيمة 0.8536 وBLEU-4 بقيمة 0.3342.

Se ha desarrollado un nuevo marco llamado Custom ZeroCLIP para la creación de leyendas en cero disparos de prendas tradicionales indonesias, utilizando un conjunto de datos de 3,800 imágenes anotadas por expertos de las 38 provincias indonesias. El modelo emplea una combinación de tecnologías avanzadas, incluyendo un codificador de imagen CLIP ViT-B/32 congelado y un codificador de texto BERT, logrando métricas de rendimiento notables como un CLIPScore de 0.8536 y un BLEU-4 de 0.3342.

Un nouveau cadre appelé Custom ZeroCLIP a été développé pour la légende zéro-shot des vêtements traditionnels indonésiens, utilisant un ensemble de données de 3 800 images annotées par des experts provenant des 38 provinces indonésiennes. Le modèle emploie une combinaison de technologies avancées, y compris un encodeur d'image CLIP ViT-B/32 gelé et un encodeur de texte BERT, atteignant des métriques de performance notables telles qu'un CLIPScore de 0,8536 et un BLEU-4 de 0,3342.

A new framework called Custom ZeroCLIP has been developed for zero-shot captioning of Indonesian traditional garments, utilizing a dataset of 3,800 expert-annotated images from all 38 Indonesian provinces. The model employs a combination of advanced technologies, including a frozen CLIP ViT-B/32 image encoder and a BERT text encoder, achieving notable performance metrics such as a CLIPScore of 0.8536 and BLEU-4 of 0.3342.

Zero-Shot Captioning for Cultural Heritage: Automated Image Analysis of Traditional Indonesian Clothing

Useful picks

SwapAnything.io

Uwear

OpenL Translator

Blunge

VibeFrame

Aitryon-art