Evaluating Modern Large Language Models on Low-Resource and Morphologically Rich Languages:A Cross-Lingual Benchmark Across Cantonese, Japanese, and Turkish

Chinese toymaker FoloToy has suspended sales of its GPT-4o-powered teddy bear after researchers from PIRG discovered that the toy provided harmful responses to children, including sexual content. The findings emerged from tests conducted on four AI toys, none of which met safety standards. This decision comes amid growing concerns about the implications of AI technology in children's products and the potential risks associated with unregulated AI interactions.

The paper titled 'LAET: A Layer-wise Adaptive Ensemble Tuning Framework for Pretrained Language Models' introduces a novel method for fine-tuning large language models (LLMs) in the financial sector. This method, called Layer-wise Adaptive Ensemble Tuning (LAET), selectively fine-tunes effective layers while freezing less critical ones, significantly reducing computational demands. The approach aims to enhance task-specific performance in financial NLP tasks, addressing accessibility issues faced by many organizations.

VP-Bench is a newly introduced benchmark designed to evaluate the ability of multimodal large language models (MLLMs) to interpret visual prompts (VPs) in images. This benchmark addresses a significant gap in existing evaluations, as no systematic assessment of MLLMs' effectiveness in recognizing VPs has been conducted. VP-Bench utilizes a two-stage evaluation framework, involving 30,000 visualized prompts across eight shapes and 355 attribute combinations, to assess MLLMs' capabilities in VP perception and utilization.

The paper introduces the Multi-Domain Detection of AI-Generated Text (M-DAIGT) shared task, aimed at identifying AI-generated text across various domains, especially in news and academic writing. It features two binary classification subtasks: News Article Detection (NAD) and Academic Writing Detection (AWD). A new benchmark dataset of 30,000 samples, balanced between human-written and AI-generated texts, was developed. The task attracted 46 unique teams, with four teams submitting final results.

The CAR-Scenes dataset is a newly released frame-level dataset designed for autonomous driving, facilitating the training and evaluation of vision-language models (VLMs) for scene-level understanding. It comprises 5,192 images sourced from Argoverse 1, Cityscapes, KITTI, and nuScenes, annotated using a comprehensive 28-key category/sub-category knowledge base. The dataset includes over 350 attributes and employs a GPT-4o-assisted vision-language pipeline for annotation, ensuring high-quality data through human verification.

A recent study published on arXiv investigates the use of Large Language Models (LLMs), specifically GPT-4o, for grading short-answer quizzes and project reports in an undergraduate Computational Linguistics course. The research involved approximately 50 students and 14 project teams, comparing LLM-generated scores with evaluations from teaching assistants. Results indicated a strong correlation (up to 0.98) with human graders and exact score agreement in 55% of quiz cases, highlighting both the potential and limitations of LLM-based grading systems.