Chinese toymaker FoloToy suspends sales of its GPT-4o-powered teddy bear, after researchers found the toy gave kids harmful responses, including sexual content (Brandon Vigliarolo/The Register)

A recent study evaluates the performance of seven advanced large language models (LLMs) on low-resource and morphologically rich languages, specifically Cantonese, Japanese, and Turkish. The research highlights the models' effectiveness in tasks such as open-domain question answering, document summarization, translation, and culturally grounded dialogue. Despite impressive results in high-resource languages, the study indicates that the effectiveness of LLMs in these less-studied languages remains underexplored.

VP-Bench is a newly introduced benchmark designed to evaluate the ability of multimodal large language models (MLLMs) to interpret visual prompts (VPs) in images. This benchmark addresses a significant gap in existing evaluations, as no systematic assessment of MLLMs' effectiveness in recognizing VPs has been conducted. VP-Bench utilizes a two-stage evaluation framework, involving 30,000 visualized prompts across eight shapes and 355 attribute combinations, to assess MLLMs' capabilities in VP perception and utilization.

The CAR-Scenes dataset is a newly released frame-level dataset designed for autonomous driving, facilitating the training and evaluation of vision-language models (VLMs) for scene-level understanding. It comprises 5,192 images sourced from Argoverse 1, Cityscapes, KITTI, and nuScenes, annotated using a comprehensive 28-key category/sub-category knowledge base. The dataset includes over 350 attributes and employs a GPT-4o-assisted vision-language pipeline for annotation, ensuring high-quality data through human verification.

A recent study published on arXiv investigates the use of Large Language Models (LLMs), specifically GPT-4o, for grading short-answer quizzes and project reports in an undergraduate Computational Linguistics course. The research involved approximately 50 students and 14 project teams, comparing LLM-generated scores with evaluations from teaching assistants. Results indicated a strong correlation (up to 0.98) with human graders and exact score agreement in 55% of quiz cases, highlighting both the potential and limitations of LLM-based grading systems.