While recognizing actions, LMMs struggle to detect core interaction events
NeutralArtificial Intelligence
- Large multi-modal models (LMMs) have shown improved performance in visual tasks, particularly in analyzing video sequences. A recent study evaluated their ability to detect core interaction events, such as when hands contact or release objects, using a new dataset with over 20,000 annotated interactions from the Something-Something-V2 dataset.
- This development is significant as it highlights the limitations of current LMMs, such as Qwen-2.5VL and GPT-4o, in accurately identifying the start and end of interactions, which is crucial for enhancing their semantic understanding and practical applications in various fields.
- The challenges faced by LMMs in detecting interaction events reflect broader concerns in the AI community regarding the reliability of visual language models. Issues such as hallucinations, stability under varying inputs, and the need for improved contextual understanding are ongoing discussions that impact the advancement of AI technologies.
— via World Pulse Now AI Editorial System
