arXiv:2510.22443v1 Announce Type: new 
Abstract: There has been a surge of interest in assistive wearable agents: agents embodied in wearable form factors (e.g., smart glasses) who take assistive actions toward a user's goal/query (e.g. "Where did I leave my keys?"). In this work, we consider the important complementary problem of inferring that goal from multi-modal contextual observations. Solving this "goal inference" problem holds the promise of eliminating the effort needed to interact with such an agent. This work focuses on creating WAGIBench, a strong benchmark to measure progress in solving this problem using vision-language models (VLMs). Given the limited prior work in this area, we collected a novel dataset comprising 29 hours of multimodal data from 348 participants across 3,477 recordings, featuring ground-truth goals alongside accompanying visual, audio, digital, and longitudinal contextual observations. We validate that human performance exceeds model performance, achieving 93% multiple-choice accuracy compared with 84% for the best-performing VLM. Generative benchmark results that evaluate several families of modern vision-language models show that larger models perform significantly better on the task, yet remain far from practical usefulness, as they produce relevant goals only 55% of the time. Through a modality ablation, we show that models benefit from extra information in relevant modalities with minimal performance degradation from irrelevant modalities.

تسلط دراسة جديدة الضوء على الاهتمام المتزايد بالوكلاء القابلين للارتداء للمساعدة، مثل النظارات الذكية، التي تساعد المستخدمين في تحقيق أهدافهم من خلال استنتاج احتياجاتهم من خلال مجموعة متنوعة من الإشارات السياقية. هذه الأبحاث مهمة لأنها تهدف إلى تبسيط تفاعلات المستخدمين مع التكنولوجيا، مما يجعلها أكثر سهولة وكفاءة. من خلال التركيز على استنتاج الأهداف، تعد الدراسة بتحسين وظائف هذه الأجهزة، مما قد يحول طريقة تفاعلنا مع بيئتنا وإدارة المهام اليومية.

Un nuevo estudio destaca el creciente interés en los agentes portátiles de asistencia, como las gafas inteligentes, que ayudan a los usuarios a alcanzar sus objetivos inferiendo sus necesidades a partir de diversas pistas contextuales. Esta investigación es significativa porque busca simplificar las interacciones del usuario con la tecnología, haciéndola más intuitiva y eficiente. Al centrarse en la inferencia de objetivos, el estudio promete mejorar la funcionalidad de estos dispositivos, transformando potencialmente nuestra interacción con el entorno y la gestión de las tareas diarias.

Une nouvelle étude met en lumière l'intérêt croissant pour les agents portables d'assistance, comme les lunettes intelligentes, qui aident les utilisateurs à atteindre leurs objectifs en inférant leurs besoins à partir de divers indices contextuels. Cette recherche est importante car elle vise à simplifier les interactions des utilisateurs avec la technologie, la rendant plus intuitive et efficace. En se concentrant sur l'inférence des objectifs, l'étude promet d'améliorer la fonctionnalité de ces dispositifs, transformant potentiellement notre interaction avec notre environnement et la gestion des tâches quotidiennes.

A new study highlights the growing interest in assistive wearable agents, like smart glasses, which help users achieve their goals by inferring their needs from various contextual cues. This research is significant because it aims to simplify user interactions with technology, making it more intuitive and efficient. By focusing on goal inference, the study promises to enhance the functionality of these devices, potentially transforming how we interact with our environment and manage daily tasks.

Benchmarking Egocentric Multimodal Goal Inference for Assistive Wearable Agents

arXiv:2512.03287v1 Announce Type: new 
Abstract: Human Activity Recognition (HAR) benefits various application domains, including health and elderly care. Traditional HAR involves constructing pipelines reliant on centralized user data, which can pose privacy concerns as they necessitate the uploading of user data to a centralized server. This work proposes multi-frequency Federated Learning (FL) to enable: (1) privacy-aware ML; (2) joint ML model learning across devices with varying sampling frequency. We focus on head-worn devices (e.g., earbuds and smart glasses), a relatively unexplored domain compared to traditional smartwatch- or smartphone-based HAR. Results have shown improvements on two datasets against frequency-specific approaches, indicating a promising future in the multi-frequency FL-HAR task. The proposed network's implementation is publicly available for further research and development.

تقدم دراسة جديدة التعلم الفيدرالي متعدد الترددات (FL) للتعرف على الأنشطة البشرية (HAR) باستخدام أجهزة استشعار ترتدي على الرأس مثل سماعات الأذن والنظارات الذكية. تتناول هذه الطريقة المخاوف المتعلقة بالخصوصية المرتبطة بجمع البيانات المركزي من خلال تمكين تدريب النماذج اللامركزية عبر الأجهزة ذات ترددات أخذ عينات متباينة.

Un nuevo estudio presenta el aprendizaje federado multi-frecuencia (FL) para el reconocimiento de actividades humanas (HAR) utilizando sensores portados en la cabeza, como auriculares y gafas inteligentes. Este enfoque aborda las preocupaciones de privacidad asociadas con la recopilación de datos centralizada al permitir el entrenamiento de modelos descentralizados entre dispositivos con diferentes frecuencias de muestreo.

Une nouvelle étude présente l'apprentissage fédéré multi-fréquence (FL) pour la reconnaissance d'activités humaines (HAR) utilisant des capteurs portés sur la tête comme des écouteurs et des lunettes intelligentes. Cette approche répond aux préoccupations en matière de confidentialité liées à la collecte de données centralisée en permettant un entraînement de modèle décentralisé entre des appareils ayant des fréquences d'échantillonnage variées.

A new study introduces multi-frequency Federated Learning (FL) for Human Activity Recognition (HAR) using head-worn sensors like earbuds and smart glasses. This approach addresses privacy concerns associated with centralized data collection by enabling decentralized model training across devices with varying sampling frequencies.

Benchmarking Egocentric Multimodal Goal Inference for Assistive Wearable Agents

Was this article worth reading? Share it

LucidQuery AI

Augmeta

Guidejar-4eb95b