New augmented reality tech can turn any surface into keyboard

Wave-Former is a new method for high-accuracy 3D shape reconstruction of completely occluded everyday objects. Utilizing millimeter-wave (mmWave) wireless signals, it can penetrate common obstructions and reflect off hidden items. Unlike previous methods that faced limitations in coverage and noise, Wave-Former employs a physics-aware shape completion model to infer full 3D geometry. Its innovative three-stage pipeline connects raw wireless signals with advancements in vision-based shape completion, enhancing applications in robotics, augmented reality, and logistics.

The article presents Uni-Hand, a universal hand motion forecasting framework designed for egocentric views. This framework addresses challenges in hand trajectory prediction methods, such as insufficient prediction targets and entangled hand-head motion. By utilizing multi-modal inputs and incorporating vision-language fusion, it aims to enhance applications in augmented reality and human-robot interaction. The framework forecasts hand waypoints in both 2D and 3D spaces, improving the accuracy of motion predictions.

The Geometry-guided Multi-View Diffusion Model (GeoMVD) has been proposed to enhance multi-view image generation, addressing challenges in maintaining cross-view consistency and producing high-resolution outputs. This model utilizes geometric information extraction techniques, including depth maps and normal maps, to create images that are structurally consistent and rich in detail. The advancements in this model hold significant implications for applications in computer vision, such as 3D reconstruction and augmented reality.

MixAR is a new framework introduced to enhance image generation through autoregressive (AR) modeling. Traditional AR approaches, which utilize discrete tokens from a limited codebook, often lose fine-grained details due to quantization. Recent advancements have shifted towards continuous latent spaces for improved quality, but these spaces present challenges for efficient modeling. MixAR addresses these issues by integrating discrete tokens as prior guidance, facilitating better continuous AR modeling and potentially leading to higher fidelity in generated images.

The Segment Anything Model (SAM) has gained popularity as a vision foundation model, but it struggles with controlling segmentation granularity, often requiring manual refinement by users. To overcome this challenge, UnSAMv2 has been introduced, allowing segmentation at any granularity without human annotations. This model builds on the divide-and-conquer strategy of its predecessor, UnSAM, by identifying numerous mask-granularity pairs and implementing a new granularity control embedding for precise segmentation scale management. The model demonstrates effectiveness with only 6,000 unlabeled …

TEyeD is the world's largest unified public dataset of eye images, featuring over 20 million images collected using seven different head-mounted eye trackers, including devices integrated into virtual and augmented reality systems. The dataset encompasses a variety of activities, such as car rides and sports, and includes detailed annotations like 2D and 3D landmarks, semantic segmentation, and gaze vectors. This resource aims to enhance research in computer vision, eye tracking, and gaze estimation.

An AI-assisted Augmented Reality (AR) assembly workflow has been developed, utilizing deep learning-based object recognition to identify assembly components and provide step-by-step instructions. The system displays bounding boxes around components in real-time, indicating their placement, thus eliminating the need for manual searching or sorting. A case study involving the assembly of LEGO sculptures demonstrates the system's feasibility and effectiveness in enhancing the assembly process.