UnSAMv2: Self-Supervised Learning Enables Segment Anything at Any Granularity

Virtual keyboards in augmented reality (AR) often frustrate users due to their slow response and high error rates. Users experience discomfort, commonly referred to as 'gorilla arm,' from raising their arms to type on these virtual surfaces.

Deep neural networks excel in visual recognition tasks but their large parameter counts hinder practical applications. One-shot pruning has emerged as a solution to reduce model size without retraining. However, aggressive pruning often leads to significant accuracy drops. Existing optimizers like SAM and CrAM help mitigate this issue but require additional computations. The proposed Variance Amplifying Regularizer (VAR) increases parameter variance during training, enhancing pruning robustness and maintaining accuracy.

SMOL-MapSeg, a new segmentation model, addresses the challenges posed by historical maps in modern segmentation tasks. Traditional deep learning models like UNet and Segment Anything Model (SAM) struggle with the variability in visual styles and symbols found in these maps. The proposed On-Need Declarative (OND) prompting method allows users to provide explicit image-label pair prompts, facilitating flexible and concept-aware segmentation tailored to historical map analysis.

LENS is a new reinforcement-learning framework designed for text-prompted image segmentation, enhancing visual understanding crucial for applications in human-computer interaction and robotics. Unlike traditional supervised methods, LENS incorporates explicit chain-of-thought reasoning during testing, improving generalization to unseen prompts. By utilizing a 3-billion-parameter vision-language model, LENS achieves an average cIoU of 81.2% on benchmark datasets, surpassing existing fine-tuning methods.

MixAR is a new framework introduced to enhance image generation through autoregressive (AR) modeling. Traditional AR approaches, which utilize discrete tokens from a limited codebook, often lose fine-grained details due to quantization. Recent advancements have shifted towards continuous latent spaces for improved quality, but these spaces present challenges for efficient modeling. MixAR addresses these issues by integrating discrete tokens as prior guidance, facilitating better continuous AR modeling and potentially leading to higher fidelity in generated images.