Downscaling Intelligence: Exploring Perception and Reasoning Bottlenecks in Small Multimodal Models

A new algorithm named BiFingerPose has been introduced for finger pose estimation on touchscreen devices, utilizing a bimodal approach that combines capacitive images and fingerprint patches from under-screen sensors. This method enhances the accuracy of estimating various finger pose parameters, particularly roll angles, which were previously challenging to assess accurately.

A new approach called Reason2Attack (R2A) has been proposed to enhance the reasoning capabilities of large language models (LLMs) in generating adversarial prompts for text-to-image (T2I) models. This method addresses the limitations of existing jailbreaking techniques that require numerous queries to bypass safety filters, thereby exposing vulnerabilities in T2I systems. R2A incorporates jailbreaking into the post-training process of LLMs, aiming to streamline the attack process.

A new study has introduced a motion transfer-enhanced StyleGAN2 model aimed at generating diverse facial expressions in macaque monkeys, addressing the challenge of limited training images for animal faces. This method utilizes data augmentation techniques to synthesize new images and refines loss functions to capture subtle movements accurately.

The PairHuman dataset has been introduced as a pioneering benchmark for generating high-fidelity dual-person portraits, comprising over 100,000 images that encompass diverse scenes and interactions. This dataset aims to enhance personalized portrait customization, which is crucial for applications like wedding photography and emotional memory preservation.

A new framework named SVG360 has been introduced, enabling the generation of multi-view Scalable Vector Graphics (SVGs) with geometric and color consistency from a single SVG input. This process involves lifting the rasterized input to a 3D representation, establishing part-level correspondences across views, and optimizing vector paths during conversion.

WorldGen has been introduced as a groundbreaking system that automates the creation of expansive, interactive 3D worlds from text prompts, transforming natural language into fully textured environments ready for exploration or editing in game engines.

The introduction of Mesh RAG, a novel framework for autoregressive mesh generation, aims to enhance the efficiency and quality of 3D mesh creation, which is crucial for various applications including gaming and robotics. This approach leverages point cloud segmentation and spatial transformations to improve the generation process without the need for extensive training.

MolSight has been introduced as a novel framework for Optical Chemical Structure Recognition (OCSR), addressing the challenges of accurately interpreting stereochemical information from chemical structure images. This system employs a three-stage training approach, enhancing the model's ability to convert visual data into machine-readable formats essential for chemical informatics.