Dejavu: Towards Experience Feedback Learning for Embodied Intelligence

A new reinforcement learning training environment, RLCAD, has been developed to facilitate the automatic generation of CAD command sequences, enhancing the design process in 3D CAD systems. This environment utilizes a policy network to generate actions based on input boundary representations, ultimately producing complex CAD geometries.

A new framework called Vision-Language Distance (VLD) has been introduced to enhance goal-conditioned navigation in robotic systems. This approach separates perception learning from policy learning, utilizing a self-supervised distance-to-goal predictor trained on extensive video data to improve navigation actions directly from image inputs.

A recent study has introduced a unified framework for applying value-based reinforcement learning (RL) to combinatorial optimization (CO) problems, utilizing Markov decision processes (MDPs) to enhance the training of neural networks as learned heuristics. This approach aims to reduce the reliance on expert-designed heuristics, potentially transforming how CO problems are addressed in various fields.

A new method for the direct transfer of optimized controllers to similar systems using dimensionless model predictive control (MPC) has been proposed, allowing for automatic tuning of closed-loop performance. This approach enhances the applicability of scaled model experiments in engineering by facilitating the transfer of controller behavior from scaled models to full-scale systems without the need for extensive retuning.

A new study has introduced a reinforcement learning-based framework for the automated construction of artificial lattice structures using a scanning tunneling microscope (STM). This method allows for the precise manipulation of carbon monoxide molecules on a copper substrate, significantly enhancing the efficiency and scale of creating atomically defined structures with designer electronic states.

A new class of methods for reinforcement learning (RL) has been introduced, focusing on auto-exploration to address the exploration-exploitation dilemma. These methods allow for parameter-free exploration of both state and action spaces, aiming to improve sample complexity and performance in RL algorithms.

A new wildfire simulator named JaxWildfire has been introduced, utilizing a probabilistic fire spread model based on cellular automata and implemented in JAX. This simulator significantly accelerates the training of reinforcement learning (RL) agents by achieving a speedup of 6-35 times compared to existing software, enabling more efficient simulations on GPUs.

VideoVLA has been introduced as a novel approach that transforms large video generation models into generalizable robotic manipulators, enhancing their ability to predict action sequences and future visual outcomes based on language instructions and images. This advancement is built on a multi-modal Diffusion Transformer, which integrates video, language, and action modalities for improved forecasting.