Physical Reinforcement Learning

A recent study has framed the self-healing process of material systems as a Reinforcement Learning (RL) problem within a Markov Decision Process (MDP), demonstrating that RL agents can autonomously derive optimal policies for maintaining structural integrity while managing resource consumption. The research highlighted the superior performance of continuous-action agents, particularly the TD3 agent, in achieving near-complete material recovery compared to traditional heuristic methods.

The introduction of the Non-stationary and Varying-discounting Markov Decision Processes (NVMDP) framework addresses the limitations faced by traditional stationary Markov Decision Processes (MDPs) in non-stationary environments. This framework allows for varying discount rates over time and transitions, making it applicable to both finite and infinite-horizon tasks.

A new refinement in temporal-difference learning has been proposed, emphasizing first-order Bellman consistency. This approach trains the learned value function to align with both the Bellman targets and their derivatives, enhancing the stability and convergence of reinforcement learning algorithms like Q-learning and actor-critic methods.

A novel Q-learning framework has been proposed for time-critical data aggregation scheduling in Internet of Things (IoT) networks, aiming to reduce latency in applications such as smart cities and industrial automation. This approach integrates aggregation tree construction and scheduling into a unified model, enhancing efficiency and scalability.