Value Improved Actor Critic Algorithms

A new model-based algorithm named BLINQ has been introduced, which learns the Whittle indices of an indexable, communicating, and unichain Markov Decision Process (MDP). This approach builds an empirical estimate of the MDP and computes its Whittle indices using an enhanced version of an existing algorithm, demonstrating convergence and computational efficiency.

Recent advancements in Contrastive Local Learning Networks (CLLNs) have demonstrated their potential for reinforcement learning (RL) applications, particularly in energy-limited environments. This study successfully applied Q-learning techniques to simulated CLLNs, showcasing their robustness and low power consumption compared to traditional digital systems.

A new reinforcement learning platform named ContagionRL has been introduced, designed for reward engineering in spatial epidemic simulations. This platform allows researchers to evaluate how different reward function designs influence survival strategies in various epidemic scenarios, integrating a spatial SIRS+D epidemiological model with adjustable environmental parameters.

A recent study has framed the self-healing process of material systems as a Reinforcement Learning (RL) problem within a Markov Decision Process (MDP), demonstrating that RL agents can autonomously derive optimal policies for maintaining structural integrity while managing resource consumption. The research highlighted the superior performance of continuous-action agents, particularly the TD3 agent, in achieving near-complete material recovery compared to traditional heuristic methods.

The introduction of the Non-stationary and Varying-discounting Markov Decision Processes (NVMDP) framework addresses the limitations faced by traditional stationary Markov Decision Processes (MDPs) in non-stationary environments. This framework allows for varying discount rates over time and transitions, making it applicable to both finite and infinite-horizon tasks.

A new framework called Multi-Objective Multi-Agent Actor-Critic (MOMA-AC) has been introduced to address gaps in Multi-Objective Multi-Agent Reinforcement Learning (MOMARL). This framework utilizes Twin Delayed Deep Deterministic Policy Gradient (TD3) and Deep Deterministic Policy Gradient (DDPG) algorithms, featuring a multi-headed actor network and a centralized critic to optimize trade-off policies across conflicting objectives in continuous environments.

A new refinement in temporal-difference learning has been proposed, emphasizing first-order Bellman consistency. This approach trains the learned value function to align with both the Bellman targets and their derivatives, enhancing the stability and convergence of reinforcement learning algorithms like Q-learning and actor-critic methods.

A novel Q-learning framework has been proposed for time-critical data aggregation scheduling in Internet of Things (IoT) networks, aiming to reduce latency in applications such as smart cities and industrial automation. This approach integrates aggregation tree construction and scheduling into a unified model, enhancing efficiency and scalability.