Optimism as Risk-Seeking in Multi-Agent Reinforcement Learning

The recent study on optimism as risk-seeking in multi-agent reinforcement learning (MARL) highlights a significant shift from traditional risk-averse strategies, which have dominated the field. While these conservative approaches prioritize robustness, they often result in suboptimal equilibria. The proposed framework interprets risk-seeking objectives as optimism, introducing optimistic value functions that formalize this concept through divergence-penalized evaluations. This theoretical grounding is crucial, as existing methods lacked such rigor. The study also derives a policy-gradient theorem for these optimistic value functions and develops decentralized actor-critic algorithms to implement the findings. Empirical results on cooperative benchmarks reveal that adopting risk-seeking optimism consistently enhances coordination among agents, outperforming both risk-neutral baselines and heuristic optimistic methods. This advancement not only enriches the theoretical landscape of MARL …