arXiv:2511.09114v2 Announce Type: replace 
Abstract: Recent advances in deep reinforcement learning for autonomous cyber defence have resulted in agents that can successfully defend simulated computer networks against cyber-attacks. However, many of these agents would need retraining to defend networks with differing topology or size, making them poorly suited to real-world networks where topology and size can vary over time. In this research we introduce a novel set of Topological Extensions for Reinforcement Learning Agents (TERLA) that provide generalisability for the defence of networks with differing topology and size, without the need for retraining. Our approach involves the use of heterogeneous graph neural network layers to produce a fixed-size latent embedding representing the observed network state. This representation learning stage is coupled with a reduced, fixed-size, semantically meaningful and interpretable action space. We apply TERLA to a standard deep reinforcement learning Proximal Policy Optimisation (PPO) agent model, and to reduce the sim-to-real gap, conduct our research using Cyber Autonomy Gym for Experimentation (CAGE) Challenge 4. This Cyber Operations Research Gym environment has many of the features of a real-world network, such as realistic Intrusion Detection System (IDS) events and multiple agents defending network segments of differing topology and size. TERLA agents retain the defensive performance of vanilla PPO agents whilst showing improved action efficiency. Generalisability has been demonstrated by showing that all TERLA agents have the same network-agnostic neural network architecture, and by deploying a single TERLA agent multiple times to defend network segments with differing topology and size, showing improved defensive performance and efficiency.

قدمت الأبحاث الحديثة نهجًا جديدًا يسمى التمديدات الطوبولوجية لوكلاء التعلم المعزز (TERLA)، والذي يعزز القابلية للتعميم لوكلاء الدفاع السيبراني لشبكات الكمبيوتر في العالم الحقيقي. تتيح هذه الطريقة للوكلاء الدفاع عن الشبكات ذات الطوبولوجيا والأحجام المتنوعة دون الحاجة إلى إعادة التدريب، باستخدام طبقات من الشبكات العصبية الرسومية غير المتجانسة لإنشاء تمثيل كامن ثابت الحجم لحالة الشبكة.

Investigaciones recientes han presentado un nuevo enfoque llamado Extensiones Topológicas para Agentes de Aprendizaje por Refuerzo (TERLA), que mejora la generalización de los agentes de defensa cibernética para redes informáticas del mundo real. Este método permite a los agentes defender redes con topologías y tamaños variables sin necesidad de reentrenamiento, utilizando capas de redes neuronales gráficas heterogéneas para crear una representación latente de tamaño fijo del estado de la red.

Des recherches récentes ont introduit une nouvelle approche appelée Extensions Topologiques pour Agents d'Apprentissage par Renforcement (TERLA), qui améliore la généralisabilité des agents de défense cybernétique pour les réseaux informatiques du monde réel. Cette méthode permet aux agents de défendre des réseaux avec des topologies et des tailles variées sans nécessiter de réentraînement, en utilisant des couches de réseaux de neurones graphiques hétérogènes pour créer un embedding latent de taille fixe de l'état du réseau.

Recent research has introduced a novel approach called Topological Extensions for Reinforcement Learning Agents (TERLA), which enhances the generalisability of cyber defence agents for real-world computer networks. This method allows agents to defend networks with varying topology and size without the need for retraining, utilizing heterogeneous graph neural network layers to create a fixed-size latent embedding of the network state.

Towards a Generalisable Cyber Defence Agent for Real-World Computer Networks

One More Thing in AI – Your Shortcut to AI Mastery

Towards a Generalisable Cyber Defence Agent for Real-World Computer Networks

Was this article worth reading? Share it

One More Thing in AI

LucidQuery AI

Cato Networks

Chattermate

Legion AI

Dyad

Ready to build your own newsroom?