Awesome Reinforcement Learning for Cyber Security

A curated list of resources dedicated to reinforcement learning applied to cyber security. Note that the list includes only work that uses reinforcement learning, general machine learning methods applied to cyber security are not included in this list.

For other related curated lists, see :

• Awesome Machine Learning for Cyber Security
• Awesome Adversarial Machine Learning

We are looking for more contributors and maintainers! Contact [email protected] if you are interested to become a maintainer.

Maintainers/Contributors:

• Kim Hammar, KTH Royal Institute of Techhology, Sweden.
• Lisa Oakley, Northeastern University, USA.
• Manuel Del Verme, Quebec Artificial Intelligence Institute, Canada.
• Pontus Johnson, KTH Royal Institute of Techhology, Sweden.
• Jakob Nyberg, KTH Royal Institute of Techhology, Sweden.
• Simon Gökstorp, KTH Royal Institute of Techhology, Sweden.
• Fabio Massimo Zennaro, University of Oslo, Norway.

Table of Contents

• RL-Environments
• Papers
• Books
• Blogposts
• Talks
• Miscellaneous

↑ Environments

gym-idsgame

gym-idsgame An Abstract Cyber Security Simulation and Markov Game for OpenAI Gym. Paper: (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play

CyberBattleSim (Microsoft)

CyberBattleSim CyberBattleSim is an experimentation research platform to investigate the interaction of automated agents operating in a simulated abstract enterprise network environment. The simulation provides a high-level abstraction of computer networks and cyber security concepts. Its Python-based Open AI Gym interface allows for the training of automated agents using reinforcement learning algorithms. Blogpost: (2021) Gamifying machine learning for stronger security and AI models

gym-malware

gym-malware Malware Env for OpenAI Gym Paper: (2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning

gym-flipit

gym-flipit Gym environment for FLIPIT: The Game of "Stealthy Takeover" invented by Marten van Dijk, Ari Juels, Alina Oprea, and Ronald L. Rivest. Paper: (2019) QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game

gym-threat-defense

gym-threat-defense Gym environment for the environment described in the paper: (2019) Optimal Defense Policies for Partially Observable Spreading Processes on Bayesian Attack Graphs

gym-nasim

gym-nasim Thesis: (2018) Autonomous Penetration Testing using Reinforcement Learning

gym-optimal-intrusion-response

gym-optimal-intrusion-response An OpenAI Gym interface to a MDP/Markov Game model for optimal intrusion response of a realistic infrastructure simulated using system traces. Paper: (2021) Learning Intrusion Prevention Policies through Optimal Stopping

sql_env

sql_env Paper: (2021) SQL Injections and Reinforcement Learning: An Empirical Evaluation of the Role of Action Structure

↑ Papers

Surveys

• (2021) Prospective Artificial Intelligence Approaches for Active Cyber Defence
• (2019) Deep Reinforcement Learning for Cyber Security

Regular Papers

• (2021) An Efficient Parallel Reinforcement Learning Approach to Cross-Layer Defense Mechanism in Industrial Control Systems
• (2021) SDN-based Moving Target Defense using Multi-agent Reinforcement Learning
• (2021) Reinforcement Learning for Industrial Control Network Cyber Security Orchestration
• (2021) Automating Privilege Escalation with Deep Reinforcement Learning
• (2021) Multi-Agent Reinforcement Learning Framework in SDN-IoT for Transient Load Detection and Prevention
• (2021) Crown Jewels Analysis using Reinforcement Learning with Attack Graphs
• (2021) Deep Q-Learning based Reinforcement Learning Approach for Network Intrusion Detection
• (2021) Deep-Reinforcement-Learning-Based Intrusion Detection in Aerial Computing Networks
• (2021) Deep Reinforcement Learning for Securing Software Defined Industrial Networks with Distributed Control Plane
• (2021) CyGIL: A Cyber Gym for Training Autonomous Agents over Emulated Network Systems
• (2021) Constraints Satisfiability Driven Reinforcement Learning for Autonomous Cyber Defense
• (2021) Catch Me If You Learn: Real-Time Attack Detection and Mitigation in Learning Enabled CPS
• (2021) Network Environment Design for Autonomous Cyberdefense
• (2021) CybORG: A Gym for the Development of Autonomous Cyber Agents
• (2021) SQL Injections and Reinforcement Learning: An Empirical Evaluation of the Role of Action Structure
• (2021) Deep Reinforcement Learning based Smart Mitigation of DDoS Flooding in Software-Defined Networks
• (2021) Towards Autonomous Defense of SDN Networks Using MuZero Based Intelligent Agent
• (2021) Intrusion Prevention through Optimal Stopping
• (2021) Defense Against Advanced Persistent Threats in Smart Grids: A Reinforcement Learning Approach
• (2021) Deep hierarchical reinforcement agents for automated penetration testing
• (2021) Learning Intrusion Prevention Policies through Optimal Stopping
• (2021) Using Cyber Terrain in Reinforcement Learning for Penetration Testing
• (2020) A Hybrid Game Theory and Reinforcement Learning Approach for Cyber-Physical Systems Security
• (2020) Machine Learning Cyberattack and Defense Strategies
• (2020) Automated Post-Breach Penetration Testing through Reinforcement Learning
• (2020) DeepBLOC: A Framework for Securing CPS through Deep Reinforcement Learning on Stochastic Games
• (2020) Deep Reinforcement Learning for DER Cyber-Attack Mitigation
• (2020) Adaptive Cyber Defense Against Multi-Stage Attacks Using Learning-Based POMDP
• (2020) Autonomous Security Analysis and Penetration Testing
• (2020) POMDP + Information-Decay: Incorporating Defender's Behaviour in Autonomous Penetration Testing
• (2020) ATMoS: Autonomous Threat Mitigation in SDN using Reinforcement Learning
• (2020) Modeling Penetration Testing with Reinforcement Learning Using Capture-the-Flag Challenges: Trade-offs between Model-free Learning and A Priori Knowledge
• (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play
• (2020) AFRL: Adaptive federated reinforcement learning for intelligent jamming defense in FANET
• (2020) Reinforcement Learning for Efficient Network Penetration Testing
• (2020) The Agent Web Model -- Modelling web hacking for reinforcement learning
• (2020) Stochastic Dynamic Information Flow Tracking Game using Supervised Learning for Detecting Advanced Persistent Threats
• (2020) Reinforcement Learning Based PHY Authentication for VANETs
• (2020) Deep Reinforcement Learning for Cybersecurity Assessment of Wind Integrated Power Systems
• (2019) Finding Needles in a Moving Haystack: Prioritizing Alerts with Adversarial Reinforcement Learning
• (2019) Evaluation of Reinforcement Learning-Based False Data Injection Attack to Automatic Voltage Control
• (2019) Study of Learning of Power Grid Defense Strategy in Adversarial Stage Game
• (2019) Learning to Cope with Adversarial Attacks
• (2019) Learning Distributed Cooperative Policies for Security Games via Deep Reinforcement Learning
• (2019) An Efficient Reinforcement Learning-Based Botnet Detection approach
• (2019) Strategic Learning for Active, Adaptive, and Autonomous Cyber Defense
• (2019) QFlip: An Adaptive Reinforcement Learning Strategy for the FlipIt Security Game
• (2019) Solving Cyber Alert Allocation Markov Games with Deep Reinforcement Learning
• (2019) Adaptive Honeypot Engagement Through Reinforcement Learning of Semi-Markov Decision Processes
• (2019) Detecting Phishing Websites through Deep Reinforcement Learning
• (2019) Adversarial Deep Reinforcement Learning based Adaptive Moving Target Defense
• (2019) Autonomous Penetration Testing using Reinforcement Learning
• (2019) A Multistage Game in Smart Grid Security: A Reinforcement Learning Solution
• (2019) Automating Penetration Testing using Reinforcement Learning
• (2019) Reinforcement Learning-Based DoS Mitigation in Software Defined Networks
• (2019) Adversarial attack and defense in reinforcement learning-from AI security view
• (2019) A Learning-Based Solution for an Adversarial Repeated Game in Cyber–Physical Power Systems
• (2018) Simulating SQL Injection Vulnerability Exploitation Using Q-Learning Reinforcement Learning Agents
• (2018) A reinforcement learning approach for attack graph analysis
• (2018) Reinforcement Learning for Autonomous Defence in Software-Defined Networking
• (2018) Learning to Evade Static PE Machine Learning Malware Models via Reinforcement Learning
• (2018) Autonomic Computer Network Defence Using Risk State and Reinforcement Learning
• (2018) Reinforcement Learning for Intelligent Penetration Testing
• (2018) Autonomous Intelligent Cyber-defense Agent (AICA) Reference Architecture
• (2018) Deep reinforecement learning based optimal defense for cyber-physical system in presence of unknown cyber-attack
• (2018) Adversarial Reinforcement Learning for Observer Design in Autonomous Systems under Cyber Attacks
• (2018) Machine learning for autonomous cyber defense
• (2018) Online Cyber-Attack Detection in Smart Grid: A Reinforcement Learning Approach
• (2018) UAV Relay in VANETs Against Smart Jamming With Reinforcement Learning
• (2018) Security in Mobile Edge Caching with Reinforcement Learning
• (2017) Adversarial Reinforcement Learning in a Cyber Security Simulation
• (2017) Q-learning Based Vulnerability Analysis of Smart Grid against Sequential Topology Attacks
• (2017) Multi-agent Reinforcement Learning Based Cognitive Anti-jamming
• (2017) Reinforcement Learning Based Mobile Offloading for Cloud-Based Malware Detection
• (2017) A Secure Mobile Crowdsensing Game With Deep Reinforcement Learning
• (2016) Markov Security Games: Learning in Spatial Security Problems
• (2016) Reinforcement Learning Based Anti-jamming with Wideband Autonomous Cognitive Radios
• (2015) Application of reinforcement learning for security enhancement incognitive radio networks
• (2015) Power control with reinforcement learning in cooperative cognitive radio networks against jamming
• (2015) Mobile Cloud Offloading for Malware Detections with Learning
• (2014) Reinforcement Learning Algorithms for Adaptive Cyber Defense against Heartbleed
• (2014) Cooperative game theoretic approach using fuzzy Q-learning for detecting and preventing intrusions in wireless sensor networks
• (2013) Multiagent Router Throttling: Decentralized Coordinated Response Against DDoS Attacks
• (2013) Hybrid Learning in Stochastic Games and Its Application in Network Security
• (2013) Competing Mobile Network Game: Embracing Antijamming and Jamming Strategies with Reinforcement Learning
• (2012) Intrusion Detection System using Log Files and Reinforcement Learning
• (2012) Anti-jamming in Cognitive Radio Networks Using Reinforcement Learning Algorithms
• (2011) An Anti-jamming Strategy for Channel Access in Cognitive Radio Networks
• (2008) Reinforcement Learning for Vulnerability Assessment in Peer-to-Peer Networks
• (2005) A Reinforcement Learning Approach for Host-Based Intrusion Detection Using Sequences of System Calls

PhD Theses

• (2014) Distributed Reinforcement Learning for Network Intrusion Response

Master Theses

• (2021) Bayesian Reinforcement Learning Methods for Network Intrusion Prevention
• (2019) Learning to Hack
• (2018) Analysis of Network Intrusion Detection System with Machine Learning Algorithms (Deep Reinforcement Learning Algorithm)

Bachelor Theses

• (2018) Autonomous Penetration Testing using Reinforcement Learning

Posters

• (2021) Learning Intrusion Prevention Policies through Optimal Stopping

↑ Books

• (2021) Game Theory and Machine Learning for Cyber Security (Chapter 5 on RL)
• (2019) Reinforcement Learning for Cyber-Physical Systems with Cybersecurity Case Studies

↑ Blogposts

• (2021) Gamifying machine learning for stronger security and AI models
• (2021) Automating Cyber-Security With Reinforcement Learning

↑ Talks

• (2021) Applying Deep Reinforcement Learning (DRL) in a Cyber Wargaming Engine
• (2021) Automated Penetration Testing using Reinforcement Learning
• (2021) Training an Autonomous Pentester with Deep RL
• (2021) Learning Intrusion Prevention Policies Through Optimal Stopping
• (2020) Finding Effective Security Strategies through Reinforcement Learning and Self-Play
• (2020) Autonomous Security Analysis and Penetration Testing: A reinforcement learning approach.
• (2019) Cost-Efficient Malware Detection Using Deep Reinforcement Learning
• (2019) A Reinforcement Learning Framework for Smart, Secure, and Efficient Cyber-Physical Autonomy

↑ Miscellaneous

• (2021) IJCAI First International Workshop on Adaptive Cyber Defense
• (2021) Self-Learning AI

Contribute

Contribution are very well welcome and will be accepted on a regular basis! Please use Github Issues and Pull requests.

License

LICENSE

Creative Commons

(C) 2021