Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

Overview

Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods

This repository is the official implementation of

  • Seohong Park, Jaekyeom Kim, Gunhee Kim. Time Discretization-Invariant Safe Action Repetition for Policy Gradient Methods. In NeurIPS, 2021.

It contains the implementations for SAR, FiGAR-C and base policy gradient algorithms (PPO, TRPO and A2C).

The code is based on Stable Baselines3 (SB3) for PPO and A2C, and Stable Baselines (SB) for TRPO.

Requirements

Run examples

PPO and A2C (based on Stable Baselines3)

To install requirements:

cd sb3
pip install -r requirements.txt
pip install -e .

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train FiGAR-C-PPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.01 --max_t=0.05

Train PPO on InvertedPendulum-v2 with the original δ:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg

Train SAR-A2C on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --algo=a2c_rg --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Action Noise" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=action --anoise_prob=0.05 --anoise_std=3

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "External Force" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_f --anoise_prob=0.05 --anoise_std=300

Train SAR-PPO on InvertedPendulum-v2 with δ = 0.002 and the "Strong External Force (Perceptible)" setting:

python repeat/main.py --env=InvertedPendulum-v2 --algo=ppo_rg --frame_skip=1 --dt=0.002 --max_t=0.05 --max_d=0.5 --anoise_type=ext_fpc --anoise_prob=0.05 --anoise_std=1000

TRPO (based on Stable Baselines)

To install requirements:

cd sb
pip install -r requirements.txt
pip install -e .

Train SAR-TRPO on InvertedPendulum-v2 with δ = 0.01:

python repeat/main.py --env=InvertedPendulum-v2 --frame_skip=1 --dt=0.01 --max_t=0.05 --max_d=0.5

License

This codebase is licensed under the MIT License. See also sb3/LICENSE_SB3 and sb/LICENSE_SB.

Owner
Seohong Park
Seohong Park
Docker Compose based system for running remote browsers (including Flash and Java support) connected to web archives

pywb Remote Browsers This repository provides a simple configuration for deploying any pywb with remote browsers provided by OWT/Shepherd Remote Brows

Webrecorder 10 Jul 28, 2022
Installation of hacking tools

Tools-Spartan This is a program that makes it easy for you to download and install tools used in Kali Linux, there are tons of tools available.

1 Nov 10, 2021
Detection tool of malware(s) by checksum (useful for forensic)

🐍 malware_checker.py Detection tool of malware(s) by checksum (useful for forensic) 📦 Dependencies installation $ pip3 install -r requirements.txt

Fayred 1 Jan 30, 2022
the swiss army knife in the hash field. fast, reliable and easy to use

hexxus Hexxus is a fast hash cracking tool which checks more than 30 thousand passwords in under 4 seconds and can crack the following types bcrypt sh

enigma146 17 Apr 05, 2022
Exploit for GitLab CVE-2021-22205 Unauthenticated Remote Code Execution

Vuln Impact An issue has been discovered in GitLab CE/EE affecting all versions starting from 11.9. GitLab was not properly validating image files tha

Hendrik Agung 2 Dec 30, 2021
HatSploit collection of generic payloads designed to provide a wide range of attacks without having to spend time writing new ones.

HatSploit collection of generic payloads designed to provide a wide range of attacks without having to spend time writing new ones.

EntySec 5 May 10, 2022
A proxy for asyncio.AbstractEventLoop for testing purposes

aioloop-proxy A proxy for asyncio.AbstractEventLoop for testing purposes. When tests writing for asyncio based code, there are controversial requireme

aio-libs 12 Dec 12, 2022
Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading

log4j-detect Simple Python 3 script to detect the "Log4j" Java library vulnerability (CVE-2021-44228) for a list of URL with multithreading The script

Wade 1 Dec 15, 2021
Deltaspy - an advanced keylogger that can send keylogs and screenshots to gmail

Deltaspy Deltaspy is a advanced keylogger which sends keylogs and screenshot to

Praanesh S 1 Dec 31, 2021
DependConfusion-X Tool is written in Python3 that scans and monitors list of hosts for Dependency Confusion

DependConfusion-X Tool is written in Python3 which allows security researcher/bug bounty hunter to scan and monitor list of hosts for Dependency Confusion.

Ali Fathi Ali Sawehli 4 Dec 21, 2021
A Python Bytecode Disassembler helping reverse engineers in dissecting Python binaries

A Python Bytecode Disassembler helping reverse engineers in dissecting Python binaries by disassembling and analyzing the compiled python byte-code(.pyc) files across all python versions (including P

neeraj 95 Dec 26, 2022
Encrypted Python Password Manager

PyPassKeep Encrypted Python Password Manager About PyPassKeep (PPK for short) is an encrypted python password manager used to secure your passwords fr

KrisIsHere 1 Nov 17, 2021
AutoScan 有多个目标时,调用xray+rad进行自动扫描

Usage: 在高级版Xray和rad同目录下运行 python3 X-AutoXray.py xxxx.txt 写的蛮人性化的哦,os,linux,windows通用 生成的xray报告会在当前目录的/result下面 Ctrl+c 打断脚本运行时还可以结算扫描进度,生成已扫描和未扫描的进度文件,

斯文 73 Jan 01, 2023
ProxyLogon Pre-Auth SSRF To Arbitrary File Write

ProxyLogon Pre-Auth SSRF To Arbitrary File Write For Education and Research Usage: C:\python proxylogon.py mail.evil.corp lulz 117 Nov 28, 2022

Cobalt Strike < 4.4 dos CVE-2021-36798

CVE-2021-36798 CVE-2021-36798 Cobalt Strike 4.3 dos 用法 python3 CVE-2021-36798.py BeaconURL 打瘫Cobalt Strike 只需要一个包 已测试 4.3 4.2 参考: https://labs.sent

37 Nov 09, 2022
Reusable Lightweight Pythonic Dependency Injection Library

Vacuna Inject everything! Vacuna is a little library to provide dependency management for your python code. Install pip install vacuna Usage import va

Fernando Martínez González 16 Sep 15, 2021
Zero-attacker is an multipurpose hacking tool with over 12 tools

Zero Attacker Zero Attacker is bunch of tools which we made for people.These all tools are for purpose of ethical hacking and discord tools. Who is th

Asjad 300 Dec 28, 2022
Infoga is a tool gathering email accounts informations (ip,hostname,country,...) from different public source

Infoga - Email OSINT Infoga is a tool gathering email accounts informations (ip,hostname,country,...) from different public source (search engines, pg

m4ll0k (mallok) 1.8k Jan 04, 2023
CamOver is a camera exploitation tool that allows to disclosure network camera admin password.

CamOver is a camera exploitation tool that allows to disclosure network camera admin password. Features Exploits vulnerabilities in most popul

EntySec 247 Jan 02, 2023
GRR Rapid Response: remote live forensics for incident response

GRR Rapid Response is an incident response framework focused on remote live forensics. Build Type Status Tests End-to-end Tests Windows Templates Linu

Google 4.3k Jan 05, 2023