Deep Reinforcement Learning for Multiplayer Online Battle Arena

Last update: Dec 18, 2022

Related tags

Deep Learning MOBA_RL

Overview

MOBA_RL

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Prerequisite

Python 3
gym-derk
Tensorflow 2.4.1
Dotaservice of TimZaman
Seed RL of Google
Ubuntu 20.04
RTX 3060 GPU, 16GB RAM is used to run Dota2 environment with rendering
RTX 3080 GPU, 46GB RAM is used to training 16 number of headless Dota2 environment together in my case

Derk Environment

We are going to train small MOBA environment called Derk.

First, move to dr-derks-mutant-battlegrounds folder.

Run below command to run the 50 parallel environemnt. I modified Seel_RL of Google for my MOBA case.

$ python learner_1.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python learner_2.py --workspace_path [your path]/dr-derks-mutant-battlegrounds/
$ python run.py -p1 bot -p2 oldbot -n 50

You can check the training progress using Tensorboard log under tboard path of workspace.

Dota2 Environment

Rendering Environment

You first need to install Dota 2 from Steam. After installation, please check there is Dota2 folder under /home/[your account]/.steam/steam/steamapps/common/dota 2 beta'. We are going to run Dota2 from terminal command.

Next, you need to download and install dotaservice. In my case, I should modity the _run_dota function of dotaservice.py like below.

async def _run_dota(self):
  script_path = os.path.join(self.dota_path, self.DOTA_SCRIPT_FILENAME)
  script_path = '/home/kimbring2/.local/share/Steam/ubuntu12_32/steam-runtime/run.sh'

  # TODO(tzaman): all these options should be put in a proto and parsed with gRPC Config.
  args = [
       script_path,
       '/home/kimbring2/.local/share/Steam/steamapps/common/dota 2 beta/game/dota.sh',
       '-botworldstatesocket_threaded',
       '-botworldstatetosocket_frames', '{}'.format(self.ticks_per_observation),
       '-botworldstatetosocket_radiant', '{}'.format(self.PORT_WORLDSTATES[TEAM_RADIANT]),
       '-botworldstatetosocket_dire', '{}'.format(self.PORT_WORLDSTATES[TEAM_DIRE]),
       '-con_logfile', 'scripts/vscripts/bots/{}'.format(self.CONSOLE_LOG_FILENAME),
       '-con_timestamp',
       '-console',
       '-dev',
       '-insecure',
       '-noip',
       '-nowatchdog',  # WatchDog will quit the game if e.g. the lua api takes a few seconds.
       '+clientport', '27006',  # Relates to steam client.
       '+dota_1v1_skip_strategy', '1',
       '+dota_surrender_on_disconnect', '0',
       '+host_timescale', '{}'.format(self.host_timescale),
       '+hostname dotaservice',
       '+sv_cheats', '1',
       '+sv_hibernate_when_empty', '0',
       '+tv_delay', '0',
       '+tv_enable', '1',
       '+tv_title', '{}'.format(self.game_id),
       '+tv_autorecord', '1',
       '+tv_transmitall', '1',  # TODO(tzaman): what does this do exactly?
  ]

Training Environment

You need to build the Docker image of Dotaservice mentioned in README of Docker of the dotaservice.

You can run the Seel RL for Dota2 using below command.

$ ./run_dotaservice.sh 16
$ ./run_impala.sh 16

Addidinally, you can terminate all process using below command.

$ ./stop.sh

Deep Reinforcement Learning for Multiplayer Online Battle Arena

Related tags

Overview

MOBA_RL

Prerequisite

Derk Environment

Dota2 Environment

Rendering Environment

Training Environment

Owner

Dohyeong Kim

Implementation of Bagging and AdaBoost Algorithm

Anomaly Transformer: Time Series Anomaly Detection with Association Discrepancy" (ICLR 2022 Spotlight)

Blender add-on: Add to Cameras menu: View → Camera, View → Add Camera, Camera → View, Previous Camera, Next Camera

PyTorch Implementation of Unsupervised Depth Completion with Calibrated Backprojection Layers (ORAL, ICCV 2021)

Deep Halftoning with Reversible Binary Pattern

Learning where to learn - Gradient sparsity in meta and continual learning

Jittor implementation of Recursive-NeRF: An Efficient and Dynamically Growing NeRF

Experimental Python implementation of OpenVINO Inference Engine (very slow, limited functionality). All codes are written in Python. Easy to read and modify.

Pytorch implementation of our paper under review -- 1xN Pattern for Pruning Convolutional Neural Networks

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Code repository for EMNLP 2021 paper 'Adversarial Attacks on Knowledge Graph Embeddings via Instance Attribution Methods'

For encoding a text longer than 512 tokens, for example 800. Set max_pos to 800 during both preprocessing and training.

Pytorch implementation of Bert and Pals: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning

Acoustic mosquito detection code with Bayesian Neural Networks

💡 Type hints for Numpy

This is a beginner-friendly repo to make a collection of some unique and awesome projects. Everyone in the community can benefit & get inspired by the amazing projects present over here.

Semi-Supervised Graph Prototypical Networks for Hyperspectral Image Classification, IGARSS, 2021.

SEC'21: Sparse Bitmap Compression for Memory-Efficient Training onthe Edge

Architecture Patterns with Python (TDD, DDD, EDM)

MODALS: Modality-agnostic Automated Data Augmentation in the Latent Space