Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

Overview

Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

This is the code for the paper Solving Graph-based Public Good Games with Tree Search and Imitation Learning by Victor-Alexandru Darvariu, Stephen Hailes and Mirco Musolesi, presented at NeurIPS 2021. If you use this code, please consider citing:

@inproceedings{darvariu_solving_2021,
  title = {Solving Graph-based Public Good Games with Tree Search and Imitation Learning},
  author = {Darvariu, Victor-Alexandru and Hailes, Stephen and Musolesi, Mirco},
  booktitle = {35th Conference on Neural Information Processing Systems (NeurIPS 2021)},
  year={2021},
}

License

MIT.

Prerequisites

Currently tested on Linux and MacOS (specifically, CentOS 7.4.1708 and Mac OS Big Sur 11.2.3), can also be adapted to Windows through WSL. The host machine requires NVIDIA CUDA toolkit version 9.0 or above (tested with NVIDIA driver version 384.81).

Makes heavy use of Docker, see e.g. here for how to install. Tested with Docker 19.03. The use of Docker largely does away with dependency and setup headaches, making it significantly easier to reproduce the reported results.

Configuration

The Docker setup uses Unix groups to control permissions. You can reuse an existing group that you are a member of, or create a new group groupadd -g GID GNAME and add your user to it usermod -a -G GNAME MYUSERNAME.

Create a file relnet.env at the root of the project (see relnet_example.env) and adjust the paths within: this is where some data generated by the container will be stored. Also specify the group ID and name created / selected above.

Add the following lines to your .bashrc, replacing /home/john/git/relnet with the path where the repository is cloned.

export RN_SOURCE_DIR='/home/john/git/relnet'
set -a
. $RN_SOURCE_DIR/relnet.env
set +a

export PATH=$PATH:$RN_SOURCE_DIR/scripts

Make the scripts executable (e.g. chmod u+x scripts/*) the first time after cloning the repository, and run apply_permissions.sh in order to create and permission the necessary directories.

Managing the containers

Some scripts are provided for convenience. To build the containers (note, this will take a significant amount of time e.g. 2 hours, as some packages are built from source):

update_container.sh

To start them:

manage_container_gpu.sh up
manage_container.sh up

To stop them:

manage_container_gpu.sh stop
manage_container.sh stop

To purge the queue and restart the containers (useful for killing tasks that were launched):

purge_and_restart.sh

Adjusting the number of workers and threads

To take maximum advantage of your machine's capacity, you may want to tweak the number of threads for the GPU and CPU workers. This configuration is provided in projectconfig.py. Additionally, you may want to enforce certain memory limits for your workers to avoid OOM errors. This can be tweaked in docker-compose.yml and manage_container_gpu.sh.

It is also relatively straightforward to add more workers from different machines you control. For this, you will need to mount the volumes on networked-attached storage (i.e., make sure paths provided in relnet.env are network-accessible) and adjust the location of backend and queue in projectconfig.py to a network location instead of localhost. On the other machines, only start the worker container (see e.g. manage_container.sh).

Setting up graph data

Synthetic data will be automatically generated when the experiments are ran and stored to $RN_EXPERIMENT_DIR/stored_graphs.

Accessing the services

There are several services running on the manager node.

  • Jupyter notebook server: http://localhost:8888
  • Flower for queue statistics: http://localhost:5555
  • Tensorboard (currently disabled due to its large memory footprint): http://localhost:6006
  • RabbitMQ management: http://localhost:15672

The first time Jupyter is accessed it will prompt for a token to enable password configuration, it can be grabbed by running docker exec -it relnet-manager /bin/bash -c "jupyter notebook list".

Accessing experiment data and results database

Experiment data and results are stored in part as files (under your configured $RN_EXPERIMENT_DATA_DIR) as well as in a MongoDB database. To access the MongoDB database with a GUI, you can use a MongoDB client such as Robo3T and point it to http://localhost:27017.

Some functionality is provided in relnet/evaluation/storage.py to insert and retrieve data, you can use it in e.g. analysis notebooks.

Running experiments

Experiments are launched from the manager container and processed (in a parallel way) by the workers. The file relnet/evaluation/experiment_conditions.py contains the configuration for the experiments reported in the paper, but you may modify e.g. agents, objective functions, hyperparameters etc. to suit your needs.

Then, you can launch all the experiments as follows:

Part 1: Hyperparameter optimization & evaluation for all aproaches except GIL

run_part1.sh

Part 2: Data collection for GIL using the UCT algorithm

run_part2.sh

Part 3: Training & hyperparameter optimization for GIL

run_part3.sh

Monitoring experiments

  • You can navigate to http://localhost:5555 for the Flower interface which shows the progress of processing tasks in the queue. You may also check logs for both manager and worker at $RN_EXPERIMENT_DATA_DIR/logs.

Reproducing the results

Jupyter notebooks are used to perform the data analysis and produce tables and figures. Navigate to http://localhost:8888, then notebooks folder.

All tables and result figures can be obtained by opening the GGNN_Evaluation.ipynb notebook, selecting the py3-relnet kernel and run all cells. Resulting .pdf figures and .tex tables can be found at $RN_EXPERIMENT_DIR/aggregate. There are additional notebooks provided for analyzing the results of hyperparameter optimization:

  • GGNN_Hyperparam_Optimisation.ipynb for UCT
  • GGNN_Hyperparam_Optimisation_IL.ipynb for GIL

Problems with jupyter kernel

In case the py3-relnet kernel is not found, try reinstalling the kernel by running docker exec -it -u 0 relnet-manager /bin/bash -c "source activate relnet-cenv; python -m ipykernel install --user --name relnet --display-name py3-relnet"

Owner
Victor-Alexandru Darvariu
Doctoral Student at University College London and The Alan Turing Institute.
Victor-Alexandru Darvariu
Self-Supervised Deep Blind Video Super-Resolution

Self-Blind-VSR Paper | Discussion Self-Supervised Deep Blind Video Super-Resolution By Haoran Bai and Jinshan Pan Abstract Existing deep learning-base

Haoran Bai 35 Dec 09, 2022
Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation

Unseen Object Clustering: Learning RGB-D Feature Embeddings for Unseen Object Instance Segmentation Introduction In this work, we propose a new method

NVIDIA Research Projects 132 Dec 13, 2022
《Rethinking Sptil Dimensions of Vision Trnsformers》(2021)

Rethinking Spatial Dimensions of Vision Transformers Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, Seong Joon Oh | Paper NAVER

NAVER AI 224 Dec 27, 2022
Ppq - A powerful offline neural network quantization tool with custimized IR

PPL Quantization Tool(PPL 量化工具) PPL Quantization Tool (PPQ) is a powerful offlin

605 Jan 03, 2023
particle tracking model, works with the ROMS output file(qck.nc, his.nc)

particle-tracking-model-for-ROMS particle tracking model, works with the ROMS output file(qck.nc, his.nc) description this is a 2-dimensional particle

xusheng 1 Jan 11, 2022
Source code for our paper "Do Not Trust Prediction Scores for Membership Inference Attacks"

Do Not Trust Prediction Scores for Membership Inference Attacks Abstract: Membership inference attacks (MIAs) aim to determine whether a specific samp

<a href=[email protected]"> 3 Oct 25, 2022
Learning 3D Part Assembly from a Single Image

Learning 3D Part Assembly from a Single Image This repository contains a PyTorch implementation of the paper: Learning 3D Part Assembly from A Single

18 Dec 21, 2022
VisionKG: Vision Knowledge Graph

VisionKG: Vision Knowledge Graph Official Repository of VisionKG by Anh Le-Tuan, Trung-Kien Tran, Manh Nguyen-Duc, Jicheng Yuan, Manfred Hauswirth and

Continuous Query Evaluation over Linked Stream (CQELS) 9 Jun 23, 2022
Reusable constraint types to use with typing.Annotated

annotated-types PEP-593 added typing.Annotated as a way of adding context-specific metadata to existing types, and specifies that Annotated[T, x] shou

125 Dec 26, 2022
MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Takuma Seno 80 Nov 06, 2022
Source code of D-HAN: Dynamic News Recommendation with Hierarchical Attention Network

D-HAN The source code of D-HAN This is the source code of D-HAN: Dynamic News Recommendation with Hierarchical Attention Network. However, only the co

30 Sep 22, 2022
Intrusion Test Tool with Python

P3ntsT00L Uma ferramenta escrita em Python, feita para Teste de intrusão. Requisitos ter o python 3.9.8 instalado em sua máquina. ter a git instalada

josh washington 2 Dec 27, 2021
Official implementation of Representer Point Selection via Local Jacobian Expansion for Post-hoc Classifier Explanation of Deep Neural Networks and Ensemble Models at NeurIPS 2021

Representer Point Selection via Local Jacobian Expansion for Classifier Explanation of Deep Neural Networks and Ensemble Models This repository is the

Yi(Amy) Sui 2 Dec 01, 2021
The code for replicating the experiments from the LFI in SSMs with Unknown Dynamics paper.

Likelihood-Free Inference in State-Space Models with Unknown Dynamics This package contains the codes required to run the experiments in the paper. Th

Alex Aushev 0 Dec 27, 2021
Soomvaar is the repo which 🏩 contains different collection of 👨‍💻🚀code in Python and 💫✨Machine 👬🏼 learning algorithms📗📕 that is made during 📃 my practice and learning of ML and Python✨💥

Soomvaar 📌 Introduction Soomvaar is the collection of various codes implement in machine learning and machine learning algorithms with python on coll

Felix-Ayush 42 Dec 30, 2022
Implementation of SSMF: Shifting Seasonal Matrix Factorization

SSMF Implementation of SSMF: Shifting Seasonal Matrix Factorization, Koki Kawabata, Siddharth Bhatia, Rui Liu, Mohit Wadhwa, Bryan Hooi. NeurIPS, 2021

Koki Kawabata 9 Jun 10, 2022
In generative deep geometry learning, we often get many obj files remain to be rendered

a python prompt cli script for blender batch render In deep generative geometry learning, we always get many .obj files to be rendered. Our rendered i

Tian-yi Liang 1 Mar 20, 2022
UltraGCN: An Ultra Simplification of Graph Convolutional Networks for Recommendation

UltraGCN This is our Pytorch implementation for our CIKM 2021 paper: Kelong Mao, Jieming Zhu, Xi Xiao, Biao Lu, Zhaowei Wang, Xiuqiang He. UltraGCN: A

XUEPAI 93 Jan 03, 2023
Ejemplo Algoritmo Viterbi - Example of a Viterbi algorithm applied to a hidden Markov model on DNA sequence

Ejemplo Algoritmo Viterbi Ejemplo de un algoritmo Viterbi aplicado a modelo ocul

Mateo Velásquez Molina 1 Jan 10, 2022
PyTorch implementation of Super SloMo by Jiang et al.

Super-SloMo PyTorch implementation of "Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation" by Jiang H., Sun

Avinash Paliwal 2.9k Jan 03, 2023