Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

[NeurIPS 2021] Code for Unsupervised Learning of Compositional Energy Concepts

PyTorch code for the paper: FeatMatch: Feature-Based Augmentation for Semi-Supervised Learning

Implementation for Homogeneous Unbalanced Regularized Optimal Transport

A general-purpose encoder-decoder framework for Tensorflow

Fully Convolutional Refined Auto Encoding Generative Adversarial Networks for 3D Multi Object Scenes

This is a library for training and applying sparse fine-tunings with torch and transformers.

Hl classification bc - A Network-Based High-Level Data Classification Algorithm Using Betweenness Centrality

Geometric Vector Perceptrons --- a rotation-equivariant GNN for learning from biomolecular structure

This is a computer vision based implementation of the popular childhood game 'Hand Cricket/Odd or Even' in python

🦕 NanoSaur is a little tracked robot ROS2 enabled, made for an NVIDIA Jetson Nano

Evaluation toolkit of the informative tracking benchmark comprising 9 scenarios, 180 diverse videos, and new challenges.

LRBoost is a scikit-learn compatible approach to performing linear residual based stacking/boosting.

This repository contains the code used for the implementation of the paper "Probabilistic Regression with HuberDistributions"

Material del curso IIC2233 Programación Avanzada 📚

Official PyTorch implementation of "Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Image", ICCV 2019

Core ML tools contain supporting tools for Core ML model conversion, editing, and validation.

Deep Compression for Dense Point Cloud Maps.

A short code in python, Enchpyter, is able to encrypt and decrypt words as you determine, of course

exponential adaptive pooling for PyTorch

An end-to-end regression problem of predicting the price of properties in Bangalore.