Generalized Decision Transformer for Offline Hindsight Information Matching

If you use this codebase for your research, please cite the paper:

@article{furuta2021generalized,
  title={Generalized Decision Transformer for Offline Hindsight Information Matching},
  author={Hiroki Furuta and Yutaka Matsuo and Shixiang Shane Gu},
  journal={arXiv preprint arXiv:2111.10364},
  year={2021}
}

Installation

Experiments require MuJoCo. Follow the instructions in the mujoco-py repo to install. Then, dependencies can be installed with the following command:

conda env create -f conda_env.yml

Downloading datasets

Datasets are stored in the data directory. Install the D4RL repo, following the instructions there. Then, run the following script in order to download the datasets and save them in our format:

python download_d4rl_datasets.py

Run experiments

Run train_cdt.py to train Categorical DT:

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_model True

python train_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_model True

Run eval_cdt.py to eval CDT using saved weights:

python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'reward' --save_rollout True
python eval_cdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --condition 'xvel' --save_rollout True

For Bi-directional DT, run train_bdt.py & eval_bdtf.py

python train_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_model True
python eval_bdt.py --env halfcheetah --dataset medium-expert --gpu 0 --seed 0 --dist_dim 30 --n_bins 31 --z_dim 16 --save_rollout True

Reference

This repository is developed on top of original Decision Transformer.

Generalized Decision Transformer for Offline Hindsight Information Matching

Related tags

Overview

Generalized Decision Transformer for Offline Hindsight Information Matching

Installation

Downloading datasets

Run experiments

Reference

Owner

Hiroki Furuta

Single-Stage 6D Object Pose Estimation, CVPR 2020

Face Recognition plus identification simply and fast | Python

WHENet: Real-time Fine-Grained Estimation for Wide Range Head Pose

This is a GUI interface which can process forest fire detection, smoke detection and fire segmentation

A package related to building quasi-fibration symmetries

Pytorch implementation of our paper LIMUSE: LIGHTWEIGHT MULTI-MODAL SPEAKER EXTRACTION.

PoolFormer: MetaFormer is Actually What You Need for Vision

deep-table implements various state-of-the-art deep learning and self-supervised learning algorithms for tabular data using PyTorch.

Pytorch implementation of the paper DocEnTr: An End-to-End Document Image Enhancement Transformer.

Hyperbolic Procrustes Analysis Using Riemannian Geometry

An inofficial PyTorch implementation of PREDATOR based on KPConv.

The repository for our EMNLP 2021 paper "Finnish Dialect Identification: The Effect of Audio and Text"

Mmdetection3d Noted - MMDetection3D is an open source object detection toolbox based on PyTorch

Hybrid CenterNet - Hybrid-supervised object detection / Weakly semi-supervised object detection

ECCV2020 paper: Fashion Captioning: Towards Generating Accurate Descriptions with Semantic Rewards. Code and Data.

Animatable Neural Radiance Fields for Modeling Dynamic Human Bodies

Code for DeepXML: A Deep Extreme Multi-Label Learning Framework Applied to Short Text Documents

code for Multi-scale Matching Networks for Semantic Correspondence, ICCV

An implementation of the efficient attention module.

[AAAI 2022] Sparse Structure Learning via Graph Neural Networks for Inductive Document Classification