Implementation of Change-Based Exploration Transfer (C-BET)

Last update: Dec 04, 2022

Overview

This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

Implementation of Change-Based Exploration Transfer (C-BET), as presented in Interesting Object, Curious Agent: Learning Task-Agnostic Exploration.

This code was built on the RIDE repository.

Codebase and MiniGrid Installation

conda create -n cbet python=3.8.10
conda activate cbet
git clone [email protected]:sparisi/cbet.git
cd cbet
pip install -r requirements.txt

Habitat Installation (not Needed for MiniGrid Experiments)

Follow the official guide and do a full install with habitat_baselines.
Download and extract Replica scenes in the root folder of cbet

WARNING! The dataset is very large!

sudo apt-get install pigz
git clone https://github.com/facebookresearch/Replica-Dataset.git
cd Replica-Dataset
./download.sh replica-path

If the script does not work, manually unzip with cat replica_v1_0.tar.gz.part* | tar -xz

How to Run Experiments

Intrinsic-only pre-training: OMP_NUM_THREADS=1 python main.py --model cbet --env --no_reward --intrinsic_reward_coef=0.005
Extrinsic-only transfer with pre-trained model: OMP_NUM_THREADS=1 python main.py --model cbet --env --intrinsic_reward_coef=0.0 --checkpoint=path/to/model.tar
Tabula-rasa training with summed intrinsic and extrinsic reward: OMP_NUM_THREADS=1 python main.py --model cbet --env --intrinsic_reward_coef=0.005

See src/arguments.py for the full list of hyperparameters.

For MiniGrid, can be MiniGrid-DoorKey-8x8-v0, MiniGrid-Unlock-v0, ...
For Habitat, can be HabitatNav-apartment_0, HabitatNav-hotel_0, ...

RE3: State Entropy Maximization with Random Encoders for Efficient Exploration

State Entropy Maximization with Random Encoders for Efficient Exploration (RE3) (ICML 2021) Code for State Entropy Maximization with Random Encoders f

47 Nov 29, 2022

Generative Exploration and Exploitation - This is an improved version of GENE.

GENE This is an improved version of GENE. In the original version, the states are generated from the decoder of VAE. We have to check whether the gere

33 Mar 23, 2022

Systemic Evolutionary Chemical Space Exploration for Drug Discovery

SECSE SECSE: Systemic Evolutionary Chemical Space Explorer Chemical space exploration is a major task of the hit-finding process during the pursuit of

64 Dec 16, 2022

A repository with exploration into using transformers to predict DNA ↔ transcription factor binding

Transcription Factor binding predictions with Attention and Transformers A repository with exploration into using transformers to predict DNA ↔ transc

62 Dec 20, 2022

Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learning Source Code

8 Sep 14, 2022

Multi-robot collaborative exploration and mapping through Voronoi partition and DRL in unknown environment

Voronoi Multi_Robot Collaborate Exploration Introduction In the unknown environment, the cooperative exploration of multiple robots is completed by Vo

6 Nov 22, 2022

[TIP 2020] Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion

Multi-Temporal Scene Classification and Scene Change Detection with Correlation based Fusion Code for Multi-Temporal Scene Classification and Scene Ch

33 Dec 12, 2022

Remote sensing change detection tool based on PaddlePaddle

PdRSCD PdRSCD（PaddlePaddle Remote Sensing Change Detection）是一个基于飞桨PaddlePaddle的遥感变化检测的项目，pypi包名为ppcd。目前0.2版本，最新支持图像列表输入的训练和预测，如多期影像、多源影像甚至多期多源影像。可以快速完

38 Aug 31, 2022

TransCD: Scene Change Detection via Transformer-based Architecture

29 Dec 11, 2022

Comments

Bugfixes
Fixed a crash with Habitat environment in test script due to missing directory

Fixed an issue where count_reset_prob is referenced, but is not tracked in the ArgumentParser by removing it

Worked around a PyTorch memory bug (Ubuntu 21.10 + Driver Version: 495.29.05 + CUDA Version: 11.5 + torch version: 1.10.1+cu113)

Failed to allocate SHM despite plenty of available handles and many GiB of both system and GPU memory

Error message indicated an internal PyTorch bug, with instructions for filing a ticket
opened by rothn 0
Problem about intrinsic reward at pre-training stage

Hi,

I think I meet a problem that my results of intrinsic reward is about 0.0014 after training of 4e7 frames and I just follow the instruction of github without changing any parameters, the environments I use is MiniGrid-KeyCorridorS3R3-v0,MiniGrid-MultiRoom-N4-S5-v0,MiniGrid-UnlockPickup-v0, which are mentioned in the paper as pre-training of many-to-many transfer. Therefore, I don't know whether there are something I missed. Hoping you can help me. Thx a lot.

opened by dong845 2
Pretrained Model

One of my favorite components of the C-BET paper was the proposed paradigm shift from tabula-rasa exploration for each task to a system where new environments are explored with the context carried over from a pretrained model. I've found that a practical starting point for similar procedures on other large models (e.g., BERTs, ResNets) is to obtain a copy of the pre-trained model. I'd love to start working with C-BET as well!

I'm very curious as to where I might be able to find the C-BET parameters from your paper. Looking forward to experimenting with this!

opened by rothn 9

Releases(v1)

v1(Feb 23, 2022)

Pre-trained exploration policies for both MiniGrid and Habitat
Source code(tar.gz)
Source code(zip)
habitat.zip(634.99 MB)
minigrid.zip(627.27 MB)

Owner

Simone Parisi

GitHub Repository

PCACE: A Statistical Approach to Ranking Neurons for CNN Interpretability

PCACE: A Statistical Approach to Ranking Neurons for CNN Interpretability PCACE is a new algorithm for ranking neurons in a CNN architecture in order

4 Jan 04, 2022

🤖 A Python library for learning and evaluating knowledge graph embeddings

PyKEEN PyKEEN (Python KnowlEdge EmbeddiNgs) is a Python package designed to train and evaluate knowledge graph embedding models (incorporating multi-m

1.1k Jan 09, 2023

Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement

Decompose to Adapt: Cross-domain Object Detection via Feature Disentanglement In this project, we proposed a Domain Disentanglement Faster-RCNN (DDF)

19 Nov 24, 2022

TOOD: Task-aligned One-stage Object Detection, ICCV2021 Oral

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of

264 Jan 09, 2023

Original code for "Zero-Shot Domain Adaptation with a Physics Prior"

Zero-Shot Domain Adaptation with a Physics Prior [arXiv] [sup. material] - ICCV 2021 Oral paper, by Attila Lengyel, Sourav Garg, Michael Milford and J

40 Dec 21, 2022

Encode and decode text application

Text Encoder and Decoder Encode and decode text in many ways using this application! Encode in: ASCII85 Base85 Base64 Base32 Base16 Url MD5 Hash SHA-1

1 Feb 12, 2022

Open source code for the paper of Neural Sparse Voxel Fields.

Neural Sparse Voxel Fields (NSVF) Project Page | Video | Paper | Data Photo-realistic free-viewpoint rendering of real-world scenes using classical co

647 Dec 27, 2022

Toward Spatially Unbiased Generative Models (ICCV 2021)

Toward Spatially Unbiased Generative Models Implementation of Toward Spatially Unbiased Generative Models (ICCV 2021) Overview Recent image generation

88 Dec 01, 2022

Finite Element Analysis

FElupe - Finite Element Analysis FElupe is a Python 3.6+ finite element analysis package focussing on the formulation and numerical solution of nonlin

20 Jan 09, 2023

Improved Fitness Optimization Landscapes for Sequence Design

ReLSO Improved Fitness Optimization Landscapes for Sequence Design Description Citation How to run Training models Original data source Description In

44 Dec 20, 2022

Sharpness-Aware Minimization for Efficiently Improving Generalization

Sharpness-Aware-Minimization-TensorFlow This repository provides a minimal implementation of sharpness-aware minimization (SAM) (Sharpness-Aware Minim

54 Dec 08, 2022

Reduce end to end training time from days to hours (or hours to minutes), and energy requirements/costs by an order of magnitude using coresets and data selection.

COResets and Data Subset selection Reduce end to end training time from days to hours (or hours to minutes), and energy requirements/costs by an order

244 Jan 09, 2023

On the Adversarial Robustness of Visual Transformer

On the Adversarial Robustness of Visual Transformer Code for our paper "On the Adversarial Robustness of Visual Transformers"

35 Dec 14, 2022

Codes for NeurIPS 2021 paper "Adversarial Neuron Pruning Purifies Backdoored Deep Models"

Adversarial Neuron Pruning Purifies Backdoored Deep Models Code for NeurIPS 2021 "Adversarial Neuron Pruning Purifies Backdoored Deep Models" by Dongx

31 Dec 11, 2022

Learned image compression

Overview Pytorch code of our recent work A Unified End-to-End Framework for Efficient Deep Image Compression. We first release the code for Variationa

163 Dec 04, 2022

Official code of the paper "Expanding Low-Density Latent Regions for Open-Set Object Detection" (CVPR 2022)

OpenDet Expanding Low-Density Latent Regions for Open-Set Object Detection (CVPR2022) Jiaming Han, Yuqiang Ren, Jian Ding, Xingjia Pan, Ke Yan, Gui-So

64 Jan 07, 2023

[CVPR 2022 Oral] EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation

EPro-PnP EPro-PnP: Generalized End-to-End Probabilistic Perspective-n-Points for Monocular Object Pose Estimation In CVPR 2022 (Oral). [paper] Hanshen

842 Jan 04, 2023