Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Last update: Jan 05, 2023

Related tags

Deep Learning MG-GAN

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

This repository contains the code for the paper

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction
Patrick Dendorfer*, Sven Elflein*, Laura Leal-Taixé (* equal contribution)
International Conference on Computer Vision (ICCV), 2021

Motivation

The distribution over future trajectories of pedestrians is often multi-modal and does not have connected support (a).

We found that single generator GANs introduce out-of-distribution (OOD) samples in this case due to GANs mapping the continuous latent variable z with a continuous function (b). These OOD samples might introduce unforseen behavior in real world applications, such as autonomous driving.

To resolve this problem, we propose to learn the target distribution in a piecewise manner using multiple generators, effectively preventing OOD samples (c).

Model

Our model consists of four key components: Encoding modules, Attention modules, and our novel contribution PM-Network learning a distribution over multiple Generators.

Setup

First, setup Python environment

conda create -f environment.yml -n mggan
conda activate mggan

Then, download the datasets (data.zip) from here and unzip in the root of this repository

unzip data.zip

which will create a folder ./data/datasets.

Training

Models can be trained using the script mggan/model/train.py using the following command

python mggan/models/pinet_multi_generator/train.py --name <name_of_experiment> --num_gens <number_of_generators>  --dataset <dataset_name> --epochs 50

This generates a output folder in ./logs/<name_of_experiment> with Tensorboard logs and the model checkpoints. You can use tensorboard --logdir ./logs/<name_of_experiment> to monitor the training process.

Evaluation

For evaluation of metrics (ADE, FDE, Precison, Recall) for k=1 to k=20 predictions, use

python scripts/evaluate.py --model_path <path_to_model_directory>  --output_folder <folder_to_store_result_csv>

One can use --eval-set <dataset_name> to evaluate models on other test sets than the dataset the model was trained on. This is useful to evaluate the BIWI models on the Garden of Forking Paths dataset (gofp) for which we report results in the paper.

Pre-trained models

We provide pre-trained models for MG-GAN with 2-8 generators together with the training configurations, on the BIWI datasets and Stanford Drone dataset (SDD) here.

Citation

If our work is useful to you, please consider citing

@inproceedings{dendorfer2021iccv,
  title={MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction}, 
  author={Dendorfer, Patrick and Elflein, Sven and Leal-Taixé, Laura},
  month={October}
  year={2021},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  }

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

DS3L This is the code for paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020. Setups The code is implem

36 Oct 19, 2022

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization This is the code for our paper ``SumGNN: Multi-typed Drug

58 Dec 21, 2022

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

HKD Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks" cifia-100 result The implementation of compared methods are ba

30 Dec 18, 2022

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

G-SFDA Code (based on pytorch 1.3) for our ICCV 2021 paper 'Generalized Source-free Domain Adaptation'. [project] [paper]. Dataset preparing Download

84 Dec 26, 2022

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

ARAPReg Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators.. Installation The cod

132 Nov 28, 2022

Comments

request to visualizer

Hello author！ I admire your work and would like to reproduce your results. There is a small requirement here that needs to trouble you. Do you have a visual code, which has shown the effect in your paper. Thanks again for your work and contributions！

opened by 12num 0
Question regarding Garden of Forking Path Dataset

Hello,

I see there are more scenes in the test set (ETH, Hotel, and ZARA1) than the train set (ETH) in your pre-processed dataset of GOFP. Could you kindly elaborate on why it is that?

Thanks, Sourav Das

opened by SodaCoder 0
Question about ETH&UCY Dataset

Hi, I notice that trajectories in some datasets are not consistent with provided in Social GAN. May I ask how do you preprocess your data? It will be helpful to conduct my experiments in a fair environment. Thanks!

opened by HRHLALALA 1
Reproducible MG-GAN code for the FPD dataset

Hello Patrick, Sven,

This is Sourav Das, a 1st year Ph.D. student at the University of Padova, Italy.

This Github repository has the reproducible implementation for the datasets: ETH, Hotel, Social_Stanford_Synthetic, Stanford, Univ, Zara1, Zara2, and GOFP.

I would like to reproduce the results on FPD datasets also. Could you kindly share with me the code with support for the FPD dataset?

Here is my Github: https://github.com/SodaCoder

Thanks in advance,

opened by SodaCoder 1

Releases(1.0)

1.0(Aug 14, 2021)

data.zip contains the datasets, checkpoints.zip contains pre-trained models for BIWI and Stanford Drone datasets.
Source code(tar.gz)
Source code(zip)
checkpoints.zip(296.92 MB)
data.zip(82.24 MB)

Code for our paper "MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction" published at ICCV 2021.

Related tags

Overview

MG-GAN: A Multi-Generator Model Preventing Out-of-Distribution Samples in Pedestrian Trajectory Prediction

Motivation

Model

Setup

Training

Evaluation

Pre-trained models

Citation

You might also like...

The implementation of the algorithm in the paper "Safe Deep Semi-Supervised Learning for Unseen-Class Unlabeled Data" published in ICML 2020.

This is the repo for the paper `SumGNN: Multi-typed Drug Interaction Prediction via Efficient Knowledge Graph Summarization'. (published in Bioinformatics'21)

Code for ICCV 2021 paper "Distilling Holistic Knowledge with Graph Neural Networks"

code for ICCV 2021 paper 'Generalized Source-free Domain Adaptation'

Code for ICCV 2021 paper: ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators..

Code for the ICCV 2021 paper "Pixel Difference Networks for Efficient Edge Detection" (Oral).

Sync2Gen Code for ICCV 2021 paper: Scene Synthesis via Uncertainty-Driven Attribute Synchronization

Code for the paper "Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds" (ICCV 2021)

Code release for ICCV 2021 paper "Anticipative Video Transformer"

Comments

request to visualizer

Question regarding Garden of Forking Path Dataset

Question about ETH&UCY Dataset

Reproducible MG-GAN code for the FPD dataset

Releases(1.0)

1.0(Aug 14, 2021)

Owner

Sven

A ssl analyzer which could analyzer target domain's certificate.

FTIR-Deep Learning - FTIR Deep Learning With Python

[ICRA 2022] An opensource framework for cooperative detection. Official implementation for OPV2V.

Cweqgen - The CW Equation Generator

PyGRANSO: A PyTorch-enabled port of GRANSO with auto-differentiation

Unsupervised Foreground Extraction via Deep Region Competition

Custom TensorFlow2 implementations of forward and backward computation of soft-DTW algorithm in batch mode.

Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNs

Pytorch implement of 'Unmixing based PAN guided fusion network for hyperspectral imagery'

NLU Dataset Diagnostics

MicRank is a Learning to Rank neural channel selection framework where a DNN is trained to rank microphone channels.

Baseline model for "GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping" (CVPR 2020)

Artifacts for paper "MMO: Meta Multi-Objectivization for Software Configuration Tuning"

Referring Video Object Segmentation

Adversarial Attacks on Probabilistic Autoregressive Forecasting Models.

Dynamic Token Normalization Improves Vision Transformers

NEO: Non Equilibrium Sampling on the orbit of a deterministic transform

Source code for the GPT-2 story generation models in the EMNLP 2020 paper "STORIUM: A Dataset and Evaluation Platform for Human-in-the-Loop Story Generation"

SiT: Self-supervised vIsion Transformer

Code to accompany the paper "Finding Bipartite Components in Hypergraphs", which is published in NeurIPS'21.