The official PyTorch code implementation of "Human Trajectory Prediction via Counterfactual Analysis" in ICCV 2021.

Overview

Human Trajectory Prediction via Counterfactual Analysis (CausalHTP)

The official PyTorch code implementation of "Human Trajectory Prediction via Counterfactual Analysis" in ICCV 2021.arxiv

News

We add the implementation of our project Causal-STGAT, where we apply our CausalHTP method to the baseline backbone network STGAT. The code of Causal-STGCNN is coming soon.

Introduction

Most trajectory prediction methods concentrate on modeling the environment interactions and aggregate these interaction clues with history behavior clues for trajectory prediction. However, there are heavy biases in the between training and deployment environment interactions. The motivation of this project is to mitigate the negative effects of the inherent biases. We propose a counterfactual analysis method to alleviate the overdependence of environment bias and highlight the trajectory clues itself. This counterfactual analysis method is a plug-and-play module which can be easily applied to any baseline predictor, and consistently improves the performance on many human trajectory prediction benchmarks.

image Figure 1. Training process of our counterfactual analysis method. We apply the counterfactual intervention by replacing the features of past trajectory with the counterfactual features such as uniform rectilinear motion, mean trajectory, or random trajectory. The counterfactual prediction denotes the biased affect from environment confounder. To alleviate the negative effect of environment bias, we subtract the counterfactual prediction from original prediction as the final causal prediction.

Requirements

  • Python 3.6+
  • PyTorch 1.3

To build all the dependency, you can follow the instruction below.

pip install -r requirements.txt

Dataset

The datasets can be found in datasets/, we provide 5 scenes including eth, hotel, univ, zara1, and zara2.

Training and Evaluation

You can train the model for eth dataset as

python train.py --dataset_name eth

To evaluate the trained model, you can use

python evaluate_model.py --dataset_name eth --resume your_checkpoint.pth.tar

The pre-trained models can be found in pretrain/

Result

Results (ADE/FDE) ETH HOTEL ZARA1 ZARA2 UNIV AVG
STGAT 0.73/1.39 0.38/0.72 0.35/0.69 0.32/0.64 0.57/1.22 0.47/0.93
Causal-STGAT 0.60/0.98 0.30/0.54 0.32/0.64 0.28/0.58 0.52/1.10 0.40/0.77

image Figure 2. Visualization examples of our Causal-STGAT method and baseline Social-STGAT method in the different scenes in the both ETH and UCY datasets. The comparisons quantitatively demonstrate the effectiveness of our counterfactual analysis on the RNN-based baselines.

Citation

Part of the code comes from STGAT. If you find this code useful then please also cite their paper.

Please use the citation provided below if this repo is useful to your research:

@inproceedings{CausalHTP,
  title={Human Trajectory Prediction via Counterfactual Analysis},
  author={Chen, Guangyi and Li, Junlong and Lu, Jiwen and Zhou, Jie},
  booktitle={ICCV},
  year={2021}
}
MultiLexNorm 2021 competition system from ÚFAL

ÚFAL at MultiLexNorm 2021: Improving Multilingual Lexical Normalization by Fine-tuning ByT5 David Samuel & Milan Straka Charles University Faculty of

ÚFAL 13 Jun 28, 2022
An LSTM based GAN for Human motion synthesis

GAN-motion-Prediction An LSTM based GAN for motion synthesis has a few issues reading H3.6M data from A.Jain et al , will fix soon. Prediction of the

Amogh Adishesha 9 Jun 17, 2022
GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models

GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Model This repository is the official PyTorch implementation of GraphRNN, a graph gene

Jiaxuan 568 Dec 29, 2022
FMA: A Dataset For Music Analysis

FMA: A Dataset For Music Analysis Michaël Defferrard, Kirell Benzi, Pierre Vandergheynst, Xavier Bresson. International Society for Music Information

Michaël Defferrard 1.8k Dec 29, 2022
End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model

onnx-facial-lmk-detector End-to-end face detection, cropping, norm estimation, and landmark detection in a single onnx model, model.onnx. Demo You can

atksh 42 Dec 30, 2022
Official PyTorch implementation of "Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient".

Edge Rewiring Goes Neural: Boosting Network Resilience via Policy Gradient This repository is the official PyTorch implementation of "Edge Rewiring Go

Shanchao Yang 4 Dec 12, 2022
The Official Implementation of Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose [NIPS 2021].

Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose Release Notes The offical PyTorch implementation of Neural View Sy

Angtian Wang 20 Oct 09, 2022
Automatic Image Background Subtraction

Automatic Image Background Subtraction This repo contains set of scripts for automatic one-shot image background subtraction task using the following

Oleg Sémery 6 Dec 05, 2022
WRENCH: Weak supeRvision bENCHmark

🔧 What is it? Wrench is a benchmark platform containing diverse weak supervision tasks. It also provides a common and easy framework for development

Jieyu Zhang 176 Dec 28, 2022
Code repo for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper.

InterpretableMDE A PyTorch implementation for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper. arXiv link: https://arxiv.or

Zunzhi You 16 Aug 12, 2022
Convnext-tf - Unofficial tensorflow keras implementation of ConvNeXt

ConvNeXt Tensorflow This is unofficial tensorflow keras implementation of ConvNe

29 Oct 06, 2022
AnimationKit: AI Upscaling & Interpolation using Real-ESRGAN+RIFE

ALPHA 2.5: Frostbite Revival (Released 12/23/21) Changelog: [ UI ] Chained design. All steps link to one another! Use the master override toggles to s

87 Nov 16, 2022
Natural Intelligence is still a pretty good idea.

Human Learn Machine Learning models should play by the rules, literally. Project Goal Back in the old days, it was common to write rule-based systems.

vincent d warmerdam 641 Dec 26, 2022
Offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation

Shunted Transformer This is the offical implementation of Shunted Self-Attention via Multi-Scale Token Aggregation by Sucheng Ren, Daquan Zhou, Shengf

156 Dec 27, 2022
Streamlit component for TensorBoard, TensorFlow's visualization toolkit

streamlit-tensorboard This is a work-in-progress, providing a function to embed TensorBoard, TensorFlow's visualization toolkit, in Streamlit apps. In

Snehan Kekre 27 Nov 13, 2022
RepVGG: Making VGG-style ConvNets Great Again

This repository is the code that needs to be submitted for OpenMMLab Algorithm Ecological Challenge,the paper is RepVGG: Making VGG-style ConvNets Great Again

Ty Feng 62 May 21, 2022
Scenic: A Jax Library for Computer Vision and Beyond

Scenic Scenic is a codebase with a focus on research around attention-based models for computer vision. Scenic has been successfully used to develop c

Google Research 1.6k Dec 27, 2022
Cockpit is a visual and statistical debugger specifically designed for deep learning.

Cockpit: A Practical Debugging Tool for Training Deep Neural Networks

Felix Dangel 421 Dec 29, 2022
Politecnico of Turin Thesis: "Implementation and Evaluation of an Educational Chatbot based on NLP Techniques"

THESIS_CAIRONE_FIORENTINO Politecnico of Turin Thesis: "Implementation and Evaluation of an Educational Chatbot based on NLP Techniques" GENERATE TOKE

cairone_fiorentino97 1 Dec 10, 2021
A PoC Corporation Relationship Knowledge Graph System on top of Nebula Graph.

Corp-Rel is a PoC of Corpartion Relationship Knowledge Graph System. It's built on top of the Open Source Graph Database: Nebula Graph with a dataset

Wey Gu 20 Dec 11, 2022