Waymo motion prediction challenge 2021: 3rd place solution

Overview

Waymo motion prediction challenge 2021: 3rd place solution

header

Team behind this solution:

  1. Artsiom Sanakoyeu [Homepage] [Twitter] [Telegram Channel] [LinkedIn]
  2. Stepan Konev [LinkedIn]
  3. Kirill Brodt [GitHub]

Dataset

Download datasets uncompressed/tf_example/{training,validation,testing}

Prerender

Change paths to input dataset and output folders

python prerender.py \
    --data /home/data/waymo/training \
    --out ./train
    
python prerender.py \
    --data /home/data/waymo/validation \
    --out ./dev \
    --use-vectorize \
    --n-shards 1
    
python prerender.py \
    --data /home/data/waymo/testing \
    --out ./test \
    --use-vectorize \
    --n-shards 1

Training

MODEL_NAME=xception71
python train.py \
    --train-data ./train \
    --dev-data ./dev \
    --save ./${MODEL_NAME} \
    --model ${MODEL_NAME} \
    --img-res 224 \
    --in-channels 25 \
    --time-limit 80 \
    --n-traj 6 \
    --lr 0.001 \
    --batch-size 48 \
    --n-epochs 120

Submit

python submit.py \
    --test-data ./test/ \
    --model-path ${MODEL_PATH_TO_JIT} \
    --save ${SAVE}

Visualize predictions

python visualize.py \
    --model ${MODEL_PATH_TO_JIT} \
    --data ${DATA_PATH} \
    --save ./viz

Citation

If you find our work useful, please cite it as:

@article{konev2021motioncnn,
  title={MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving},
  author={Konev, Stepan and Brodt, Kirill and Sanakoyeu, Artsiom},
  year={2021}
}

Related repos

Comments
  • Metrics

    Metrics

    Hi! @kbrodt Thanks for sharing this great code!

    Where are the codes of the evaluation metrics (for example: ADE, FDE, minADE, minFDE and so on)? Or where can I find it?

    Looking forward to your reply!

    opened by chx-Github 9
  • Regarding the training epochs

    Regarding the training epochs

    Thanks for sharing this awesome codes!

    Is it necessary to train the model for 120 epochs? Since there are more than 1M training samples. Can you share some performance during the training progress? Such as the performance with 30epochs, 60 epochs, 90 epochs? Since I trained it for several epochs but the loss is still very large.

    To double check the training process, can you share how many training samples for each epoch?

    Thanks so much!

    opened by FutureOpenAI 8
  • Loss

    Loss

    Hi! I tried your method and I observed that in training, l2loss and log_softmax have so large difference. so my network does not learn multimodal tracks, only one best track is fitted. Do you have any solution?

    opened by zsgj-Xxx 7
  • Angle conversion error at prerender.py

    Angle conversion error at prerender.py

    Thank you for amazing works.

    I found the conversion error at prerender.py

    https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L706 https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L707

    These two lines has conversion error.

    In order to convert coordinate properly, it should be changed like this tmp[3] = other_v_yaw - ANGLE tmp[4] = other_bbox_yaw - ANGLE

    opened by KyuhwanYeon 3
  • Question About the waymo dataset lane type

    Question About the waymo dataset lane type

    Hi, @kbrodt,

    Thank you for your great work. I am new to this area, so I have a question regarding to the lane type of waymo dataset. In general, the lanes in waymo dataset, can be broadly categorized into 6: lanecenters, roadlines, stopsign, speedbump, roadedge and crosswalks. So, is the lane centers are just the the center of each line in which the vehicles can drive on? Theoretically, a car's center should be on the lanecenters if this car is staying in this lane. Is my interpretation right? Also, is the lanecenters in waymo the same as the centerlines in Argoverse Dataset? Are you familiar with that dataset? I am looking forward to your reply. Thank you in advance.

    opened by SwagJ 2
  • Vehicles driving on the same lane

    Vehicles driving on the same lane

    Hello!

    Thanks for sharing you work!

    I have one questions: from the motion dataset, is that possible to know if two vehicles are driving on the same lane? I notice that there is a column called "roadgraph_samples/id" but I am not sure whether that means the lane ID.

    Thanks.

    opened by 18627242758 2
  • Congratulations

    Congratulations

    Привет Кирилл,

    Я вижу что ты разбираешься в машинном обучении. Я хотел бы сконнектиться и перенять опыт, если возможно. Как с тобой можно связаться?

    Дмитрий

    opened by Rendok 1
  • I can't download the dataset

    I can't download the dataset

    The infomation is showed as below: Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission.

    opened by fengsky401 1
  • socket.gaierror: [Errno -2] Name or service not known

    socket.gaierror: [Errno -2] Name or service not known

    After running the following command :

    (venvpy37cu10) [[email protected] project]$ python train.py --train-data ./train --dev-data ./dev --save ./xception71 --model xception71 --img-res 224 --in-channels 25 --time-limit 80 --n-traj 6 --lr 0.001 --batch-size 48 --n-epochs 120

    Below error has occurred :

    Screen Shot 2022-07-30 at 10 59 36 PM
    opened by rohansd 1
  • magic_const and shift

    magic_const and shift

    Hi,

    Thanks for your open-source work.

    I don't understand the magic_const and shift in rasterize() function prerender.py.

    Would you please give some explanation?

    opened by ShoufaChen 2
  • Preprocessing issue

    Preprocessing issue

    I tried running with all requirements , it keeps on reading the records but when it tries to write down .....here it fails , please help me to proceed further

    [email protected]:/app/waymo-adas-main/waymo-motion-prediction-2021# python3 prerender.py --data /app/waymo-adas-main/waymo-dataset/original/validation/ --out /app/waymo-adas-main/data/train1
    False
    Namespace(data='/app/waymo-adas-main/waymo-dataset/original/validation/', each=0, n_jobs=20, n_shards=8, no_valid=False, out='/app/waymo-adas-main/data/train1', use_vectorize=False)
    1215it [00:28, 42.50it/s]
      0%|                                                                                                                                                                                   | 0/1215 [00:00<?, ?it/s]
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "prerender.py", line 731, in merge
        parsed = tf.io.parse_single_example(data, features_description)
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
        tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    """
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "prerender.py", line 836, in <module>
        main()
      File "prerender.py", line 832, in main
        r.get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    
    opened by karanveersingh5623 9
  • About preprocessing

    About preprocessing

    Congrats! Actually I have a question which has confused me a few days. When we rasterize the map information, why we need to shift and rotate the local coordinate system that the target agent is located at a specific location? I konw that many papers has used this method like buliding a relative coordinate system and the center is target agent. why? and what is the meaning of "to eliminate the redundant degrees of freedom" in your report? Thank you in advance!

    opened by zyandtom 1
Releases(0.1)
Official repo for QHack—the quantum machine learning hackathon

Note: This repository has been frozen while we consider the submissions for the QHack Open Hackathon. We hope you enjoyed the event! Welcome to QHack,

Xanadu 118 Jan 05, 2023
Code to reproduce the results in the paper "Tensor Component Analysis for Interpreting the Latent Space of GANs".

Tensor Component Analysis for Interpreting the Latent Space of GANs [ paper | project page ] Code to reproduce the results in the paper "Tensor Compon

James Oldfield 4 Jun 17, 2022
A Deep Learning Based Knowledge Extraction Toolkit for Knowledge Base Population

DeepKE is a knowledge extraction toolkit supporting low-resource and document-level scenarios for entity, relation and attribute extraction. We provide comprehensive documents, Google Colab tutorials

ZJUNLP 1.6k Jan 05, 2023
Fibonacci Method Gradient Descent

An implementation of the Fibonacci method for gradient descent, featuring a TKinter GUI for inputting the function / parameters to be examined and a matplotlib plot of the function and results.

Emma 1 Jan 28, 2022
Fastshap: A fast, approximate shap kernel

fastshap: A fast, approximate shap kernel fastshap was designed to be: Fast Calculating shap values can take an extremely long time. fastshap utilizes

Samuel Wilson 22 Sep 24, 2022
Multi-Modal Machine Learning toolkit based on PyTorch.

简体中文 | English TorchMM 简介 多模态学习工具包 TorchMM 旨在于提供模态联合学习和跨模态学习算法模型库,为处理图片文本等多模态数据提供高效的解决方案,助力多模态学习应用落地。 近期更新 2022.1.5 发布 TorchMM 初始版本 v1.0 特性 丰富的任务场景:工具

njustkmg 1 Jan 05, 2022
D2Go is a toolkit for efficient deep learning

D2Go D2Go is a production ready software system from FacebookResearch, which supports end-to-end model training and deployment for mobile platforms. W

Facebook Research 744 Jan 04, 2023
Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes

Neural Scene Flow Fields PyTorch implementation of paper "Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenes", CVPR 2021 [Projec

Zhengqi Li 583 Dec 30, 2022
Source code for the ACL-IJCNLP 2021 paper entitled "T-DNA: Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation" by Shizhe Diao et al.

T-DNA Source code for the ACL-IJCNLP 2021 paper entitled Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adapta

shizhediao 17 Dec 22, 2022
Created as part of CS50 AI's coursework. This AI makes use of knowledge entailment to calculate the best probabilities to win Minesweeper.

Minesweeper-AI Created as part of CS50 AI's coursework. This AI makes use of knowledge entailment to calculate the best probabilities to win Minesweep

Beckham 0 Jul 20, 2022
This is the code for Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning

This is the code for Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning It includes /bert, which is the original BERT repos

Mitchell Gordon 11 Nov 15, 2022
Rational Activation Functions - Replacing Padé Activation Units

Rational Activations - Learnable Rational Activation Functions First introduce as PAU in Padé Activation Units: End-to-end Learning of Activation Func

<a href=[email protected]"> 38 Nov 22, 2022
Learning trajectory representations using self-supervision and programmatic supervision.

Trajectory Embedding for Behavior Analysis (TREBA) Implementation from the paper: Jennifer J. Sun, Ann Kennedy, Eric Zhan, David J. Anderson, Yisong Y

58 Jan 06, 2023
CellRank's reproducibility repository.

CellRank's reproducibility repository We believe that reproducibility is key and have made it as simple as possible to reproduce our results. Please e

Theis Lab 8 Oct 08, 2022
FEDn is an open-source, modular and ML-framework agnostic framework for Federated Machine Learning

FEDn is an open-source, modular and ML-framework agnostic framework for Federated Machine Learning (FedML) developed and maintained by Scaleout Systems. FEDn enables highly scalable cross-silo and cr

Scaleout 75 Nov 09, 2022
MonoScene: Monocular 3D Semantic Scene Completion

MonoScene: Monocular 3D Semantic Scene Completion MonoScene: Monocular 3D Semantic Scene Completion] [arXiv + supp] | [Project page] Anh-Quan Cao, Rao

298 Jan 08, 2023
Supplemental Code for "ImpressionNet :A Multi view Approach to Predict Socio Facial Impressions"

Supplemental Code for "ImpressionNet :A Multi view Approach to Predict Socio Facial Impressions" Environment requirement This code is based on Python

Rohan Kumar Gupta 1 Dec 19, 2021
An open framework for Federated Learning.

Welcome to Intel® Open Federated Learning Federated learning is a distributed machine learning approach that enables organizations to collaborate on m

Intel Corporation 397 Dec 27, 2022
Robbing the FED: Directly Obtaining Private Data in Federated Learning with Modified Models

Robbing the FED: Directly Obtaining Private Data in Federated Learning with Modified Models This repo contains a barebones implementation for the atta

16 Dec 04, 2022
This repository contains code from the paper "TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network"

TTS-GAN: A Transformer-based Time-Series Generative Adversarial Network This repository contains code from the paper "TTS-GAN: A Transformer-based Tim

Intelligent Multimodal Computing and Sensing Laboratory (IMICS Lab) - Texas State University 108 Dec 29, 2022