Waymo motion prediction challenge 2021: 3rd place solution

Overview

Waymo motion prediction challenge 2021: 3rd place solution

header

Team behind this solution:

  1. Artsiom Sanakoyeu [Homepage] [Twitter] [Telegram Channel] [LinkedIn]
  2. Stepan Konev [LinkedIn]
  3. Kirill Brodt [GitHub]

Dataset

Download datasets uncompressed/tf_example/{training,validation,testing}

Prerender

Change paths to input dataset and output folders

python prerender.py \
    --data /home/data/waymo/training \
    --out ./train
    
python prerender.py \
    --data /home/data/waymo/validation \
    --out ./dev \
    --use-vectorize \
    --n-shards 1
    
python prerender.py \
    --data /home/data/waymo/testing \
    --out ./test \
    --use-vectorize \
    --n-shards 1

Training

MODEL_NAME=xception71
python train.py \
    --train-data ./train \
    --dev-data ./dev \
    --save ./${MODEL_NAME} \
    --model ${MODEL_NAME} \
    --img-res 224 \
    --in-channels 25 \
    --time-limit 80 \
    --n-traj 6 \
    --lr 0.001 \
    --batch-size 48 \
    --n-epochs 120

Submit

python submit.py \
    --test-data ./test/ \
    --model-path ${MODEL_PATH_TO_JIT} \
    --save ${SAVE}

Visualize predictions

python visualize.py \
    --model ${MODEL_PATH_TO_JIT} \
    --data ${DATA_PATH} \
    --save ./viz

Citation

If you find our work useful, please cite it as:

@article{konev2021motioncnn,
  title={MotionCNN: A Strong Baseline for Motion Prediction in Autonomous Driving},
  author={Konev, Stepan and Brodt, Kirill and Sanakoyeu, Artsiom},
  year={2021}
}

Related repos

Comments
  • Metrics

    Metrics

    Hi! @kbrodt Thanks for sharing this great code!

    Where are the codes of the evaluation metrics (for example: ADE, FDE, minADE, minFDE and so on)? Or where can I find it?

    Looking forward to your reply!

    opened by chx-Github 9
  • Regarding the training epochs

    Regarding the training epochs

    Thanks for sharing this awesome codes!

    Is it necessary to train the model for 120 epochs? Since there are more than 1M training samples. Can you share some performance during the training progress? Such as the performance with 30epochs, 60 epochs, 90 epochs? Since I trained it for several epochs but the loss is still very large.

    To double check the training process, can you share how many training samples for each epoch?

    Thanks so much!

    opened by FutureOpenAI 8
  • Loss

    Loss

    Hi! I tried your method and I observed that in training, l2loss and log_softmax have so large difference. so my network does not learn multimodal tracks, only one best track is fitted. Do you have any solution?

    opened by zsgj-Xxx 7
  • Angle conversion error at prerender.py

    Angle conversion error at prerender.py

    Thank you for amazing works.

    I found the conversion error at prerender.py

    https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L706 https://github.com/kbrodt/waymo-motion-prediction-2021/blob/3665d4b3ba39a6b879b663747f93b6e525018c00/prerender.py#L707

    These two lines has conversion error.

    In order to convert coordinate properly, it should be changed like this tmp[3] = other_v_yaw - ANGLE tmp[4] = other_bbox_yaw - ANGLE

    opened by KyuhwanYeon 3
  • Question About the waymo dataset lane type

    Question About the waymo dataset lane type

    Hi, @kbrodt,

    Thank you for your great work. I am new to this area, so I have a question regarding to the lane type of waymo dataset. In general, the lanes in waymo dataset, can be broadly categorized into 6: lanecenters, roadlines, stopsign, speedbump, roadedge and crosswalks. So, is the lane centers are just the the center of each line in which the vehicles can drive on? Theoretically, a car's center should be on the lanecenters if this car is staying in this lane. Is my interpretation right? Also, is the lanecenters in waymo the same as the centerlines in Argoverse Dataset? Are you familiar with that dataset? I am looking forward to your reply. Thank you in advance.

    opened by SwagJ 2
  • Vehicles driving on the same lane

    Vehicles driving on the same lane

    Hello!

    Thanks for sharing you work!

    I have one questions: from the motion dataset, is that possible to know if two vehicles are driving on the same lane? I notice that there is a column called "roadgraph_samples/id" but I am not sure whether that means the lane ID.

    Thanks.

    opened by 18627242758 2
  • Congratulations

    Congratulations

    Привет Кирилл,

    Я вижу что ты разбираешься в машинном обучении. Я хотел бы сконнектиться и перенять опыт, если возможно. Как с тобой можно связаться?

    Дмитрий

    opened by Rendok 1
  • I can't download the dataset

    I can't download the dataset

    The infomation is showed as below: Additional permissions required to list objects in this bucket. Ask a bucket owner to grant you 'storage.objects.list' permission.

    opened by fengsky401 1
  • socket.gaierror: [Errno -2] Name or service not known

    socket.gaierror: [Errno -2] Name or service not known

    After running the following command :

    (venvpy37cu10) [[email protected] project]$ python train.py --train-data ./train --dev-data ./dev --save ./xception71 --model xception71 --img-res 224 --in-channels 25 --time-limit 80 --n-traj 6 --lr 0.001 --batch-size 48 --n-epochs 120

    Below error has occurred :

    Screen Shot 2022-07-30 at 10 59 36 PM
    opened by rohansd 1
  • magic_const and shift

    magic_const and shift

    Hi,

    Thanks for your open-source work.

    I don't understand the magic_const and shift in rasterize() function prerender.py.

    Would you please give some explanation?

    opened by ShoufaChen 2
  • Preprocessing issue

    Preprocessing issue

    I tried running with all requirements , it keeps on reading the records but when it tries to write down .....here it fails , please help me to proceed further

    [email protected]:/app/waymo-adas-main/waymo-motion-prediction-2021# python3 prerender.py --data /app/waymo-adas-main/waymo-dataset/original/validation/ --out /app/waymo-adas-main/data/train1
    False
    Namespace(data='/app/waymo-adas-main/waymo-dataset/original/validation/', each=0, n_jobs=20, n_shards=8, no_valid=False, out='/app/waymo-adas-main/data/train1', use_vectorize=False)
    1215it [00:28, 42.50it/s]
      0%|                                                                                                                                                                                   | 0/1215 [00:00<?, ?it/s]
    multiprocessing.pool.RemoteTraceback:
    """
    Traceback (most recent call last):
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 125, in worker
        result = (True, func(*args, **kwds))
      File "prerender.py", line 731, in merge
        parsed = tf.io.parse_single_example(data, features_description)
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py", line 153, in error_handler
        raise e.with_traceback(filtered_tb) from None
      File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/eager/execute.py", line 58, in quick_execute
        tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    """
    
    The above exception was the direct cause of the following exception:
    
    Traceback (most recent call last):
      File "prerender.py", line 836, in <module>
        main()
      File "prerender.py", line 832, in main
        r.get()
      File "/usr/lib/python3.8/multiprocessing/pool.py", line 771, in get
        raise self._value
    UnicodeDecodeError: 'utf-8' codec can't decode byte 0x96 in position 40: invalid start byte
    
    opened by karanveersingh5623 9
  • About preprocessing

    About preprocessing

    Congrats! Actually I have a question which has confused me a few days. When we rasterize the map information, why we need to shift and rotate the local coordinate system that the target agent is located at a specific location? I konw that many papers has used this method like buliding a relative coordinate system and the center is target agent. why? and what is the meaning of "to eliminate the redundant degrees of freedom" in your report? Thank you in advance!

    opened by zyandtom 1
Releases(0.1)
Little tool in python to watch anime from the terminal (the better way to watch anime)

ani-cli Script working again :), thanks to the fork by Dink4n for the alternative approach to by pass the captcha on gogoanime A cli to browse and wat

Harshith 4.5k Dec 31, 2022
[NeurIPS 2021] “Improving Contrastive Learning on Imbalanced Data via Open-World Sampling”,

Improving Contrastive Learning on Imbalanced Data via Open-World Sampling Introduction Contrastive learning approaches have achieved great success in

VITA 24 Dec 17, 2022
Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition - NeurIPS2021

Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition Project Page | Video | Paper Implementation for Neural-PIL. A novel method wh

Computergraphics (University of Tübingen) 64 Dec 29, 2022
An original implementation of "Noisy Channel Language Model Prompting for Few-Shot Text Classification"

Channel LM Prompting (and beyond) This includes an original implementation of Sewon Min, Mike Lewis, Hannaneh Hajishirzi, Luke Zettlemoyer. "Noisy Cha

Sewon Min 92 Jan 07, 2023
Paddle implementation for "Highly Efficient Knowledge Graph Embedding Learning with Closed-Form Orthogonal Procrustes Analysis" (NAACL 2021)

ProcrustEs-KGE Paddle implementation for Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis 🙈 A more detailed re

Lincedo Lab 4 Jun 09, 2021
This repository contains FEDOT - an open-source framework for automated modeling and machine learning (AutoML)

package tests docs license stats support This repository contains FEDOT - an open-source framework for automated modeling and machine learning (AutoML

National Center for Cognitive Research of ITMO University 482 Dec 26, 2022
Weakly Supervised Posture Mining with Reverse Cross-entropy for Fine-grained Classification

Fine-grainedImageClassification Weakly Supervised Posture Mining with Reverse Cross-entropy for Fine-grained Classification We trained model here: lin

ZhenchaoTang 14 Oct 21, 2022
YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone

YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyone In our recent paper we propose the YourTTS model. YourTTS bri

Edresson Casanova 390 Dec 29, 2022
Create Data & AI apps in 20 lines of code with Shimoku

Install with: pip install shimoku-api-python Start with: from os import getenv import shimoku_api_python.client as Shimoku

Shimoku 5 Nov 07, 2022
An implementation of IMLE-Net: An Interpretable Multi-level Multi-channel Model for ECG Classification

IMLE-Net: An Interpretable Multi-level Multi-channel Model for ECG Classification The repostiory consists of the code, results and data set links for

12 Dec 26, 2022
All materials of Cassandra Event, Udyam'22

Cassandra 2022 Workspace Workshop Materials Workshop-1 Workshop-2 Workshop-3 Workshop-4 Assignments Assignment-1 Assignment-2 Assignment-3 Resources P

36 Dec 31, 2022
Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

Modeling Category-Selective Cortical Regions with Topographic Variational Autoencoders

1 Oct 11, 2021
Official implementation of the paper "Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering"

Light Field Networks Project Page | Paper | Data | Pretrained Models Vincent Sitzmann*, Semon Rezchikov*, William Freeman, Joshua Tenenbaum, Frédo Dur

Vincent Sitzmann 130 Dec 29, 2022
Using pytorch to implement unet network for liver image segmentation.

Using pytorch to implement unet network for liver image segmentation.

zxq 1 Dec 17, 2021
Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically.

Experimenting with computer vision techniques to generate annotated image datasets from gameplay recordings automatically. The collected data will then be used to train a deep neural network that can

Martin Valchev 3 Apr 24, 2022
Code accompanying "Evolving spiking neuron cellular automata and networks to emulate in vitro neuronal activity," accepted to IEEE SSCI ICES 2021

Evolving-spiking-neuron-cellular-automata-and-networks-to-emulate-in-vitro-neuronal-activity Code accompanying "Evolving spiking neuron cellular autom

SOCRATES: Self-Organizing Computational substRATES 2 Dec 02, 2022
Voxel-based Network for Shape Completion by Leveraging Edge Generation (ICCV 2021, oral)

Voxel-based Network for Shape Completion by Leveraging Edge Generation This is the PyTorch implementation for the paper "Voxel-based Network for Shape

10 Dec 04, 2022
Pretrained models for Jax/Haiku; MobileNet, ResNet, VGG, Xception.

Pre-trained image classification models for Jax/Haiku Jax/Haiku Applications are deep learning models that are made available alongside pre-trained we

Alper Baris CELIK 14 Dec 20, 2022
Miscellaneous and lightweight network tools

Network Tools Collection of miscellaneous and lightweight network tools to simplify daily operations, administration, and troubleshooting of networks.

Nicholas Russo 22 Mar 22, 2022
Gray Zone Assessment

Gray Zone Assessment Get started Clone github repository git clone https://github.com/andreanne-lemay/gray_zone_assessment.git Build docker image dock

1 Jan 08, 2022