Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

Overview

LiDAR fog simulation

PWC

Created by Martin Hahner at the Computer Vision Lab of ETH Zurich.

This is the official code release of the paper
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather
by Martin Hahner, Christos Sakaridis, Dengxin Dai, and Luc van Gool, accepted at ICCV 2021.

Please visit our paper website for more details.

pointcloud_viewer

Overview

.
├── file_lists                          # contains file lists for pointcloud_viewer.py
│   └── ...
├── integral_lookup_tables              # contains lookup tables to speed up the fog simulation
│   └── ... 
├── extract_fog.py                      # to extract real fog noise* from the SeeingThroughFog dataset
├── fog_simulation.py                   # to augment a clear weather pointcloud with artificial fog (used during training)
├── generate_integral_lookup_table.py   # to precompute the integral inside the fog equation
├── pointcloud_viewer.py                # to visualize entire point clouds of different datasets with the option to augment fog into their scenes
├── README.md
└── theory.py                           # to visualize the theory behind a single LiDAR beam in foggy conditions

* Contains returns not only from fog, but also from physical objects that are closeby.

Datasets supported by pointcloud_viewer.py:

License

This software is made available for non-commercial use under a Creative Commons License.
A summary of the license can be found here.

Acknowledgments

This work is supported by Toyota via the TRACE project.

Furthermore, we would like to thank the authors of SeeingThroughFog for their great work.
In this repository, we use a fork of their original repository to visualize annotations and compare to their fog simulation. Their code is licensed via the MIT License.

Citation

If you find this work useful, please consider citing our paper.

@inproceedings{HahnerICCV21,
  author = {Hahner, Martin and Sakaridis, Christos and Dai, Dengxin and Van Gool, Luc},
  title = {Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather},
  booktitle = {IEEE International Conference on Computer Vision (ICCV)},
  year = {2021},
}

Getting Started

Setup

  1. Install anaconda.

  2. Create a new conda environment.

conda create --name foggy_lidar python=3.9 -y
  1. Activate the newly created conda environment.
conda activate foggy_lidar
  1. Install all necessary packages.
conda install matplotlib numpy opencv pandas plyfile pyopengl pyqt pyqtgraph quaternion scipy tqdm -c conda-forge -y
pip install pyquaternion
  1. Clone this repository (including submodules).
git clone [email protected]:MartinHahner/LiDAR_fog_sim.git --recursive
cd LiDAR_fog_sim

Usage

How to run the script that visualizes the theory behind a single LiDAR beam in foggy conditions:

python theory.py

theory

How to run the script that visualizes entire point clouds of different datasets:

python pointcloud_viewer.py -d <path_to_where_you_store_your_datasets>

Note:

You may also have to adjust the relative paths in pointcloud_viewer.py (right at the beginning of the file) to be compatible with your datasets relative folder structure.

Disclaimer

The code has been successfully tested on

  • Ubuntu 18.04.5 LTS
  • macOS Big Sur 11.2.1
  • Debian GNU/Linux 9.13

using conda 4.9.2.

Contributions

Please feel free to suggest improvements to this repository.
We are always open to merge usefull pull request.

Model-based 3D Hand Reconstruction via Self-Supervised Learning, CVPR2021

S2HAND: Model-based 3D Hand Reconstruction via Self-Supervised Learning S2HAND presents a self-supervised 3D hand reconstruction network that can join

Yujin Chen 72 Dec 12, 2022
Predict halo masses from simulations via graph neural networks

HaloGraphNet Predict halo masses from simulations via Graph Neural Networks. Given a dark matter halo and its galaxies, creates a graph with informati

Pablo Villanueva Domingo 20 Nov 15, 2022
Code release for NeRF (Neural Radiance Fields)

NeRF: Neural Radiance Fields Project Page | Video | Paper | Data Tensorflow implementation of optimizing a neural representation for a single scene an

6.5k Jan 01, 2023
Fully Convolutional DenseNet (A.K.A 100 layer tiramisu) for semantic segmentation of images implemented in TensorFlow.

FC-DenseNet-Tensorflow This is a re-implementation of the 100 layer tiramisu, technically a fully convolutional DenseNet, in TensorFlow (Tiramisu). Th

Hasnain Raza 121 Oct 12, 2022
Face recognition with trained classifiers for detecting objects using OpenCV

Face_Detector Face recognition with trained classifiers for detecting objects using OpenCV Libraries required to be installed using pip Command: cv2 n

Chumui Tripura 0 Oct 31, 2021
Applying curriculum to meta-learning for few shot classification

Curriculum Meta-Learning for Few-shot Classification We propose an adaptation of the curriculum training framework, applicable to state-of-the-art met

Stergiadis Manos 3 Oct 25, 2022
RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation

Multipath RefineNet A MATLAB based framework for semantic image segmentation and general dense prediction tasks on images. This is the source code for

Guosheng Lin 575 Dec 06, 2022
Codes to calculate solar-sensor zenith and azimuth angles directly from hyperspectral images collected by UAV. Works only for UAVs that have high resolution GNSS/IMU unit.

UAV Solar-Sensor Angle Calculation Table of Contents About The Project Built With Getting Started Prerequisites Installation Datasets Contributing Lic

Sourav Bhadra 1 Jan 15, 2022
v objective diffusion inference code for JAX.

v-diffusion-jax v objective diffusion inference code for JAX, by Katherine Crowson (@RiversHaveWings) and Chainbreakers AI (@jd_pressman). The models

Katherine Crowson 186 Dec 21, 2022
AI Face Mesh: This is a simple face mesh detection program based on Artificial intelligence.

AI Face Mesh: This is a simple face mesh detection program based on Artificial Intelligence which made with Python. It's able to detect 468 different

Md. Rakibul Islam 1 Jan 13, 2022
Annotated notes and summaries of the TensorFlow white paper, along with SVG figures and links to documentation

TensorFlow White Paper Notes Features Notes broken down section by section, as well as subsection by subsection Relevant links to documentation, resou

Sam Abrahams 437 Oct 09, 2022
領域を指定し、キーを入力することで画像を保存するツールです。クラス分類用のデータセット作成を想定しています。

image-capture-class-annotation 領域を指定し、キーを入力することで画像を保存するツールです。 クラス分類用のデータセット作成を想定しています。 Requirement OpenCV 3.4.2 or later Usage 実行方法は以下です。 起動後はマウスクリック4

KazuhitoTakahashi 5 May 28, 2021
NHS AI Lab Skunkworks project: Long Stayer Risk Stratification

NHS AI Lab Skunkworks project: Long Stayer Risk Stratification A pilot project for the NHS AI Lab Skunkworks team, Long Stayer Risk Stratification use

NHSX 21 Nov 14, 2022
Multi-Objective Reinforced Active Learning

Multi-Objective Reinforced Active Learning Dependencies wandb tqdm pytorch = 1.7.0 numpy = 1.20.0 scipy = 1.1.0 pycolab == 1.2 Weights and Biases O

Markus Peschl 6 Nov 19, 2022
SigOpt wrappers for scikit-learn methods

SigOpt + scikit-learn Interfacing This package implements useful interfaces and wrappers for using SigOpt and scikit-learn together Getting Started In

SigOpt 73 Sep 30, 2022
PyTorch implementation of Self-supervised Contrastive Regularization for DG (SelfReg)

SelfReg PyTorch official implementation of Self-supervised Contrastive Regularization for Domain Generalization (SelfReg, https://arxiv.org/abs/2104.0

64 Dec 16, 2022
NBEATSx: Neural basis expansion analysis with exogenous variables

NBEATSx: Neural basis expansion analysis with exogenous variables We extend the NBEATS model to incorporate exogenous factors. The resulting method, c

Cristian Challu 100 Dec 31, 2022
Image Recognition using Pytorch

PyTorch Project Template A simple and well designed structure is essential for any Deep Learning project, so after a lot practice and contributing in

Sarat Chinni 1 Nov 02, 2021
Pytorch implemenation of Stochastic Multi-Label Image-to-image Translation (SMIT)

SMIT: Stochastic Multi-Label Image-to-image Translation This repository provides a PyTorch implementation of SMIT. SMIT can stochastically translate a

Biomedical Computer Vision Group @ Uniandes 37 Mar 01, 2022
Code repository accompanying the paper "On Adversarial Robustness: A Neural Architecture Search perspective"

On Adversarial Robustness: A Neural Architecture Search perspective Preparation: Clone the repository: https://github.com/tdchaitanya/nas-robustness.g

Chaitanya Devaguptapu 4 Nov 10, 2022