A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes

Overview

Awesome-LiDAR-Camera-Calibration

Awesome

A Collection of LiDAR-Camera-Calibration Papers, Toolboxes and Notes.

Outline

0. Introduction

For applications such as autonomous driving, robotics, navigation systems, and 3-D scene reconstruction, data of the same scene is often captured using both lidar and camera sensors. To accurately interpret the objects in a scene, it is necessary to fuse the lidar and the camera outputs together. Lidar camera calibration estimates a rigid transformation matrix (extrinsics, rotation+translation, 6 DoF) that establishes the correspondences between the points in the 3-D lidar plane and the pixels in the image plane.

Example

1. Target-based methods

Paper Target Feature Optimization Toolbox Note
Extrinsic Calibration of a Camera and Laser Range Finder (improves camera calibration), 2004 checkerboard C:Plane (a), L: pts in plane (m) point-to-plane CamLaserCalibraTool CN
Fast Extrinsic Calibration of a Laser Rangefinder to a Camera, 2005 checkerboard C: Plane (a), L: Plane (m) plane(n/d) correspondence, point-to-plane LCCT *
Extrinsic calibration of a 3D laser scanner and an omnidirectional camera, 2010 checkerboard C: plane (a), L: pts in plane (m) point-to-plane cam_lidar_calib *
LiDAR-Camera Calibration using 3D-3D Point correspondences, 2017 cardboard + ArUco C: 3D corners (a), L: 3D corners (m) ICP lidar_camera_calibration *
Reflectance Intensity Assisted Automatic and Accurate Extrinsic Calibration of 3D LiDAR and Panoramic Camera Using a Printed Chessboard, 2017 checkerboard C: 2D corners (a), L: 3D corners (a) PnP, angle difference ILCC *
Extrinsic Calibration of Lidar and Camera with Polygon, 2018 regular cardboard C: 2D edge, corners (a), L: 3D edge, pts in plane (a) point-to-line, point-inside-polygon ram-lab/plycal *
Automatic Extrinsic Calibration of a Camera and a 3D LiDAR using Line and Plane Correspondences, 2018 checkerboard C: 3D edge, plane(a), L: 3D edge, pts in plane (a) direcion/normal, point-to-line, point-to-plane Matlab LiDAR Toolbox *
Improvements to Target-Based 3D LiDAR to Camera Calibration, 2020 cardboard with ArUco C: 2d corners (a), L: 3D corners (a) PnP, IOU github *
ACSC: Automatic Calibration for Non-repetitive Scanning Solid-State LiDAR and Camera Systems, 2020 checkerboard C: 2D corners (a), L: 3D corners (a) PnP ACSC *
Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups, 2021 cardboard with circle & Aruco C: 3D points (a), L: 3D points (a) ICP velo2cam_ calibration *

C: camera, L: LiDAR, a: automaic, m: manual

2. Targetless methods

2.1. Motion-based methods

Paper Feature Optimization Toolbox Note
LiDAR and Camera Calibration Using Motions Estimated by Sensor Fusion Odometry, 2018 C: motion (ICP), L: motion (VO) hand-eye calibration * *

2.2. Scene-based methods

2.2.1. Traditional methods

Paper Feature Optimization Toolbox Note
Automatic Targetless Extrinsic Calibration of a 3D Lidar and Camera by Maximizing Mutual Information, 2012 C:grayscale, L: reflectivity mutual information, BB steepest gradient ascent Extrinsic Calib *
Automatic Calibration of Lidar and Camera Images using Normalized Mutual Information, 2013 C:grayscale, L: reflectivity, noraml normalized MI, particle swarm * *
Automatic Online Calibration of Cameras and Lasers, 2013 C: Canny edge, L: depth-discontinuous edge correlation, grid search * *
SOIC: Semantic Online Initialization and Calibration for LiDAR and Camera, 2020 semantic centroid PnP * *
A Low-cost and Accurate Lidar-assisted Visual SLAM System, 2021 C: edge(grayscale), L: edge (reflectivity, depth projection) ICP, coordinate descent algorithms CamVox *
Pixel-level Extrinsic Self Calibration of High Resolution LiDAR and Camera in Targetless Environments,2021 C:Canny edge(grayscale), L: depth-continuous edge point-to-line, Gaussian-Newton livox_camera_calib *
CRLF: Automatic Calibration and Refinement based on Line Feature for LiDAR and Camera in Road Scenes, 2021 C:straight line, L: straight line perspective3-lines (P3L) * CN

2.2.2. Deep-learning methods

Pape Toolbox Note
RegNet: Multimodal sensor registration using deep neural networks, 2017,IV regnet *
CalibNet: Geometrically supervised extrinsic calibration using 3d spatial transformer networks,2018,IROS CalibNet *

3. Other toolboxes

Toolbox Introduction Note
Apollo sensor calibration tools targetless method, no source code CN
Autoware camera lidar calibrator pick points mannually, PnP *
Autoware calibration camera lidar checkerboard, similar to LCCT CN
livox_camera_lidar_calibration pick points mannually, PnP *
All materials of Cassandra Event, Udyam'22

Cassandra 2022 Workspace Workshop Materials Workshop-1 Workshop-2 Workshop-3 Workshop-4 Assignments Assignment-1 Assignment-2 Assignment-3 Resources P

36 Dec 31, 2022
The source code for Adaptive Kernel Graph Neural Network at AAAI2022

AKGNN The source code for Adaptive Kernel Graph Neural Network at AAAI2022. Please cite our paper if you think our work is helpful to you: @inproceedi

11 Nov 25, 2022
LibFewShot: A Comprehensive Library for Few-shot Learning.

LibFewShot Make few-shot learning easy. Supported Methods Meta MAML(ICML'17) ANIL(ICLR'20) R2D2(ICLR'19) Versa(NeurIPS'18) LEO(ICLR'19) MTL(CVPR'19) M

<a href=[email protected]&L"> 603 Jan 05, 2023
[NeurIPS 2020] This project provides a strong single-stage baseline for Long-Tailed Classification, Detection, and Instance Segmentation (LVIS).

A Strong Single-Stage Baseline for Long-Tailed Problems This project provides a strong single-stage baseline for Long-Tailed Classification (under Ima

Kaihua Tang 514 Dec 23, 2022
Multi-Template Mouse Brain MRI Atlas (MBMA): both in-vivo and ex-vivo

Multi-template MRI mouse brain atlas (both in vivo and ex vivo) Mouse Brain MRI atlas (both in-vivo and ex-vivo) (repository relocated from the origin

8 Nov 18, 2022
🔪 Elimination based Lightweight Neural Net with Pretrained Weights

ELimNet ELimNet: Eliminating Layers in a Neural Network Pretrained with Large Dataset for Downstream Task Removed top layers from pretrained Efficient

snoop2head 4 Jul 12, 2022
Supervised domain-agnostic prediction framework for probabilistic modelling

A supervised domain-agnostic framework that allows for probabilistic modelling, namely the prediction of probability distributions for individual data

The Alan Turing Institute 112 Oct 23, 2022
Official Pytorch implementation of paper "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Generated Images"

Reverse_Engineering_GMs Official Pytorch implementation of paper "Reverse Engineering of Generative Models: Inferring Model Hyperparameters from Gener

100 Dec 18, 2022
Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

Distributed Evolutionary Algorithms in Python 4.9k Jan 05, 2023
PyTorch implementation for 3D human pose estimation

Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach This repository is the PyTorch implementation for the network presented in:

Xingyi Zhou 579 Dec 22, 2022
[ICCV '21] In this repository you find the code to our paper Keypoint Communities

Keypoint Communities In this repository you will find the code to our ICCV '21 paper: Keypoint Communities Duncan Zauss, Sven Kreiss, Alexandre Alahi,

Duncan Zauss 262 Dec 13, 2022
QR2Pass-project - A proof of concept for an alternative (passwordless) authentication system to a web server

QR2Pass This is a proof of concept for an alternative (passwordless) authenticat

4 Dec 09, 2022
Cross-Document Coreference Resolution

Cross-Document Coreference Resolution This repository contains code and models for end-to-end cross-document coreference resolution, as decribed in ou

Arie Cattan 29 Nov 28, 2022
An end-to-end framework for mixed-integer optimization with data-driven learned constraints.

OptiCL OptiCL is an end-to-end framework for mixed-integer optimization (MIO) with data-driven learned constraints. We address a problem setting in wh

Holly Wiberg 57 Dec 26, 2022
ICML 21 - Voice2Series: Reprogramming Acoustic Models for Time Series Classification

Voice2Series-Reprogramming Voice2Series: Reprogramming Acoustic Models for Time Series Classification International Conference on Machine Learning (IC

49 Jan 03, 2023
ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representation from common sense knowledge graphs.

ZSL-KG is a general-purpose zero-shot learning framework with a novel transformer graph convolutional network (TrGCN) to learn class representa

Bats Research 94 Nov 21, 2022
Pixel Consensus Voting for Panoptic Segmentation (CVPR 2020)

Implementation for Pixel Consensus Voting (CVPR 2020). This codebase contains the essential ingredients of PCV, including various spatial discretizati

Haochen 23 Oct 25, 2022
A simple but complete full-attention transformer with a set of promising experimental features from various papers

x-transformers A concise but fully-featured transformer, complete with a set of promising experimental features from various papers. Install $ pip ins

Phil Wang 2.3k Jan 03, 2023
R-package accompanying the paper "Dynamic Factor Model for Functional Time Series: Identification, Estimation, and Prediction"

dffm The goal of dffm is to provide functionality to apply the methods developed in the paper “Dynamic Factor Model for Functional Time Series: Identi

Sven Otto 3 Dec 09, 2022