Official page of Struct-MDC (RA-L'22 with IROS'22 option); Depth completion from Visual-SLAM using point & line features

Overview

Struct-MDC

video

journal arxiv

(click the above buttons for redirection!)


Official page of "Struct-MDC: Mesh-Refined Unsupervised Depth Completion Leveraging Structural Regularities from Visual SLAM", which is accepted in IEEE RA-L'22 (IROS'22 are still being under-reviewed.)

  • Depth completion from Visual(-inertial) SLAM using point & line features.

README & code & Dataset are still being edited.

  • Code (including source code, utility code for visualization) & Dataset will be finalized & released soon! (goal: I'm still organizing the code structure, until publish date)
  • version info
    • (04/20) docker image has been uploaded.
    • (04/21) Dataset has been uploaded.
    • (04/21) Visusal-SLAM module (modified UV-SLAM) has been uploaded.



Results

  • 3D Depth estimation results
    • VOID (left three columns) and NYUv2 (right three columns)
    • detected features (top row), estimation from baseline (middle row) and ours (bottom row)

  • 2D Depth estimation results
Ground truth Baseline Struct-MDC (Ours)



Installation

1. Prerequisites (we've validated our code in the following environment!)

  • Common
  • Visual-SLAM module
    • OpenCV 3.2.0 (under 3.4.1)
    • Ceres Solver-1.14.0
    • Eigen-3.3.9
    • CDT library
      git clone https://github.com/artem-ogre/CDT.git
      cd CDT
      mkdir build && cd build
      cmake -DCDT_USE_AS_COMPILED_LIBRARY=ON -DCDT_USE_BOOST=ON ..
      cmake --build . && cmake --install .
      sudo make install
      
  • Depth completion module
    • Python 3.7.7
    • PyTorch 1.5.0 (you can easily reproduce equivalent environment using our docker image)

2. Build

  • Visual-SLAM module

    • As visual-SLAM, we modified the UV-SLAM, which is implemented in ROS environment.
    • make sure that your catkin workspace has following cmake args: -DCMAKE_BUILD_TYPE=Release
    cd ~/$(PATH_TO_YOUR_ROS_WORKSPACE)/src
    git clone --recursive https://github.com/url-kaist/Struct-MDC
    cd ..
    catkin build
    source ~/$(PATH_TO_YOUR_ROS_WORKSPACE)/devel/setup.bash
    
  • Depth completion module

    • Our depth compeltion module is based on the popular Deep-Learning framework, PyTorch.
    • For your convenience, we share our environment as Docker image. We assume that you have already installed the Docker. For Docker installation, please refer here
    # pull our docker image into your local machine
    docker pull zinuok/nvidia-torch:latest
    
    # run the image mounting our source
    docker run -it --gpus "device=0" -v $(PATH_TO_YOUR_LOCAL_FOLER):/workspace zinuok/nvidia-torch:latest bash
    

3. Trouble shooting

  • any issues found will be updated in this section.
  • if you've found any other issues, please post it on Issues tab. We'll do our best to resolve your issues.
Owner
Urban Robotics Lab. @ KAIST
Urban Robotics Lab. @ KAIST
Cryptocurrency Prediction with Artificial Intelligence (Deep Learning via LSTM Neural Networks)

Cryptocurrency Prediction with Artificial Intelligence (Deep Learning via LSTM Neural Networks)- Emirhan BULUT

Emirhan BULUT 102 Nov 18, 2022
A deep learning CNN model to identify and classify and check if a person is wearing a mask or not.

Face Mask Detection The Model is designed to check if any human is wearing a mask or not. Dataset Description The Dataset contains a total of 11,792 i

1 Mar 01, 2022
The Curious Layperson: Fine-Grained Image Recognition without Expert Labels (BMVC 2021)

The Curious Layperson: Fine-Grained Image Recognition without Expert Labels Subhabrata Choudhury, Iro Laina, Christian Rupprecht, Andrea Vedaldi Code

Subhabrata Choudhury 18 Dec 27, 2022
hipCaffe: the HIP port of Caffe

Caffe Caffe is a deep learning framework made with expression, speed, and modularity in mind. It is developed by the Berkeley Vision and Learning Cent

ROCm Software Platform 126 Dec 05, 2022
Adversarial Learning for Modeling Human Motion

Adversarial Learning for Modeling Human Motion This repository contains the open source code which reproduces the results for the paper: Adversarial l

wangqi 6 Jun 15, 2021
A curated list of awesome Model-Based RL resources

Awesome Model-Based Reinforcement Learning This is a collection of research papers for model-based reinforcement learning (mbrl). And the repository w

OpenDILab 427 Jan 03, 2023
Neural Network Libraries

Neural Network Libraries Neural Network Libraries is a deep learning framework that is intended to be used for research, development and production. W

Sony 2.6k Dec 30, 2022
AI assistant built in python.the features are it can display time,say weather,open-google,youtube,instagram.

AI assistant built in python.the features are it can display time,say weather,open-google,youtube,instagram.

AK-Shanmugananthan 1 Nov 29, 2021
B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search

B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search This is the offical implementation of the

SNU ADSL 0 Feb 07, 2022
Generative Flow Networks for Discrete Probabilistic Modeling

Energy-based GFlowNets Code for Generative Flow Networks for Discrete Probabilistic Modeling by Dinghuai Zhang, Nikolay Malkin, Zhen Liu, Alexandra Vo

Narsil-Dinghuai Zhang 51 Dec 20, 2022
DNA sequence classification by Deep Neural Network

DNA sequence classification by Deep Neural Network: Project Overview worked on the DNA sequence classification problem where the input is the DNA sequ

Mohammed Jawwadul Islam Fida 0 Aug 02, 2022
Removing Inter-Experimental Variability from Functional Data in Systems Neuroscience

Removing Inter-Experimental Variability from Functional Data in Systems Neuroscience This repository is the official implementation of [https://www.bi

Eulerlab 6 Oct 09, 2022
Decentralized Reinforcment Learning: Global Decision-Making via Local Economic Transactions (ICML 2020)

Decentralized Reinforcement Learning This is the code complementing the paper Decentralized Reinforcment Learning: Global Decision-Making via Local Ec

40 Oct 30, 2022
TF Image Segmentation: Image Segmentation framework

TF Image Segmentation: Image Segmentation framework The aim of the TF Image Segmentation framework is to provide/provide a simplified way for: Convert

Daniil Pakhomov 546 Dec 17, 2022
The code for paper "Learning Implicit Fields for Generative Shape Modeling".

implicit-decoder The tensorflow code for paper "Learning Implicit Fields for Generative Shape Modeling", Zhiqin Chen, Hao (Richard) Zhang. Project pag

Zhiqin Chen 353 Dec 30, 2022
pytorch implementation of GPV-Pose

GPV-Pose Pytorch implementation of GPV-Pose: Category-level Object Pose Estimation via Geometry-guided Point-wise Voting. (link) UPDATE A new version

40 Dec 01, 2022
Code for "Single-view robot pose and joint angle estimation via render & compare", CVPR 2021 (Oral).

Single-view robot pose and joint angle estimation via render & compare Yann Labbé, Justin Carpentier, Mathieu Aubry, Josef Sivic CVPR: Conference on C

Yann Labbé 51 Oct 14, 2022
Official Implementation of VAT

Semantic correspondence Few-shot segmentation Cost Aggregation Is All You Need for Few-Shot Segmentation For more information, check out project [Proj

Hamacojr 114 Dec 27, 2022
Solution to the Weather4cast 2021 challenge

This code was used for the entry by the team "antfugue" for the Weather4cast 2021 Challenge. Below, you can find the instructions for generating predi

Jussi Leinonen 13 Jan 03, 2023
IEEE Winter Conference on Applications of Computer Vision 2022 Accepted

SSKT(Accepted WACV2022) Concept map Dataset Image dataset CIFAR10 (torchvision) CIFAR100 (torchvision) STL10 (torchvision) Pascal VOC (torchvision) Im

1 Nov 17, 2022