Code for paper "ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation"

Related tags

Deep LearningASAP-Net
Overview

ASAP-Net

This project implements ASAP-Net of paper ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation (BMVC2020).

Semantic segmentation result on SemanticKITTI

Overview

We improve spatio-temporal point cloud feature learning with a flexible module called ASAP module considering both attention and structure information across frames, which can be combined with different backbones. Incorporating our module into backbones brings semantic segmentation performance improvements on both Synthia and SemanticKITTI datasets (+3.4 to +15.2 mIoU points with different backbones).

Installation

The Synthia experiments is implemented with TensorFlow and the SemanticKITTI experiments is implemented with PyTorch. We tested the codes under TensorFlow 1.13.1 GPU version, PyTorch 1.1.0, CUDA 10.0, g++ 5.4.0 and Python 3.6.9 on Ubuntu 16.04.12 with TITAN RTX GPU. For SemanticKITTI experiments, you should have a GPU memory of at least 16GB.

Compile TF Operators for Synthia Experiments

We use the implementation in xingyul/meteornet. Please follow the instructions below.

The TF operators are included under Synthia_experiments/tf_ops, you need to compile them first by make under each ops subfolder (check Makefile) or directly use the following commands:

cd Synthia_experiments
sh command_make.sh

Please update arch in the Makefiles for different CUDA Compute Capability that suits your GPU if necessary.

Compile Torch Operators for SemanticKITTI Experiments

We use the PoinNet++ implementation in sshaoshuai/Pointnet2.PyTorch. Use the commands below to build Torch operators.

cd SemanticKITTI_experiments/ASAP-Net_PointNet2/pointnet2
python setup.py install

Experiments on Synthia

The codes for experiments on Synthia is in Synthia_experiments/semantic_seg_synthia. Please refer to Synthia_experiments/semantic_seg_synthia/README.md for more information on data preprocessing and running instructions.

Experiments on SemanticKITTI

The SemanticKITTI_experiments/ImageSet2 folder contains dataset split information. Please put it under your semanticKITTI dataset like Path to semanticKITTI dataset/dataset/sequences.

PointNet++ as Backbone

The codes for framework with PointNet++ as Backbone is in SemanticKITTI_experiments/ASAP-Net_PointNet2. Please refer to SemanticKITTI_experiments/ASAP-Net_PointNet2/README.md for more information on running instructions.

SqueezeSegV2 as Backbone

The codes for framework with SqueezeSegV2 as Backbone is in SemanticKITTI_experiments/ASAP-Net_SqueezeSegV2. Please refer to SemanticKITTI_experiments/ASAP-Net_SqueezeSegV2/README.md for more information on running instructions.

Acknowledgements

Special thanks for open source codes including xingyul/meteornet, sshaoshuai/Pointnet2.PyTorch and PRBonn/lidar-bonnetal.

Citation

Please cite these papers in your publications if it helps your research:

@article{caoasap,
  title={ASAP-Net: Attention and Structure Aware Point Cloud Sequence Segmentation},
  author={Cao, Hanwen and Lu, Yongyi and Lu, Cewu and Pang, Bo and Liu, Gongshen and Yuille, Alan}
  booktitle={British Machine Vision Conference (BMVC)},
  year={2020}
}
Owner
Hanwen Cao
Ph.D. candidate at University of California, San Diego (UCSD)
Hanwen Cao
A Pythonic library for Nvidia Codec.

A Pythonic library for Nvidia Codec. The project is still in active development; expect breaking changes. Why another Python library for Nvidia Codec?

Zesen Qian 12 Dec 27, 2022
Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Neural Networks.

Dynamic-Graphs-Construction Official Codes for Graph Modularity:Towards Understanding the Cross-Layer Transition of Feature Representations in Deep Ne

11 Dec 14, 2022
Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation

SUCP Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation () Direct Friends (i.e., users who follow each o

Kosar 8 Nov 26, 2022
Code for our TKDE paper "Understanding WeChat User Preferences and “Wow” Diffusion"

wechat-wow-analysis Understanding WeChat User Preferences and “Wow” Diffusion. Fanjin Zhang, Jie Tang, Xueyi Liu, Zhenyu Hou, Yuxiao Dong, Jing Zhang,

18 Sep 16, 2022
code for Fast Point Cloud Registration with Optimal Transport

robot This is the repository for the paper "Accurate Point Cloud Registration with Robust Optimal Transport". We are in the process of refactoring the

28 Jan 04, 2023
Implementation of "Efficient Regional Memory Network for Video Object Segmentation" (Xie et al., CVPR 2021).

RMNet This repository contains the source code for the paper Efficient Regional Memory Network for Video Object Segmentation. Cite this work @inprocee

Haozhe Xie 76 Dec 14, 2022
Pytorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

Parallel Tacotron2 Pytorch Implementation of Google's Parallel Tacotron 2: A Non-Autoregressive Neural TTS Model with Differentiable Duration Modeling

Keon Lee 170 Dec 27, 2022
A complete end-to-end demonstration in which we collect training data in Unity and use that data to train a deep neural network to predict the pose of a cube. This model is then deployed in a simulated robotic pick-and-place task.

Object Pose Estimation Demo This tutorial will go through the steps necessary to perform pose estimation with a UR3 robotic arm in Unity. You’ll gain

Unity Technologies 187 Dec 24, 2022
Polyp-PVT: Polyp Segmentation with Pyramid Vision Transformers (arXiv2021)

Polyp-PVT by Bo Dong, Wenhai Wang, Deng-Ping Fan, Jinpeng Li, Huazhu Fu, & Ling Shao. This repo is the official implementation of "Polyp-PVT: Polyp Se

Deng-Ping Fan 102 Jan 05, 2023
A framework for GPU based high-performance medical image processing and visualization

FAST is an open-source cross-platform framework with the main goal of making it easier to do high-performance processing and visualization of medical images on heterogeneous systems utilizing both mu

Erik Smistad 315 Dec 30, 2022
[CVPR'21] Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration

Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registration This repository contains the implementation of our paper Locally Aware Pi

sfwang 70 Dec 19, 2022
A PyTorch implementation of "From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network" (ICCV2021)

From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network The official code of VisionLAN (ICCV2021). VisionLAN successfully a

81 Dec 12, 2022
Implementation for Stankevičiūtė et al. "Conformal time-series forecasting", NeurIPS 2021.

Conformal time-series forecasting Implementation for Stankevičiūtė et al. "Conformal time-series forecasting", NeurIPS 2021. If you use our code in yo

Kamilė Stankevičiūtė 36 Nov 21, 2022
DNA-RECON { Automatic Web Reconnaissance Tool }

ABOUT TOOL : DNA-RECON is an automatic web reconnaissance tool written in python. This tool made for reconnaissance and information gathering with an

NIKUNJ BHATT 25 Aug 11, 2021
Anderson Acceleration for Deep Learning

Anderson Accelerated Deep Learning (AADL) AADL is a Python package that implements the Anderson acceleration to speed-up the training of deep learning

Oak Ridge National Laboratory 7 Nov 24, 2022
An implementation of Equivariant e2 convolutional kernals into a convolutional self attention network, applied to radio astronomy data.

EquivariantSelfAttention An implementation of Equivariant e2 convolutional kernals into a convolutional self attention network, applied to radio astro

2 Nov 09, 2021
Unet network with mean teacher for altrasound image segmentation

Unet network with mean teacher for altrasound image segmentation

5 Nov 21, 2022
A Python module for the generation and training of an entry-level feedforward neural network.

ff-neural-network A Python module for the generation and training of an entry-level feedforward neural network. This repository serves as a repurposin

Riadh 2 Jan 31, 2022
Train Scene Graph Generation for Visual Genome and GQA in PyTorch >= 1.2 with improved zero and few-shot generalization.

Scene Graph Generation Object Detections Ground truth Scene Graph Generated Scene Graph In this visualization, woman sitting on rock is a zero-shot tr

Boris Knyazev 93 Dec 28, 2022
Repo for CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning

CReST in Tensorflow 2 Code for the paper: "CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning" by Chen Wei, Ki

Google Research 75 Nov 01, 2022