Trainable Bilateral Filter Layer (PyTorch)

Last update: Dec 25, 2022

Overview

Trainable Bilateral Filter Layer (PyTorch)

This repository contains our GPU-accelerated trainable bilateral filter layer (three spatial and one range filter dimension) that can be directly included in any Pytorch graph, just as any conventional layer (FCL, CNN, ...). By calculating the analytical derivative of the bilateral filter with respect to its parameters and its input, the (so far) hyperparameters can be automatically optimized via backpropagation for a calculated loss.

Our corresponding paper Ultra low-parameter denoising: Trainable bilateral filter layers in computed tomography can be found on arXiv.

Implementation:

The general structure of the implementation follows the PyTorch documentation for creating custom C++ and CUDA extensions. The forward pass implementation of the layer is based on code from the Project MONAI framework, originally published under the Apache License, Version 2.0. The correct implementation of the analytical forward and backward pass can be verified by running the gradcheck.py script, comparing numerical gradients with the derived analytical gradient using the PyTorch built-in gradcheck function.

Setup:

The C++/CUDA implemented forward and backward functions are compiled via the setup.py script using setuptools:

Create and activate a python environment (python>=3.7).
Install Torch (tested versions: 1.7.1, 1.9.0).
Navigate into the extracted repo.
Compile/install the bilateral filter layer by calling

python setup.py install

Example scripts:

Try out the forward pass by running the example_filter.py (requires Matplotlib and scikit-image).
Run the gradcheck.py script to verify the correct gradient implementation.
Run example_optimization.py to optimize the parameters of a bilateral filter layer to automatically denoise an image.

Optimized bilateral filter prediction:

Citation:

If you find our code useful, please cite our work

@article{wagner2022ultra,
  title={Ultra low-parameter denoising: Trainable bilateral filter layers in computed tomography},
  author={Wagner, Fabian and Thies, Mareike and Gu, Mingxuan and Huang, Yixing and Pechmann, Sabrina and Patwari, Mayank and Ploner, Stefan and Aust, Oliver and Uderhardt, Stefan and Schett, Georg and Christiansen, Silke and Maier, Andreas},
  journal={arXiv preprint arXiv:2201.10345},
  year={2022}
}

You might also like...

📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

HaloNet - Pytorch Implementation of the Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones. This re

189 Nov 22, 2022

Comments

3D case example

Hi, I am grateful that you have shared such awesome codes. I have downloaded the codes and tested 2D case. It works well. But when I used 3D images, the results seemed weird. As shown below.

The input 3D image is like this:

Could you provide 3D demo case? Thank you very much!

opened by cs123951 2

Trainable Bilateral Filter Layer (PyTorch)

Related tags

Overview

Trainable Bilateral Filter Layer (PyTorch)

Implementation:

Setup:

Example scripts:

Optimized bilateral filter prediction:

Citation:

You might also like...

📦 PyTorch based visualization package for generating layer-wise explanations for CNNs.

PyTorch Implementation of NCSOFT's FastPitchFormant: Source-filter based Decomposed Modeling for Speech Synthesis

[CVPR 2021] Official PyTorch Implementation for "Iterative Filter Adaptive Network for Single Image Defocus Deblurring"

Pytorch Implementation of Continual Learning With Filter Atom Swapping (ICLR'22 Spolight) Paper

Predictive AI layer for existing databases.

Predictive AI layer for existing databases.

An abstraction layer for mathematical optimization solvers.

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021)

Implementation of the 😇 Attention layer from the paper, Scaling Local Self-Attention For Parameter Efficient Visual Backbones

Comments

3D case example

Releases(1.1.0)

1.1.0(Aug 1, 2022)

Owner

FabianWagner

Code for reproducing our paper: LMSOC: An Approach for Socially Sensitive Pretraining

GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond

FastReID is a research platform that implements state-of-the-art re-identification algorithms.

[NeurIPS 2021] Garment4D: Garment Reconstruction from Point Cloud Sequences

Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently

PyTorch implementation of paper "IBRNet: Learning Multi-View Image-Based Rendering", CVPR 2021.

A curated list of programmatic weak supervision papers and resources

Code for Overinterpretation paper Overinterpretation reveals image classification model pathologies

Code release for "Self-Tuning for Data-Efficient Deep Learning" (ICML 2021)

Code for EMNLP2021 paper "Allocating Large Vocabulary Capacity for Cross-lingual Language Model Pre-training"

Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation (CVPR 2021)

Official Implementation of LARGE: Latent-Based Regression through GAN Semantics

The code of “Similarity Reasoning and Filtration for Image-Text Matching” [AAAI2021]

Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

T2F: text to face generation using Deep Learning

DyNet: The Dynamic Neural Network Toolkit

Motion planning environment for Sampling-based Planners

Official code release for "GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis"

Predicting Axillary Lymph Node Metastasis in Early Breast Cancer Using Deep Learning on Primary Tumor Biopsy Slides

Semantic graph parser based on Categorial grammars