Set of models for classifcation of 3D volumes

Last update: Dec 28, 2022

Related tags

Overview

Classification models 3D Zoo - Keras and TF.Keras

This repository contains 3D variants of popular CNN models for classification like ResNets, DenseNets, VGG, etc. It also contains weights obtained by converting ImageNet weights from the same 2D models.

This repository is based on great classification_models repo by @qubvel

Architectures:

VGG [16, 19]
ResNet [18, 34, 50, 101, 152]
ResNeXt [50, 101]
SE-ResNet [18, 34, 50, 101, 152]
SE-ResNeXt [50, 101]
SE-Net [154]
DenseNet [121, 169, 201]
Inception ResNet V2
Inception V3
MobileNet
MobileNet v2

Installation

pip install classification-models-3D

Examples

Loading model with `imagenet` weights:

# for keras
from classification_models_3D.keras import Classifiers

# for tensorflow.keras
# from classification_models_3D.tfkeras import Classifiers

ResNet18, preprocess_input = Classifiers.get('resnet18')
model = ResNet18(input_shape=(128, 128, 128, 3), weights='imagenet')

All possible nets for Classifiers.get() method: 'resnet18, 'resnet34', 'resnet50', 'resnet101', 'resnet152', 'seresnet18', 'seresnet34', 'seresnet50', 'seresnet101', 'seresnet152', 'seresnext50', 'seresnext101', 'senet154', 'resnext50', 'resnext101', 'vgg16', 'vgg19', 'densenet121', 'densenet169', 'densenet201', 'inceptionresnetv2', 'inceptionv3', 'mobilenet', 'mobilenetv2'

Convert imagenet weights (2D -> 3D)

Code to convert 2D imagenet weights to 3D variant is available here: convert_imagenet_weights_to_3D_models.py. Weights were obtained with TF2, but works OK with Keras + TF1 as well.

How to choose input shape

If initial 2D model had shape (512, 512, 3) then you can use shape (D, H, W, 3) where D * H * W ~= 512*512, so something like (64, 64, 64, 3) will be ok.

Training with single NVIDIA 1080Ti (11 GB) worked with:

DenseNet121, DenseNet169 and ResNet50 with shape (96, 128, 128, 3) and batch size 6
DenseNet201 with shape (96, 128, 128, 3) and batch size 5
ResNet18 with shape (128, 160, 160, 3) and batch size 6

Related repositories

https://github.com/qubvel/classification_models - original 2D repo
volumentations - 3D augmentations
segmentation models 3D - models for segmentation in 3D
driven_data_repo - code for training and inference on real dataset

Unresolved problems

There is no DepthwiseConv3D layer in keras, so repo used custom layer from this repo by @alexandrosstergiou which can be slower than native implementation.
There is no imagenet weights for 'inceptionresnetv2' and 'inceptionv3'.

Description

This code was used to get 1st place in DrivenData: Advance Alzheimer’s Research with Stall Catchers competition.

More details on ArXiv: https://arxiv.org/abs/2104.01687

Citation

If you find this code useful, please cite it as:

@InProceedings{RSolovyev_2021_stalled,
  author = {Solovyev, Roman and Kalinin, Alexandr A. and Gabruseva, Tatiana},
  title = {3D Convolutional Neural Networks for Stalled Brain Capillary Detection},
  booktitle = {Arxiv: 2104.01687},
  month = {April},
  year = {2021}
}

Comments

Update __init__.py

Using keras 2.9.0, import keras_applications as ka gives the following error:- ModuleNotFoundError: No module named 'keras_applications'

Instead using from keras import applications as ka works!

opened by msmuskan 0
Pushing current version to PyPI

Hello @ZFTurbo,

if you have time, please push the current updated status (with ConvNeXt) of this repo to PyPI. :)

Thanks again for the great work and your time!

Cheers, Dominik

opened by muellerdo 0
Grad cam issue

Hello ,

base_model, preprocess_input = Classifiers.get('seresnext50') model = base_model(input_shape=(512, 512, 20, 1 ), weights=None , include_top = False ) x = Flatten()(model.output) x = Dense(1024, activation= 'sigmoid')(x) x = Dense(2, activation= 'sigmoid')(x)

Trying to train a model , the accuracy is everything resides upto expectation, but the gradcam are quite off from the region of the focus - how the accuracy is good but the grad cam is off the focus of targeted area .

Using the layer - 'activation-161' as output ref - https://github.com/fitushar/3D-Grad-CAM/blob/master/3DGrad-CAM.ipynb for the gradcam generation code , the results are always at the border of the image.

opened by ntirupathirao18 0
ImportError: cannot import name 'VersionAwareLayers' from 'keras.layers'

Thank you for the great work.

I am experiencing the following error over and over, even though I created a brand new tensorflow environment and installed all the necessary libraries in it. Could you please have a look on it and guide me how do I solve this problem? Thank you.

ImportError: Unable to import 'VersionAwareLayers' from 'keras.layers' (/home/ubuntu/anaconda3/envs/cm_3d/lib/python3.7/site-packages/keras/layers/init.py)

opened by nasir3843 2
3D DenseNet

Hello and sorry to bother you beforehand,

I am currently conducting my master thesis project and I am trying to implement a 3D DenseNet-121 with knee MRIs as input data. While I was searching on how to implement a 3D version of the DenseNet I came across your repository and tried to change it for my application.

I have some issues regarding my try and I didn't know where else to ask about it and again I am sorry if I am completely of topic asking them here.

Firstly, my input shapes are (250,320,18,1) and when I give them as input to the 3D DenseNet I developed with stride_size=1 for my Conv_block and pooling_size=(2,2,2) and strides=(2,2,1) for my AveragePooling3D layer in the transition block, the model is constructed properly with the specific input_size, while when I am trying to load a DenseNet121 from classification_models_3d.tfkeras classifiers I am unable to construct it with input_shape(250,320,18,1), stride_size=1 and kernel_size=2. It gives as an error "Negative dimension size... for node pool4_pool/AvgPool3D". Is there a way to specifically define the strides for AvgPool3D layer in the transition block?

And secondly, I was thinking to load the 3D weights to my 3D DenseNet 121, is there a folder in your repository where I can find your pre-trained weights on imagenet??

Again thank you for having this repository publicly available and sorry if I am completely of topic asking such things here.

I look forward for you answer, Kind regards, Anastasis

opened by alexopoulosanastasis 4

What are the limitations on Inceptionv3 input shape?

I seem to always get this error when I try to create InceptionV3 model no matter what input_shape. What are the limitations on input shape there?

InvalidArgumentError: Negative dimension size caused by subtracting 3 from 2 for '{{node conv3d_314/Conv3D}} = 
Conv3D[T=DT_FLOAT, data_format="NDHWC", dilations=[1, 1, 1, 1, 1], padding="VALID", strides=[1, 2, 2, 2, 1]](Placeholder, 
conv3d_314/Conv3D/ReadVariableOp)' with input shapes: [?,2,17,17,192], [3,3,3,192,320].

opened by mazatov 0

Releases(v1.0.4)

Owner

GitHub Repository

[ICLR'21] FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization This is the PyTorch implemention of our paper FedBN: Federated Learning on

[email protected]"> 156 Dec 15, 2022

A Deep Learning based project for creating line art portraits.

ArtLine The main aim of the project is to create amazing line art portraits. Sounds Intresting,let's get to the pictures!! Model-(Smooth) Model-(Quali

3.3k Jan 07, 2023

Code for 1st place solution in Sleep AI Challenge SNU Hospital

Sleep AI Challenge SNU Hospital 2021 Code for 1st place solution for Sleep AI Challenge (Note that the code is not fully organized) Refer to the notio

13 Jan 03, 2022

Code for "Learning the Best Pooling Strategy for Visual Semantic Embedding", CVPR 2021

Learning the Best Pooling Strategy for Visual Semantic Embedding Official PyTorch implementation of the paper Learning the Best Pooling Strategy for V

106 Jan 06, 2023

This repository is an implementation of paper : Improving the Training of Graph Neural Networks with Consistency Regularization

CRGNN Paper ： Improving the Training of Graph Neural Networks with Consistency Regularization Environments Implementing environment: GeForce RTX™ 3090

28 Dec 09, 2022

Learning to Draw: Emergent Communication through Sketching

Learning to Draw: Emergent Communication through Sketching This is the official code for the paper "Learning to Draw: Emergent Communication through S

19 Jul 22, 2022

PyTorch code for EMNLP 2021 paper: Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System

25 Sep 06, 2022

Animate molecular orbital transitions using Psi4 and Blender

Molecular Orbital Transitions (MOT) Animate molecular orbital transitions using Psi4 and Blender Author: Maximilian Paradiz Dominguez, University of A

3 Feb 01, 2022

UNAVOIDS: Unsupervised and Nonparametric Approach for Visualizing Outliers and Invariant Detection Scoring

UNAVOIDS: Unsupervised and Nonparametric Approach for Visualizing Outliers and Invariant Detection Scoring Code Summary aggregate.py: this script aggr

1 Dec 28, 2021

Demonstrates how to divide a DL model into multiple IR model files (division) and introduce a simplest way to implement a custom layer works with OpenVINO IR models.

Demonstration of OpenVINO techniques - Model-division and a simplest-way to support custom layers Description: Model Optimizer in Intel(r) OpenVINO(tm

12 Nov 09, 2022

Multi-Anchor Active Domain Adaptation for Semantic Segmentation (ICCV 2021 Oral)

Multi-Anchor Active Domain Adaptation for Semantic Segmentation Munan Ning*, Donghuan Lu*, Dong Wei†, Cheng Bian, Chenglang Yuan, Shuang Yu, Kai Ma, Y

36 Dec 07, 2022

Random Forests for Regression with Missing Entries

Random Forests for Regression with Missing Entries These are specific codes used in the article: On the Consistency of a Random Forest Algorithm in th

1 Nov 15, 2021

A Pytorch Implementation of a continuously rate adjustable learned image compression framework.

GainedVAE A Pytorch Implementation of a continuously rate adjustable learned image compression framework, Gained Variational Autoencoder(GainedVAE). N

39 Dec 24, 2022

Code for "Learning Graph Cellular Automata"

Learning Graph Cellular Automata This code implements the experiments from the NeurIPS 2021 paper: "Learning Graph Cellular Automata" Daniele Grattaro

37 Oct 26, 2022

A graph neural network (GNN) model to predict protein-protein interactions (PPI) with no sample features

2 Jul 25, 2022

GAN encoders in PyTorch that could match PGGAN, StyleGAN v1/v2, and BigGAN. Code also integrates the implementation of these GANs.

MTV-TSA: Adaptable GAN Encoders for Image Reconstruction via Multi-type Latent Vectors with Two-scale Attentions. This is the official code release fo

37 Dec 24, 2022