FAIMED 3D

use fastai to quickly train fully three-dimensional models on radiological data

Classification

from faimed3d.all import *

Load data in various medical formats (DICOM, NIfTI, NRRD) or even videos as simple as in fastai.

d = pd.read_csv('../data/radiopaedia_cases.csv')
dls = ImageDataLoaders3D.from_df(d,
                                 item_tfms = Resize3D((20, 112, 112)),
                                 batch_tfms = aug_transforms_3d(), 
                                 bs = 2, val_bs = 2)

Faimed3d provides multiple model architectures, pretrained on the UCF101 dataset for action recoginiton, which can be used for transfer learning.

Model	3-fold accuracy	duration/epoch	model size
efficientnet b0	92.5 %	9M:35S	48.8 MB
efficientnet b1	90.1 %	13M:20S	80.5 MB
resnet 18	87.6 %	6M:57S	339.1 MB
resnet 50	94.8 %	12M:16S	561.2 MB
resnet 101	96.0 %	17M:20S	1,030 MB

# slow
learn = cnn_learner_3d(dls, efficientnet_b0)

# slow
learn.lr_find()

SuggestedLRs(lr_min=0.014454397559165954, lr_steep=6.309573450380412e-07)

Click here for a more in-depth classification example.

Segmentation

dls = SegmentationDataLoaders3D.from_df(d,
                                 item_tfms = Resize3D((20, 112, 112)),
                                 batch_tfms = aug_transforms_3d(), 
                                 bs = 2, val_bs = 2)

All models in faimed3d can be used as a backbone for U-Nets, even with pre-trained weights.

# slow
learn = unet_learner_3d(dls, efficientnet_b0, n_out = 2)

# slow 
learn.lr_find()

SuggestedLRs(lr_min=0.33113112449646, lr_steep=0.10000000149011612)

Click here for a more in-depth segmentation example.

Extension to fastai for volumetric medical data

Related tags

Overview

FAIMED 3D

Classification

Segmentation

Owner

Keno

Multi-modal Content Creation Model Training Infrastructure including the FACT model (AI Choreographer) implementation.

Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Clouds (CVPR 2022, Oral)

Retinal Vessel Segmentation with Pixel-wise Adaptive Filters (ISBI 2022)

Deep Reinforcement Learning based Trading Agent for Bitcoin

[ECE NTUA] 👁 Computer Vision - Lab Projects & Theoretical Problem Sets (2020-2021)

Example repository for custom C++/CUDA operators for TorchScript

An Efficient Implementation of Analytic Mesh Algorithm for 3D Iso-surface Extraction from Neural Networks

[CVPR 2020] Transform and Tell: Entity-Aware News Image Captioning

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

A parallel framework for population-based multi-agent reinforcement learning.

Transformers4Rec is a flexible and efficient library for sequential and session-based recommendation, available for both PyTorch and Tensorflow.

(Arxiv 2021) NeRF--: Neural Radiance Fields Without Known Camera Parameters

GT China coal model

This implements one of result networks from Large-scale evolution of image classifiers

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification

Any-to-any voice conversion using synthetic specific-speaker speeches as intermedium features

Cross-view Transformers for real-time Map-view Semantic Segmentation (CVPR 2022 Oral)

Official implementation for TTT++: When Does Self-supervised Test-time Training Fail or Thrive

Multimodal Descriptions of Social Concepts: Automatic Modeling and Detection of (Highly Abstract) Social Concepts evoked by Art Images

[ICCV 2021] FaPN: Feature-aligned Pyramid Network for Dense Image Prediction