Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Last update: Dec 27, 2022

Related tags

Deep Learning SW-CV-ModelZoo

Overview

SW-CV-ModelZoo

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Framework: TF/Keras 2.7

Training SQLite DB built using fire-egg's tools: https://github.com/fire-eggs/Danbooru2019

Currently training on Danbooru2021, 512px SFW subset (sans the rating:q images that had been included in the 2022-01-21 release of the dataset)

Reference:

Anonymous, The Danbooru Community, & Gwern Branwen; “Danbooru2021: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”, 2022-01-21. Web. Accessed 2022-01-28 https://www.gwern.net/Danbooru2021

Journal

06/02/2022: great news crew! TRC allowed me to use a bunch of TPUs!

To make better use of this amount of compute I had to overhaul a number of components, so a bunch of things are likely to have fallen to bitrot in the process. I can only guarantee NFNet can work pretty much as before with the right arguments.
NFResNet changes should have left it retrocompatible with the previous version.
ResNet has been streamlined to be mostly in line with the Bag-of-Tricks paper (arXiv:1812.01187) with the exception of the stem. It is not compatible with the previous version of the code.

The training labels have been included in the 2021_0000_0899 folder for convenience.
The list of files used for training is going to be uploaded as a GitHub Release.

Now for some numbers:
compared to my previous best run, the one that resulted in NFNetL1V1-100-0.57141:

I'm using 1.86x the amount of images: 2.8M vs 1.5M
I'm training bigger models: 61M vs 45M params
... in less time: 232 vs 700 hours of processor time
don't get me started on actual wall clock time
with a few amenities thrown in: ECA for channel attention, SiLU activation

And it's all thanks to the folks at TRC, so shout out to them!

I currently have a few runs in progress across a couple of dimensions:

effect of model size with NFNet L0/L1/L2, with SiLU and ECA for all three of them
effect of activation function with NFNet L0, with SiLU/HSwish/ReLU, no ECA

Once the experiments are over, the plan is to select the network definitions that lay on the Pareto curve between throughput and F1 score and release the trained weights.

One last thing.
I'd like to call your attention to the tools/cleanlab_stuff.py script.
It reads two files: one with the binarized labels from the database, the other with the predicted probabilities.
It then uses the cleanlab package to estimate whether if an image in a set could be missing a given label. At the end it stores its conclusions in a json file.
This file could, potentially, be used in some tool to assist human intervention to add the missing tags.

You might also like...

Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

71 Jan 5, 2023

Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

2.2k Jan 8, 2023

QKeras: a quantization deep learning library for Tensorflow Keras

QKeras github.com/google/qkeras QKeras 0.8 highlights: Automatic quantization using QKeras; Stochastic behavior (including stochastic rouding) is disa

437 Jan 3, 2023

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

Hyperparameter Optimization for Keras Talos • Key Features • Examples • Install • Support • Docs • Issues • License • Download Talos radically changes

1.6k Dec 15, 2022

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

5.7k Jan 9, 2023

Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

107 Nov 2, 2022

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

1.8k Jan 8, 2023

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image

102 Dec 17, 2022

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

1.5k Jan 3, 2023

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

ConvNext B, ViT B16
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
top 5500 tags (2021_0000_0899_5500/selected_tags.csv)
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_09_25_2022_05h13m55s | B | 93.2M | 448 | 0.3673 | 0.6941 | | ViTB16_09_25_2022_04h53m38s | B16 | 90.5M | 448 | 0.3663 | 0.6918 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_09_25_2022_05h13m55s.7z(322.58 MB)
ViTB16_09_25_2022_04h53m38s.7z(312.96 MB)
convnexts_db2021_2022_03_22(Mar 22, 2022)

ConvNext, T/S/B
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_03_10_2022_21h41m23s | B | 90.01M | 448 | 0.3372 | 0.6892 | | ConvNextSV1_03_11_2022_17h49m56s | S | 51.28M | 384 | 0.3301 | 0.6798 | | ConvNextTV1_03_05_2022_15h56m42s | T | 29.65M | 320 | 0.3259 | 0.6595 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_03_10_2022_21h41m23s.7z(311.29 MB)
ConvNextSV1_03_11_2022_17h49m56s.7z(177.36 MB)
ConvNextTV1_03_05_2022_15h56m42s.7z(102.96 MB)
nfnets_db2021_2022_03_04(Mar 4, 2022)

NFNet, L0/L1/L2 (based on timm Lx model definitions) Trained on Danbooru2021 512px SFW subset, modulos 0000-0899 alpha to white padding to make the image square is white channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | NFNetL2V1_02_20_2022_10h27m08s | L2 | 60.96M | 448 | 0.3231 | 0.6785 | | NFNetL1V1_02_17_2022_20h18m38s | L1 | 45.65M | 384 | 0.3259 | 0.6691 | | NFNetL0V1_02_10_2022_17h50m14s | L0 | 27.32M | 320 | 0.3190 | 0.6509 |
Source code(tar.gz)
Source code(zip)
NFNetL0V1_02_10_2022_17h50m14s.7z(94.98 MB)
NFNetL1V1_02_17_2022_20h18m38s.7z(157.97 MB)
NFNetL2V1_02_20_2022_10h27m08s.7z(210.49 MB)
nfnet_tpu_training(Feb 6, 2022)

Attaching the lists of files currently in use for training and validation, along with the selected labels.

Training and validation files have been selected from the 512px SFW subset.
Source code(tar.gz)
Source code(zip)
2021_0000_0899.7z(1.32 MB)
NFNetL1V1-100-0.57141(Dec 31, 2021)
NFNet, L1 (based on timm Lx model definitions), 100 epochs, F1 @ 0.4 at the end of the 100th epoch was 0.57141

Trained on Danbooru2020 512px SFW subset, modulos 0000-0599

320px per side

alpha to white

padding to make the image square is white

channel order is BGR, scaled to 0-1

mixup alpha = 0.2 during epochs 76-100

analyze_metrics on Danbooru2020 original set, modulos 0984-0999: {'thres': 0.3485, 'F1': 0.6133, 'F2': 0.6133, 'MCC': 0.6094, 'A': 0.9923, 'R': 0.6133, 'P': 0.6133}

analyze_metrics on image IDs 4970000-5000000: {'thres': 0.3148, 'F1': 0.5942, 'F2': 0.5941, 'MCC': 0.5892, 'A': 0.9901, 'R': 0.5940, 'P': 0.5943}

Source code(tar.gz)
Source code(zip)
NFNetL1V1-100-0.57141.7z(158.09 MB)

Owner

GitHub Repository

Styled text-to-drawing synthesis method. Featured at the 2021 NeurIPS Workshop on Machine Learning for Creativity and Design

247 Dec 23, 2022

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

HiSD: Image-to-image Translation via Hierarchical Style Disentanglement Official pytorch implementation of paper "Image-to-image Translation

364 Dec 14, 2022

Source code and notebooks to reproduce experiments and benchmarks on Bias Faces in the Wild (BFW).

Face Recognition: Too Bias, or Not Too Bias? Robinson, Joseph P., Gennady Livitz, Yann Henon, Can Qin, Yun Fu, and Samson Timoner. "Face recognition:

41 Dec 12, 2022

Using image super resolution models with vapoursynth and speeding them up with TensorRT

vs-RealEsrganAnime-tensorrt-docker Using image super resolution models with vapoursynth and speeding them up with TensorRT. Also a docker image since

4 Aug 23, 2022

A new framework, collaborative cascade prediction based on graph neural networks (CCasGNN) to jointly utilize the structural characteristics, sequence features, and user profiles.

CCasGNN A new framework, collaborative cascade prediction based on graph neural networks (CCasGNN) to jointly utilize the structural characteristics,

5 Apr 29, 2022

PyTorch implementation of "PatchGame: Learning to Signal Mid-level Patches in Referential Games" to appear in NeurIPS 2021

PatchGame: Learning to Signal Mid-level Patches in Referential Games This repository is the official implementation of the paper - "PatchGame: Learnin

22 Mar 16, 2022

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Related tags

Overview

SW-CV-ModelZoo

Reference:

Journal

You might also like...

Human head pose estimation using Keras over TensorFlow.

Graph Neural Networks with Keras and Tensorflow 2.

QKeras: a quantization deep learning library for Tensorflow Keras

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Deep GPs built on top of TensorFlow/Keras and GPflow

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

convnexts_db2021_2022_03_22(Mar 22, 2022)

nfnets_db2021_2022_03_04(Mar 4, 2022)

nfnet_tpu_training(Feb 6, 2022)

NFNetL1V1-100-0.57141(Dec 31, 2021)

Owner

Styled text-to-drawing synthesis method. Featured at the 2021 NeurIPS Workshop on Machine Learning for Creativity and Design

Official pytorch implementation of paper "Image-to-image Translation via Hierarchical Style Disentanglement".

Source code and notebooks to reproduce experiments and benchmarks on Bias Faces in the Wild (BFW).

Using image super resolution models with vapoursynth and speeding them up with TensorRT

A new framework, collaborative cascade prediction based on graph neural networks (CCasGNN) to jointly utilize the structural characteristics, sequence features, and user profiles.

Action Segmentation Evaluation

RIFE - Real-Time Intermediate Flow Estimation for Video Frame Interpolation

Self-Learning - Books Papers, Courses & more I have to learn soon

Official implementation of the PICASO: Permutation-Invariant Cascaded Attentional Set Operator

Dynamic wallpaper generator.

Affine / perspective transformation in Pose Estimation with Tensorflow 2

Distributed DataLoader For Pytorch Based On Ray

[SIGGRAPH Asia 2021] DeepVecFont: Synthesizing High-quality Vector Fonts via Dual-modality Learning.

MoveNet Single Pose on DepthAI

ONNX Runtime Web demo is an interactive demo portal showing real use cases running ONNX Runtime Web in VueJS.

An experiment to bait a generalized frontrunning MEV bot

Source code of our BMVC 2021 paper: AniFormer: Data-driven 3D Animation with Transformer

Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)

Stochastic Normalizing Flows

PyTorch implementation of "PatchGame: Learning to Signal Mid-level Patches in Referential Games" to appear in NeurIPS 2021