Self-Supervised Learning

Last update: Dec 14, 2021

Overview

Self-Supervised Learning

Features

self_supervised offers features like

modular framework
support for multi-gpu training using PyTorch Lightning
easy to use and written in a PyTorch like style
supports custom backbone models for self-supervised pre-training

Supported Models

Supported Loss Function

Benchmarks

Currently implemented models and their accuracy on cifar10 and imagenette.

ImageNette

Model	Epochs	Batch Size	Test Accuracy
MoCo	800	256	0.827
SimCLR	800	256	0.847
SimSiam	800	256	0.827
BarlowTwins	800	256	0.801
BYOL	800	256	0.851

Cifar10

Model	Epochs	Batch Size	Test Accuracy
MoCo	200	128	0.83
SimCLR	200	128	0.78
SimSiam	200	128	0.73
BarlowTwins	200	128	0.84
BYOL	200	128	0.85
MoCo	200	512	0.85
SimCLR	200	512	0.83
SimSiam	200	512	0.81
BarlowTwins	200	512	0.78
BYOL	200	512	0.84
MoCo	800	128	0.89
SimCLR	800	128	0.87
SimSiam	800	128	0.80
MoCo	800	512	0.90
SimCLR	800	512	0.89
SimSiam	800	512	0.91

Tutorials

Want to jump to the tutorials and see lightly in action?

Self-Supervised Learning

Related tags

Overview

Self-Supervised Learning

Features

Supported Models

Supported Loss Function

Benchmarks

ImageNette

Cifar10

Tutorials

Further Reading

Owner

Robin

Bayesian Generative Adversarial Networks in Tensorflow

Code for our NeurIPS 2021 paper 'Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation'

Implementation of popular SOTA self-supervised learning algorithms as Fastai Callbacks.

Ranger - a synergistic optimizer using RAdam (Rectified Adam), Gradient Centralization and LookAhead in one codebase

The Official Implementation of the ICCV-2021 Paper: Semantically Coherent Out-of-Distribution Detection.

Model Zoo for MindSpore

Improving Deep Network Debuggability via Sparse Decision Layers

Alternatives to Deep Neural Networks for Function Approximations in Finance

Repo for the paper Extrapolating from a Single Image to a Thousand Classes using Distillation

A voice recognition assistant similar to amazon alexa, siri and google assistant.

An Intelligent Self-driving Truck System For Highway Transportation

Official PyTorch implementation of the paper Image-Based CLIP-Guided Essence Transfer.

Data cleaning, missing value handle, EDA use in this project

A module that used for encrypt code which includes RSA and AES

Dataset and Source code of paper 'Enhancing Keyphrase Extraction from Academic Articles with their Reference Information'.

这是一个yolox-keras的源码，可以用于训练自己的模型。

A new play-and-plug method of controlling an existing generative model with conditioning attributes and their compositions.

A Simple Long-Tailed Rocognition Baseline via Vision-Language Model

Official code for ICCV2021 paper "M3D-VTON: A Monocular-to-3D Virtual Try-on Network"

Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On