RVT: Robust Vision Transformers

This repository contains PyTorch code for Robust Vision Transformers.

For details see Rethinking the Design Principles of Robust Vision Transformer by Xiaofeng Mao, Gege Qi, Yuefeng Chen, Yuan He and Hui Xue.

Usage

First, clone the repository locally:

git clone https://github.com/vtddggg/Robust-Vision-Transformer.git

Then, install PyTorch 1.7.0+ and torchvision 0.8.1+ and pytorch-image-models 0.3.2:

conda install -c pytorch pytorch torchvision
pip install timm==0.3.2

We use 4 nodes with 8 gpus to train RVT-Ti, RVT-S and RVT-B:

Training RVT-Ti

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_tiny --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-S

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_small --data-path /path/to/imagenet --output_dir output --dist-eval

Training RVT-B

python -m torch.distributed.launch --nproc_per_node=8 --nnodes=4 main.py --model rvt_base --data-path /path/to/imagenet --output_dir output --batch-size 32 --dist-eval

If you want to train RVT-Ti*, RVT-S* or RVT-B*, simply add --use_mask and --use_patch_aug to enable positon-aware attention scaling and patch-wise augmentation.

This repository contains PyTorch code for Robust Vision Transformers.

Related tags

Overview

RVT: Robust Vision Transformers

Usage

Training RVT-Ti

Training RVT-S

Training RVT-B

Owner

PyTorch implementation for "Sharpness-aware Quantization for Deep Neural Networks".

A High-Level Fusion Scheme for Circular Quantities published at the 20th International Conference on Advanced Robotics

Deeprl - Standard DQN and dueling network for simple games

Code for NeurIPS 2021 paper: Invariant Causal Imitation Learning for Generalizable Policies

[CVPR 2020] GAN Compression: Efficient Architectures for Interactive Conditional GANs

Inhomogeneous Social Recommendation with Hypergraph Convolutional Networks

CountDown to New Year and shoot fireworks

This repository includes code of my study about Asynchronous in Frequency domain of GAN images.

Back to Event Basics: SSL of Image Reconstruction for Event Cameras

Code for our paper "Graph Pre-training for AMR Parsing and Generation" in ACL2022

LRBoost is a scikit-learn compatible approach to performing linear residual based stacking/boosting.

Machine Learning Toolkit for Kubernetes

TSDF++: A Multi-Object Formulation for Dynamic Object Tracking and Reconstruction

Sign Language Transformers (CVPR'20)

code for CVPR paper Zero-shot Instance Segmentation

Uses Open AI Gym environment to create autonomous cryptocurrency bot to trade cryptocurrencies.

Centroid-UNet is deep neural network model to detect centroids from satellite images.

Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations, CVPR 2019 (Oral)

use machine learning to recognize gesture on raspberrypi

This Jupyter notebook shows one way to implement a simple first-order low-pass filter on sampled data in discrete time.