VOS: Learning What You Don’t Know by Virtual Outlier Synthesis

Related tags

Deep Learningvos
Overview

VOS

This is the source code accompanying the paper VOS: Learning What You Don’t Know by Virtual Outlier Synthesis by Xuefeng Du, Zhaoning Wang, Mu Cai, and Yixuan Li

The codebase is heavily based on ProbDet and Detectron2.

Dataset Preparation

PASCAL VOC

Download the processed VOC 2007 and 2012 dataset from here.

The VOC dataset folder should have the following structure:

 └── VOC_DATASET_ROOT
     |
     ├── JPEGImages
     ├── voc0712_train_all.json
     └── val_coco_format.json

COCO

Download COCO2017 dataset from the official website.

Download the OOD dataset (json file) when the in-distribution dataset is Pascal VOC from here.

Download the OOD dataset (json file) when the in-distribution dataset is BDD-100k from here.

Put the two processed OOD json files to ./anntoations

The COCO dataset folder should have the following structure:

 └── COCO_DATASET_ROOT
     |
     ├── annotations
        ├── xxx (the original json files)
        ├── instances_val2017_ood_wrt_bdd_rm_overlap.json
        └── instances_val2017_ood_rm_overlap.json
     ├── train2017
     └── val2017

BDD-100k

Donwload the BDD-100k images from the official website.

Download the processed BDD-100k json files from here and here.

The BDD dataset folder should have the following structure:

 └── BDD_DATASET_ROOT
     |
     ├── images
     ├── val_bdd_converted.json
     └── train_bdd_converted.json

OpenImages

Download our OpenImages validation splits here. We created a tarball that contains the out-of-distribution data splits used in our paper for hyperparameter tuning. Do not modify or rename the internal folders as those paths are hard coded in the dataset reader. The OpenImages dataset is created in a similar way following this paper.

The OpenImages dataset folder should have the following structure:

 └── OEPNIMAGES_DATASET_ROOT
     |
     ├── coco_classes
     └── ood_classes_rm_overlap

Before training, modify the dataset address in the ./detection/core/datasets/setup_datasets.py according to your local dataset address.

Visualization of the OOD datasets

The OOD images with respect to different in-distribution datasets can be downloaded from ID-VOC-OOD-COCO, ID-VOC-OOD-openimages, ID-BDD-OOD-COCO, ID-BDD-OOD-openimages.

Training

Firstly, enter the detection folder by running

cd detection

Vanilla Faster-RCNN with VOC as the in-distribution dataset


python train_net.py
--num-gpus 8
--config-file VOC-Detection/faster-rcnn/vanilla.yaml 
--random-seed 0 
--resume

Vanilla Faster-RCNN with BDD as the in-distribution dataset

python train_net.py 
--num-gpus 8 
--config-file BDD-Detection/faster-rcnn/vanilla.yaml 
--random-seed 0 
--resume

VOS on ResNet

python train_net_gmm.py 
--num-gpus 8 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--random-seed 0 
--resume

VOS on RegNet

Before training using the RegNet as the backbone, download the pretrained RegNet backbone from here.

python train_net_gmm.py 
--num-gpus 8 
--config-file VOC-Detection/faster-rcnn/regnetx.yaml 
--random-seed 0 
--resume

Before training on VOS, change "VOS.STARTING_ITER" and "VOS.SAMPLE_NUMBER" in the config file to the desired numbers in paper.

Evaluation

Evaluation with the in-distribution dataset to be VOC

Firstly run on the in-distribution dataset:

python apply_net.py 
--test-dataset voc_custom_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Then run on the OOD dataset:

python apply_net.py
--test-dataset coco_ood_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Obtain the metrics using:

python voc_coco_plot.py 
--name vos 
--thres xxx 
--energy 1 
--seed 0

Here the threshold is determined according to ProbDet. It will be displayed in the screen as you finish evaluating on the in-distribution dataset.

Evaluation with the in-distribution dataset to be BDD

Firstly run on the in-distribution dataset:

python apply_net.py 
--test-dataset bdd_custom_val 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Then run on the OOD dataset:

python apply_net.py 
--test-dataset coco_ood_val_bdd 
--config-file VOC-Detection/faster-rcnn/vos.yaml 
--inference-config Inference/standard_nms.yaml 
--random-seed 0 
--image-corruption-level 0 
--visualize 0

Obtain the metrics using:

python bdd_coco_plot.py
--name vos 
--thres xxx 
--energy 1 
--seed 0

Pretrained models

The pretrained models for Pascal-VOC can be downloaded from vanilla and VOS-ResNet and VOS-RegNet.

The pretrained models for BDD-100k can be downloaded from vanilla and VOS-ResNet and VOS-RegNet.

VOS on Classification models

Train on WideResNet

cd classification/CIFAR/ & 
python train_virtual.py 
--start_epoch 40 
--sample_number 1000 
--sample_from 10000 
--select 1 
--loss_weight 0.1 

where "start_epoch" denotes the starting epoch of the uncertainty regularization branch.

"sample_number" denotes the size of the in-distribution queue.

"sample_from" and "select" are used to approximate the likelihood threshold during virtual outlier synthesis.

"loss_weight" denotes the weight of the regularization loss.

Please see Section 3 and Section 4.1 in the paper for details.

Train on DenseNet

cd classification/CIFAR/ &
python train_virtual_dense.py 
--start_epoch 40 
--sample_number 1000 
--sample_from 10000 
--select 1 
--loss_weight 0.1 

Evaluation on different classifiers

cd classification/CIFAR/ & 
python test.py 
--model_name xx 
--method_name xx 
--score energy 
--num_to_avg 10

where "model_name" denotes the model architectures. ("res" denotes the WideResNet and "dense" denotes the DenseNet.)

"method_name" denotes the checkpoint name you are loading.

Pretrained models

We provide the pretrained models using WideResNet and DenseNet with the in-distribution dataset to be CIFAR-10.

Citation

If you found any part of this code is useful in your research, please consider citing our paper:

 @article{du2022vos,
      title={VOS: Learning What You Don’t Know by Virtual Outlier Synthesis}, 
      author={Du, Xuefeng and Wang, Zhaoning and Cai, Mu and Li, Yixuan},
      journal={Proceedings of the International Conference on Learning Representations},
      year={2022}
}
Owner
CS Research Group led by Prof. Sharon Li
pixelNeRF: Neural Radiance Fields from One or Few Images

pixelNeRF: Neural Radiance Fields from One or Few Images Alex Yu, Vickie Ye, Matthew Tancik, Angjoo Kanazawa UC Berkeley arXiv: http://arxiv.org/abs/2

Alex Yu 1k Jan 04, 2023
Latent Network Models to Account for Noisy, Multiply-Reported Social Network Data

VIMuRe Latent Network Models to Account for Noisy, Multiply-Reported Social Network Data. If you use this code please cite this article (preprint). De

6 Dec 15, 2022
PyTorch implementation of the implicit Q-learning algorithm (IQL)

Implicit-Q-Learning (IQL) PyTorch implementation of the implicit Q-learning algorithm IQL (Paper) Currently only implemented for online learning. Offl

Sebastian Dittert 27 Dec 30, 2022
An unofficial PyTorch implementation of a federated learning algorithm, FedAvg.

Federated Averaging (FedAvg) in PyTorch An unofficial implementation of FederatedAveraging (or FedAvg) algorithm proposed in the paper Communication-E

Seok-Ju Hahn 123 Jan 06, 2023
A light and fast one class detection framework for edge devices. We provide face detector, head detector, pedestrian detector, vehicle detector......

A Light and Fast Face Detector for Edge Devices Big News: LFD, which is a big update of LFFD, now is released (2021.03.09). It is strongly recommended

YonghaoHe 1.3k Dec 25, 2022
The official implementation of CircleNet: Anchor-free Detection with Circle Representation, MICCAI 2030

CircleNet: Anchor-free Detection with Circle Representation The official implementation of CircleNet, MICCAI 2020 [PyTorch] [project page] [MICCAI pap

The Biomedical Data Representation and Learning Lab 45 Nov 18, 2022
Dense Prediction Transformers

Vision Transformers for Dense Prediction This repository contains code and models for our paper: Vision Transformers for Dense Prediction René Ranftl,

Intel ISL (Intel Intelligent Systems Lab) 1.3k Dec 28, 2022
Collection of generative models in Tensorflow

tensorflow-generative-model-collections Tensorflow implementation of various GANs and VAEs. Related Repositories Pytorch version Pytorch version of th

3.8k Dec 30, 2022
some classic model used to segment the medical images like CT、X-ray and so on

github_project This is a project for medical image segmentation. This project includes common medical image segmentation models such as U-net, FCN, De

2 Mar 30, 2022
Collapse by Conditioning: Training Class-conditional GANs with Limited Data

Collapse by Conditioning: Training Class-conditional GANs with Limited Data Moha

Mohamad Shahbazi 33 Dec 06, 2022
[ICLR2021] Unlearnable Examples: Making Personal Data Unexploitable

Unlearnable Examples Code for ICLR2021 Spotlight Paper "Unlearnable Examples: Making Personal Data Unexploitable " by Hanxun Huang, Xingjun Ma, Sarah

Hanxun Huang 98 Dec 07, 2022
Spectral Temporal Graph Neural Network (StemGNN in short) for Multivariate Time-series Forecasting

Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting This repository is the official implementation of Spectral Temporal Gr

Microsoft 306 Dec 29, 2022
PyTorch implementaton of our CVPR 2021 paper "Bridging the Visual Gap: Wide-Range Image Blending"

Bridging the Visual Gap: Wide-Range Image Blending PyTorch implementaton of our CVPR 2021 paper "Bridging the Visual Gap: Wide-Range Image Blending".

Chia-Ni Lu 69 Dec 20, 2022
Residual Dense Net De-Interlace Filter (RDNDIF)

Residual Dense Net De-Interlace Filter (RDNDIF) Work in progress deep de-interlacer filter. It is based on the architecture proposed by Bernasconi et

Louis 7 Feb 15, 2022
Deep ViT Features as Dense Visual Descriptors

dino-vit-features [paper] [project page] Official implementation of the paper "Deep ViT Features as Dense Visual Descriptors". We demonstrate the effe

Shir Amir 113 Dec 24, 2022
[TNNLS 2021] The official code for the paper "Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement"

CSDNet-CSDGAN this is the code for the paper "Learning Deep Context-Sensitive Decomposition for Low-Light Image Enhancement" Environment Preparing pyt

Jiaao Zhang 17 Nov 05, 2022
An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

PYPARSVD This implementation allows for a singular value decomposition which is: Distributed using MPI4Py Streaming - data can be shown in batches to

Romit Maulik 44 Dec 31, 2022
Pointer networks Tensorflow2

Pointer networks Tensorflow2 原文:https://arxiv.org/abs/1506.03134 仅供参考与学习,内含代码备注 环境 tensorflow==2.6.0 tqdm matplotlib numpy 《pointer networks》阅读笔记 应用场景

HUANG HAO 7 Oct 27, 2022
Causal Influence Detection for Improving Efficiency in Reinforcement Learning

Causal Influence Detection for Improving Efficiency in Reinforcement Learning This repository contains the code release for the paper "Causal Influenc

Autonomous Learning Group 21 Nov 29, 2022
Codes for CVPR2021 paper "PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization"

PWCLO-Net: Deep LiDAR Odometry in 3D Point Clouds Using Hierarchical Embedding Mask Optimization (CVPR 2021) This is the official implementation of PW

Intelligent Robotics and Machine Vision Lab 42 Dec 18, 2022