Contrastive Learning with Non-Semantic Negatives

This repository is the official implementation of Robust Contrastive Learning Using Negative Samples with Diminished Semantics. Contrastive learning utilizes positive pairs which preserve semantic information while perturbing superficial features in the training images. Similarly, we propose to generate negative samples to make the model more robust, where only the superfluous instead of the semantic features are preserved.

Preparation

Install PyTorch and check preprocess/ for ImageNet-100 and ImageNet-Texture preprocessing scripts.

Training

The following code is used to pre-train MoCo-v2 + patch / texture-based NS. The major code is developed with minimal modifications from the official implementation.

python moco-non-sem-neg.py -a resnet50 --lr 0.03 --batch-size 128 --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed --world-size 1 --rank 0 \
  --mlp --moco-t 0.2 --aug-plus --cos --moco-k 16384 \
  --robust nonsem --num-nonsem 1 --alpha 2 --epochs 200 --patch-ratio 16 72 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200  \
  /path/to/imagenet-100/ 

python moco-non-sem-neg.py -a resnet50 --lr 0.03 --batch-size 128 --dist-url 'tcp://localhost:10001' \
  --multiprocessing-distributed --world-size 1 --rank 0 \
  --mlp --moco-t 0.2 --aug-plus --cos --moco-k 16384 \
  --robust texture_syn --num-nonsem 1 --alpha 2 --epochs 200 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_texture_nn1_alpha2_epoch200 \
  /path/to/imagenet-100-texture/

Change /path/to/imagenet-100/ with the ImageNet-100 dataset directory.
Change --alpha and -moco-k to reproduce results with different configurations.

Linear Evaluation

Run following code is used to reproduce MoCo-v2 + patch-based NS model reported in Table 1.

python main_lincls.py -a resnet50 --lr 10.0 --batch-size 128 --epochs 60 \
  --pretrained ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200/checkpoint_0199.pth.tar \
  --dist-url 'tcp://localhost:10001' --multiprocessing-distributed --world-size 1 --rank 0 \
  --ckpt_path ./ckpts/mocov2_mocok16384_bs128_lr0.03_nonsem_16_72_noaug_nn1_alpha2_epoch200 \
  /path/to/imagenet-100/

Pre-trained Models

You can download pretrained models here:

	moco-k	alpha	ImageNet-100	Corruption	Sketch	Stylized	Rendition	Checkpoints
MoCo-v2	16384	-	77.88±0.28	43.08±0.27	28.24±0.58	16.20±0.55	32.92±0.12	Run1, Run2, Run3
+ Texture	16384	2	77.76±0.17	43.58±0.33	29.11±0.39	16.59±0.17	33.36±0.15	Run1, Run2, Run3
+ Patch	16384	2	79.35±0.12	45.13±0.35	31.76±0.88	17.37±0.19	34.78±0.15	Run1, Run2, Run3
+ Patch	16384	3	75.58±0.52	44.45±0.15	34.03±0.58	18.60±0.26	36.89±0.11	Run1, Run2, Run3
MoCo-v2	8192	-	77.73±0.38	43.22±0.39	28.45±0.36	16.83±0.12	33.19±0.44	Run1, Run2, Run3
+ Patch	8192	2	79.54±0.32	45.48±0.20	33.36±0.45	17.81±0.32	36.31±0.37	Run1, Run2, Run3

Contrastive Learning with Non-Semantic Negatives

Related tags

Overview

Contrastive Learning with Non-Semantic Negatives

Preparation

Training

Linear Evaluation

Pre-trained Models

Owner

Rename Images with Auto Generated Neural Image Captions

Code for A Volumetric Transformer for Accurate 3D Tumor Segmentation

Code release for "Masked-attention Mask Transformer for Universal Image Segmentation"

STMTrack: Template-free Visual Tracking with Space-time Memory Networks

A PyTorch implementation of EfficientDet.

Codes for paper "Towards Diverse Paragraph Captioning for Untrimmed Videos". CVPR 2021

An intuitive library to extract features from time series

Code for paper Adaptively Aligned Image Captioning via Adaptive Attention Time

Implementation of Memformer, a Memory-augmented Transformer, in Pytorch

Universal Adversarial Triggers for Attacking and Analyzing NLP (EMNLP 2019)

AquaTimer - Programmable Timer for Aquariums based on ATtiny414/814/1614

It's A ML based Web Site build with python and Django to find the breed of the dog

nnFormer: Interleaved Transformer for Volumetric Segmentation

Visualize Camera's Pose Using Extrinsic Parameter by Plotting Pyramid Model on 3D Space

FairMOT for Multi-Class MOT using YOLOX as Detector

Highway networks implemented in PyTorch.

NOD: Taking a Closer Look at Detection under Extreme Low-Light Conditions with Night Object Detection Dataset

Code for Paper: Self-supervised Learning of Motion Capture

Neural Turing Machine (NTM) & Differentiable Neural Computer (DNC) with pytorch & visdom

Character Grounding and Re-Identification in Story of Videos and Text Descriptions