Multitask Learning Strengthens Adversarial Robustness

Related tags

Deep LearningMTRobust
Overview

Multitask Learning Strengthens Adversarial Robustness

@inproceedings{mao2020multitask,
  author    = {Chengzhi Mao and
               Amogh Gupta and
               Vikram Nitin and
               Baishakhi Ray and
               Shuran Song and
               Junfeng Yang and
               Carl Vondrick},
  title     = {Multitask Learning Strengthens Adversarial Robustness},
  booktitle = {Computer Vision - {ECCV} 2020 - 16th European Conference, Glasgow,
               UK, August 23-28, 2020, Proceedings, Part {II}},
  series    = {Lecture Notes in Computer Science},
  volume    = {12347},
  pages     = {158--174},
  publisher = {Springer},
  year      = {2020},
  url       = {https://doi.org/10.1007/978-3-030-58536-5\_10},
  doi       = {10.1007/978-3-030-58536-5\_10},
}

Demo for Robustness under multitask attack

Download Cityscapes dataset from Cityscapes.

Download pretrained DRN-22 model from DRN model zoo.

Modify the path to data and model in demo_mtlrobust.py.

Run demo to see the trend that model overall robustness is increased when the output dimension increased.

To see the gradient norm measurement of robustness, set get_grad=True,

To see the actually robust accuracy for model, set test_acc_output_dim=False

python demo_mtlrobust.py

which explains why segmentation is inherently robust.

CityScape

Data preprocessing

Run python data_resize_cityscape.py to resize to smaller images.

Train Robust model against single task attack

  1. Set up the path to data in config/drn_d_22_cityscape_config.json

  2. Run cityscape_example.sh to train a main task with auxiliary task for robustness.

Taskonomy

Data Preprocessing

You can use our preprocessed data from preprocessed data

Or do from scratch

  1. Download data from official raw data.

  2. Run python data_resize_taskonomy.py to resize to smaller images.

  3. Rename segment_semantic to segmentsemantic.

Train Robust model against single task attack

  1. Set up the path to data in config/resnet18_taskonomy_config.json

  2. Run taskonomy_example.sh to train a main task with auxiliary task for robustness. For different task, we have different different setup, refer to our paper and supplementary for details.

Model evaluation

We offer our pretrained models to download here: Cityscapes segmentation depth and Taskonomy taskonomy segmentation demo

After setting up the path to your downloaded models in test_cityscapes_seg.py and test_taskonomy_seg.py,

Run python test_cityscapes_seg.py and python test_taskonomy_seg.py for evaluating the robustness of multitask models under single task attacks.

Pretrained models for other tasks for Taskonomy can be downloaded [here, comming soon](comming soon)

Acknowledgement

Our code refer the code at: https://github.com/fyu/drn/blob/master/drn.py Taskonomy https://github.com/tstandley/taskgrouping,

We thank the authors for open sourcing their code.

Owner
Columbia University
Columbia University
Robust Partial Matching for Person Search in the Wild

APNet for Person Search Introduction This is the code of Robust Partial Matching for Person Search in the Wild accepted in CVPR2020. The Align-to-Part

Yingji Zhong 36 Dec 18, 2022
Tool for live presentations using manim

manim-presentation Tool for live presentations using manim Install pip install manim-presentation opencv-python Usage Use the class Slide as your sce

Federico Galatolo 146 Jan 06, 2023
Recurrent Conditional Query Learning

Recurrent Conditional Query Learning (RCQL) This repository contains the Pytorch implementation of One Model Packs Thousands of Items with Recurrent C

Dongda 4 Nov 28, 2022
The official PyTorch code for NeurIPS 2021 ML4AD Paper, "Does Thermal data make the detection systems more reliable?"

MultiModal-Collaborative (MMC) Learning Framework for integrating RGB and Thermal spectral modalities This is the official code for NeurIPS 2021 Machi

NeurAI 12 Nov 02, 2022
[CVPR 2021] Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

[CVPR 2021] Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers

Fudan Zhang Vision Group 897 Jan 05, 2023
Implementation of the "Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos" paper.

Point 4D Transformer Networks for Spatio-Temporal Modeling in Point Cloud Videos Introduction Point cloud videos exhibit irregularities and lack of or

Hehe Fan 101 Dec 29, 2022
SAT: 2D Semantics Assisted Training for 3D Visual Grounding, ICCV 2021 (Oral)

SAT: 2D Semantics Assisted Training for 3D Visual Grounding SAT: 2D Semantics Assisted Training for 3D Visual Grounding by Zhengyuan Yang, Songyang Zh

Zhengyuan Yang 22 Nov 30, 2022
Code accompanying "Adaptive Methods for Aggregated Domain Generalization"

Adaptive Methods for Aggregated Domain Generalization (AdaClust) Official Pytorch Implementation of Adaptive Methods for Aggregated Domain Generalizat

Xavier Thomas 15 Sep 20, 2022
A multi-mode modulator for multi-domain few-shot classification (ICCV)

A multi-mode modulator for multi-domain few-shot classification (ICCV)

Yanbin Liu 8 Apr 28, 2022
Codes for "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation"

CSDI This is the github repository for the NeurIPS 2021 paper "CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputation

106 Jan 04, 2023
Place holder for HOPE: a human-centric and task-oriented MT evaluation framework using professional post-editing

HOPE: A Task-Oriented and Human-Centric Evaluation Framework Using Professional Post-Editing Towards More Effective MT Evaluation Place holder for dat

Lifeng Han 1 Apr 25, 2022
🔥RandLA-Net in Tensorflow (CVPR 2020, Oral & IEEE TPAMI 2021)

RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Clouds (CVPR 2020) This is the official implementation of RandLA-Net (CVPR2020, Oral

Qingyong 1k Dec 30, 2022
Wordle-solver - Wordle answer generation program in python

🟨 Wordle Solver 🟩 Wordle answer generation program in python ✔️ Requirements U

Dahyun Kang 4 May 28, 2022
RipsNet: a general architecture for fast and robust estimation of the persistent homology of point clouds

RipsNet: a general architecture for fast and robust estimation of the persistent homology of point clouds This repository contains the code asscoiated

Felix Hensel 14 Dec 12, 2022
TumorInsight is a Brain Tumor Detection and Classification model built using RESNET50 architecture.

A Brain Tumor Detection and Classification Model built using RESNET50 architecture. The model is also deployed as a web application using Flask framework.

Pranav Khurana 0 Aug 17, 2021
Official code base for the poster "On the use of Cortical Magnification and Saccades as Biological Proxies for Data Augmentation" published in NeurIPS 2021 Workshop (SVRHM)

Self-Supervised Learning (SimCLR) with Biological Plausible Image Augmentations Official code base for the poster "On the use of Cortical Magnificatio

Binxu 8 Aug 17, 2022
Dist2Dec: A Simplicial Neural Network for Homology Localization

Dist2Dec: A Simplicial Neural Network for Homology Localization

Alexandros Keros 6 Jun 12, 2022
Official implementation of Densely connected normalizing flows

Densely connected normalizing flows This repository is the official implementation of NeurIPS 2021 paper Densely connected normalizing flows. Poster a

Matej Grcić 31 Dec 12, 2022
Details about the wide minima density hypothesis and metrics to compute width of a minima

wide-minima-density-hypothesis Details about the wide minima density hypothesis and metrics to compute width of a minima This repo presents the wide m

Nikhil Iyer 9 Dec 27, 2022
Little tool in python to watch anime from the terminal (the better way to watch anime)

ani-cli Script working again :), thanks to the fork by Dink4n for the alternative approach to by pass the captcha on gogoanime A cli to browse and wat

Harshith 4.5k Dec 31, 2022