GradAttack is a Python library for easy evaluation of privacy risks in public gradients in Federated Learning

Overview

GradAttack

GradAttack CI

GradAttack is a Python library for easy evaluation of privacy risks in public gradients in Federated Learning, as well as corresponding mitigation strategies. The current version focuses on the gradient inversion attack in the image classification task, which recovers private images from public gradients.

Motivation

Recent research shows that sending gradients instead of data in Federated Learning can leak private information (see this growing list of attack paper). These attacks demonstrate that an adversary eavesdropping on a client’s communications (i.e. observing the global modelweights and client update) can accurately reconstruct a client’s private data using a class of techniques known as “gradient inversion attacks", which raise serious concerns about such privacy leakage.

To counter these attacks, researchers have proposed defense mechanisms (see this growing list of defense paper). We are developing this framework to evaluate different defense mechanisms against state-of-the-art attacks.

Why GradAttack?

There are lots of reasons to use GradAttack:

  • 😈   Evaluate the privacy risk of your Federated Learning pipeline by running on it various attacks supported by GradAttack

  • 💊   Enhance the privacy of your Federated Learning pipeline by applying defenses supported by GradAttack in a plug-and-play fashion

  • 🔧   Research and develop new gradient attacks and defenses by reusing the simple and extensible APIs in GradAttack

Slack Channel

For help and realtime updates related to GradAttack, please join the GradAttack Slack!

Installation

You may install GradAttack directly from PyPi using pip:

pip install gradattack

You can also install directly from the source for the latest features:

git clone https://github.com/Princeton-SysML/GradAttack
cd GradAttack
pip install -e .

Getting started

To evaluate your model's privacy leakage against the gradient inversion attack, all you need to do is to:

  1. Define your deep learning pipeline
datamodule = CIFAR10DataModule()
model = create_lightning_module(
        'ResNet18',
        training_loss_metric=loss,
        **hparams,
    )
trainer = pl.Trainer(
        gpus=devices,
        check_val_every_n_epoch=1,
        logger=logger,
        max_epochs=args.n_epoch,
        callbacks=[early_stop_callback],
    )
pipeline = TrainingPipeline(model, datamodule, trainer)
  1. (Optional) Apply defenses to the pipeline
defense_pack = DefensePack(args, logger)
defense_pack.apply_defense(pipeline)
  1. Run training with the pipeline (see detailed example scripts and bashes in examples)
pipeline.run()
pipeline.test()

You may use the tensorboard logs to track your training and to compare results of different runs:

tensorboard --logdir PATH_TO_TRAIN_LOGS

Example of training logs

  1. Run attack on the pipeline (see detailed example scripts and bashes in examples)
# Fetch a victim batch and define an attack instance
example_batch = pipeline.get_datamodule_batch()
batch_gradients, step_results = pipeline.model.get_batch_gradients(
        example_batch, 0)
batch_inputs_transform, batch_targets_transform = step_results[
    "transformed_batch"]
attack_instance = GradientReconstructor(
    pipeline,
    ground_truth_inputs=batch_inputs_transform,
    ground_truth_gradients=batch_gradients,
    ground_truth_labels=batch_targets_transform,
)

# Define the attack instance and launch the attack
attack_trainer = pl.Trainer(
    max_epochs=10000,
)
attack_trainer.fit(attack_instance,)

You may use the tensorboard logs to track your attack and to compare results of different runs:

tensorboard --logdir PATH_TO_ATTACK_LOGS

Example of training logs

  1. Evalute the attack results (see examples)
python examples/calc_metric.py --dir PATH_TO_ATTACK_RESULTS

Contributing to GradAttack

GradAttack is currently in an "alpha" stage in which we are working to improve its capabilities and design.

Contributions are welcome! See the contributing guide for detailed instructions on how to contribute to our project.

Citing GradAttack

If you want to use GradAttack for your research (much appreciated!), you can cite it as follows:

@inproceedings{huang2021evaluating,
  title={Evaluating Gradient Inversion Attacks and Defenses in Federated Learning},
  author={Huang, Yangsibo and Gupta, Samyak and Song, Zhao and Li, Kai and Arora, Sanjeev},
  booktitle={NeurIPS},
  year={2021}
}

Acknowledgement

This project is supported in part by Ma Huateng Foundation, Schmidt Foundation, NSF, Simons Foundation, ONR and DARPA/SRC. Yangsibo Huang and Samyak Gupta are supported in part by the Princeton Graduate Fellowship. We would like to thank Quanzheng Li, Xiaoxiao Li, Hongxu Yin and Aoxiao Zhong for helpful discussions, and members of Kai Li’s and Sanjeev Arora’s research groups for comments on early versions of this library.

A library of multi-agent reinforcement learning components and systems

Mava: a research framework for distributed multi-agent reinforcement learning Table of Contents Overview Getting Started Supported Environments System

InstaDeep Ltd 463 Dec 23, 2022
A cross-lingual COVID-19 fake news dataset

CrossFake An English-Chinese COVID-19 fake&real news dataset from the ICDMW 2021 paper below: Cross-lingual COVID-19 Fake News Detection. Jiangshu Du,

Yingtong Dou 11 Dec 01, 2022
Official Implementation of CoSMo: Content-Style Modulation for Image Retrieval with Text Feedback

CoSMo.pytorch Official Implementation of CoSMo: Content-Style Modulation for Image Retrieval with Text Feedback, Seungmin Lee*, Dongwan Kim*, Bohyung

Seung Min Lee 54 Dec 08, 2022
PyTorch implementations of the beta divergence loss.

Beta Divergence Loss - PyTorch Implementation This repository contains code for a PyTorch implementation of the beta divergence loss. Dependencies Thi

Billy Carson 7 Nov 09, 2022
Tacotron 2 - PyTorch implementation with faster-than-realtime inference

Tacotron 2 (without wavenet) PyTorch implementation of Natural TTS Synthesis By Conditioning Wavenet On Mel Spectrogram Predictions. This implementati

NVIDIA Corporation 4.1k Jan 03, 2023
Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

TorchCAM: class activation explorer Simple way to leverage the class-specific activation of convolutional layers in PyTorch. Quick Tour Setting your C

F-G Fernandez 1.2k Dec 29, 2022
EMNLP 2020 - Summarizing Text on Any Aspects

Summarizing Text on Any Aspects This repo contains preliminary code of the following paper: Summarizing Text on Any Aspects: A Knowledge-Informed Weak

Bowen Tan 35 Nov 14, 2022
Tensorflow implementation of Semi-supervised Sequence Learning (https://arxiv.org/abs/1511.01432)

Transfer Learning for Text Classification with Tensorflow Tensorflow implementation of Semi-supervised Sequence Learning(https://arxiv.org/abs/1511.01

DONGJUN LEE 82 Oct 22, 2022
Official PyTorch implementation of UACANet: Uncertainty Aware Context Attention for Polyp Segmentation

UACANet: Uncertainty Aware Context Attention for Polyp Segmentation Official pytorch implementation of UACANet: Uncertainty Aware Context Attention fo

Taehun Kim 85 Dec 14, 2022
Official Pytorch implementation of RePOSE (ICCV2021)

RePOSE: Iterative Rendering and Refinement for 6D Object Detection (ICCV2021) [Link] Abstract We present RePOSE, a fast iterative refinement method fo

Shun Iwase 68 Nov 15, 2022
Code associated with the paper "Deep Optics for Single-shot High-dynamic-range Imaging"

Deep Optics for Single-shot High-dynamic-range Imaging Code associated with the paper "Deep Optics for Single-shot High-dynamic-range Imaging" CVPR, 2

Stanford Computational Imaging Lab 40 Dec 12, 2022
Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechanism for Generalized Face Presentation Attack Detection

LMFD-PAD Note This is the official repository of the paper: LMFD-PAD: Learnable Multi-level Frequency Decomposition and Hierarchical Attention Mechani

28 Dec 02, 2022
A PyTorch implementation of "Predict then Propagate: Graph Neural Networks meet Personalized PageRank" (ICLR 2019).

APPNP ⠀ A PyTorch implementation of Predict then Propagate: Graph Neural Networks meet Personalized PageRank (ICLR 2019). Abstract Neural message pass

Benedek Rozemberczki 329 Dec 30, 2022
Machine Learning University: Accelerated Computer Vision Class

Machine Learning University: Accelerated Computer Vision Class This repository contains slides, notebooks, and datasets for the Machine Learning Unive

AWS Samples 1.3k Dec 28, 2022
GULAG: GUessing LAnGuages with neural networks

GULAG: GUessing LAnGuages with neural networks Classify languages in text via neural networks. Привет! My name is Egor. Was für ein herrliches Frühl

Egor Spirin 12 Sep 02, 2022
Code accompanying paper: Meta-Learning to Improve Pre-Training

Meta-Learning to Improve Pre-Training This folder contains code to run experiments in the paper Meta-Learning to Improve Pre-Training, NeurIPS 2021. P

28 Dec 31, 2022
Pytorch implementation of Bert and Pals: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning

PyTorch implementation of BERT and PALs Introduction Work by Asa Cooper Stickland and Iain Murray, University of Edinburgh. Code for BERT and PALs; mo

Asa Cooper Stickland 70 Dec 29, 2022
Release of the ConditionalQA dataset

ConditionalQA Datasets accompanying the paper ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers. Disclaimer This dataset

14 Oct 17, 2022
Code for testing various M1 Chip benchmarks with TensorFlow.

M1, M1 Pro, M1 Max Machine Learning Speed Test Comparison This repo contains some sample code to benchmark the new M1 MacBooks (M1 Pro and M1 Max) aga

Daniel Bourke 348 Jan 04, 2023
P-Tuning v2: Prompt Tuning Can Be Comparable to Finetuning Universally Across Scales and Tasks

P-tuning v2 P-Tuning v2: Prompt Tuning Can Be Comparable to Finetuning Universally Across Scales and Tasks An optimized prompt tuning strategy achievi

THUDM 540 Dec 30, 2022