Laplace Redux -- Effortless Bayesian Deep Learning

Overview

Laplace Redux - Effortless Bayesian Deep Learning

This repository contains the code to run the experiments for the paper Laplace Redux - Effortless Bayesian Deep Learning (NeurIPS 2021), using our library laplace.

Requirements

After cloning the repository and creating a new virtual environment, install the package including all requirements with:

pip install .

For the BBB baseline, please follow the instructions in the corresponding README.

For running the WILDS experiments, please follow the instructions for installing the WILDS library and the required dependencies in the WILDS GitHub repository. Our experiments also require the transformers library (as mentioned in the WILDS GitHub repo under the section Installation/Default models). Our experiments were run and tested with version 1.1.0 of the WILDS library.

Uncertainty Quantification Experiments (Sections 4.2 and 4.3)

The script uq.py runs the distribution shift (rotated (F)MNIST, corrupted CIFAR-10) and OOD ((F)MNIST and CIFAR-10 as in-distribution) experiments reported in Section 4.2, as well as the experiments on the WILDS benchmark reported in Section 4.3. It expects pre-trained models, which can be downloaded here; they should be placed in the models directory. Due to the large filesize the SWAG models are not included. Please contact us if you are interested in obtaining them.

To more conveniently run the experiments with the same parameters as we used in the paper, we provide some dedicated config files for the results with the Laplace approximation ({x/y} highlights options x and y); note that you might want to change the download flag or the data_root in the config file:

python uq.py --benchmark {R-MNIST/MNIST-OOD} --config configs/post_hoc_laplace/mnist_{default/bestood}.yaml
python uq.py --benchmark {CIFAR-10-C/CIFAR-10-OOD} --config configs/post_hoc_laplace/cifar10_{default/bestood}.yaml

The config files with *_default contains the default library setting of the Laplace approximation (LA in the paper) and *_bestood the setting which performs best on OOD data (LA* in the paper).

For running the baselines, take a look at the commands in run_uq_baslines.sh.

Continual Learning Experiments (Section 4.4)

Run

python continual_learning.py

to reproduce the LA-KFAC result and run

python continual_learning.py --hessian_structure diag

to reproduce the LA-DIAG result of the continual learning experiment in Section 4.4.

Training Baselines

In order to train the baselines, please note the following:

  • Symlink your dataset dir to your ~/Datasets, e.g. ln -s /your/dataset/dir ~/Datasets.
  • Always run the training scripts from the project's root directory, e.g. python baselines/bbb/train.py.
Owner
Runa Eschenhagen
Runa Eschenhagen
Deep Video Matting via Spatio-Temporal Alignment and Aggregation [CVPR2021]

Deep Video Matting via Spatio-Temporal Alignment and Aggregation [CVPR2021] Paper: https://arxiv.org/abs/2104.11208 Introduction Despite the significa

76 Dec 07, 2022
mlpack: a scalable C++ machine learning library --

a fast, flexible machine learning library Home | Documentation | Doxygen | Community | Help | IRC Chat Download: current stable version (3.4.2) mlpack

mlpack 4.2k Jan 09, 2023
Simulation code and tutorial for BBHnet training data

Simulation Dataset for BBHnet NOTE: OLD README, UPDATE IN PROGRESS We generate simulation dataset to train BBHnet, our deep learning framework for det

0 May 31, 2022
Awesome Graph Classification - A collection of important graph embedding, classification and representation learning papers with implementations.

A collection of graph classification methods, covering embedding, deep learning, graph kernel and factorization papers

Benedek Rozemberczki 4.5k Jan 01, 2023
Companion code for "Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees"

Companion code for "Bayesian logistic regression for online recalibration and revision of risk prediction models with performance guarantees" Installa

0 Oct 13, 2021
neural image generation

pixray Pixray is an image generation system. It combines previous ideas including: Perception Engines which uses image augmentation and iteratively op

dribnet 398 Dec 17, 2022
The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography using a CNN-based orientation classifier')

The PyTorch re-implement of a 3D CNN Tracker to extract coronary artery centerlines with state-of-the-art (SOTA) performance. (paper: 'Coronary artery centerline extraction in cardiac CT angiography

James 135 Dec 23, 2022
NEATEST: Evolving Neural Networks Through Augmenting Topologies with Evolution Strategy Training

NEATEST: Evolving Neural Networks Through Augmenting Topologies with Evolution Strategy Training

Göktuğ Karakaşlı 16 Dec 05, 2022
Deep Q-Learning Network in pytorch (not actively maintained)

pytoch-dqn This project is pytorch implementation of Human-level control through deep reinforcement learning and I also plan to implement the followin

Hung-Tu Chen 342 Jan 01, 2023
Implementation of SSMF: Shifting Seasonal Matrix Factorization

SSMF Implementation of SSMF: Shifting Seasonal Matrix Factorization, Koki Kawabata, Siddharth Bhatia, Rui Liu, Mohit Wadhwa, Bryan Hooi. NeurIPS, 2021

Koki Kawabata 9 Jun 10, 2022
GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

GLaRA: Graph-based Labeling Rule Augmentation for Weakly Supervised Named Entity Recognition

Xinyan Zhao 29 Dec 26, 2022
FMA: A Dataset For Music Analysis

FMA: A Dataset For Music Analysis Michaël Defferrard, Kirell Benzi, Pierre Vandergheynst, Xavier Bresson. International Society for Music Information

Michaël Defferrard 1.8k Dec 29, 2022
OREO: Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning (NeurIPS 2021)

OREO: Object-Aware Regularization for Addressing Causal Confusion in Imitation Learning (NeurIPS 2021) Video demo We here provide a video demo from co

20 Nov 25, 2022
NAS-Bench-x11 and the Power of Learning Curves

NAS-Bench-x11 NAS-Bench-x11 and the Power of Learning Curves Shen Yan, Colin White, Yash Savani, Frank Hutter. NeurIPS 2021. Surrogate NAS benchmarks

AutoML-Freiburg-Hannover 13 Nov 18, 2022
Gif-caption - A straightforward GIF Captioner written in Python

Broksy's GIF Captioner Have you ever wanted to easily caption a GIF without havi

3 Apr 09, 2022
An experiment on the performance of homemade Q-learning AIs in Agar.io depending on their state representation and available actions

Agar.io_Q-Learning_AI An experiment on the performance of homemade Q-learning AIs in Agar.io depending on their state representation and available act

1 Jun 09, 2022
A complete, self-contained example for training ImageNet at state-of-the-art speed with FFCV

ffcv ImageNet Training A minimal, single-file PyTorch ImageNet training script designed for hackability. Run train_imagenet.py to get... ...high accur

FFCV 92 Dec 31, 2022
Hydra Lightning Template for Structured Configs

Hydra Lightning Template for Structured Configs Template for creating projects with pytorch-lightning and hydra. How to use this template? Create your

Model-driven Machine Learning 4 Jul 19, 2022
Bianace Prediction Pytorch Model

Bianace Prediction Pytorch Model Main Results ETHUSDT from 2021-01-01 00:00:00 t

RoyYang 4 Jul 20, 2022
This is a Tensorflow implementation of Learning to See in the Dark in CVPR 2018

Learning-to-See-in-the-Dark This is a Tensorflow implementation of Learning to See in the Dark in CVPR 2018, by Chen Chen, Qifeng Chen, Jia Xu, and Vl

5.3k Jan 01, 2023