Code snippets created for the PyTorch discussion board

Overview

PyTorch misc

Collection of code snippets I've written for the PyTorch discussion board.

All scripts were testes using the PyTorch 1.0 preview and torchvision 0.2.1.

Additional libraries, e.g. numpy or pandas, are used in a few scripts.

Some scripts might be a good starter to create a tutorial.

Overview

  • accumulate_gradients - Comparison of accumulated gradients/losses to vanilla batch update.
  • adaptive_batchnorm- Adaptive BN implementation using two additional parameters: out = a * x + b * bn(x).
  • adaptive_pooling_torchvision - Example of using adaptive pooling layers in pretrained models to use different spatial input shapes.
  • batch_norm_manual - Comparison of PyTorch BatchNorm layers and a manual calculation.
  • change_crop_in_dataset - Change the image crop size on the fly using a Dataset.
  • channel_to_patches - Permute image data so that channel values of each pixel are flattened to an image patch around the pixel.
  • conv_rnn - Combines a 3DCNN with an RNN; uses windowed frames as inputs.
  • csv_chunk_read - Provide data chunks from continuous .csv file.
  • densenet_forwardhook - Use forward hooks to get intermediate activations from densenet121. Uses separate modules to process these activations further.
  • edge_weighting_segmentation - Apply weighting to edges for a segmentation task.
  • image_rotation_with_matrix - Rotate an image given an angle using 1.) a nested loop and 2.) a rotation matrix and mesh grid.
  • LocallyConnected2d - Implementation of a locally connected 2d layer.
  • mnist_autoencoder - Simple autoencoder for MNIST data. Includes visualizations of output images, intermediate activations and conv kernels.
  • mnist_permuted - MNIST training using permuted pixel locations.
  • model_sharding_data_parallel - Model sharding with DataParallel using 2 pairs of 2 GPUs.
  • momentum_update_nograd - Script to see how parameters are updated when an optimizer is used with momentum/running estimates, even if gradients are zero.
  • pytorch_redis - Script to demonstrate the loading data from redis using a PyTorch Dataset and DataLoader.
  • shared_array - Script to demonstrate the usage of shared arrays using multiple workers.
  • shared_dict - Script to demonstrate the usage of shared dicts using multiple workers.
  • unet_demo - Simple UNet demo.
  • weighted_sampling - Usage of WeightedRandomSampler using an imbalanced dataset with class imbalance 99 to 1.

Feedback is very welcome!

Owner
Deep Learning Frameworks @NVIDIA
PyTorch to TensorFlow Lite converter

PyTorch to TensorFlow Lite converter

Omer Ferhat Sarioglu 140 Dec 13, 2022
A PyTorch implementation of EfficientNet

EfficientNet PyTorch Quickstart Install with pip install efficientnet_pytorch and load a pretrained EfficientNet with: from efficientnet_pytorch impor

Luke Melas-Kyriazi 7.2k Jan 06, 2023
A pure Python implementation of Compact Bilinear Pooling and Count Sketch for PyTorch.

Compact Bilinear Pooling for PyTorch. This repository has a pure Python implementation of Compact Bilinear Pooling and Count Sketch for PyTorch. This

Grégoire Payen de La Garanderie 234 Dec 07, 2022
Model summary in PyTorch similar to `model.summary()` in Keras

Keras style model.summary() in PyTorch Keras has a neat API to view the visualization of the model which is very helpful while debugging your network.

Shubham Chandel 3.7k Dec 29, 2022
270 Dec 24, 2022
Kaldi-compatible feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and autograd

Kaldi-compatible feature extraction with PyTorch, supporting CUDA, batch processing, chunk processing, and autograd

Fangjun Kuang 119 Jan 03, 2023
PyGCL: Graph Contrastive Learning Library for PyTorch

PyGCL is an open-source library for graph contrastive learning (GCL), which features modularized GCL components from published papers, standardized evaluation, and experiment management.

GCL: Graph Contrastive Learning Library for PyTorch 592 Jan 07, 2023
A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch

Torchmeta A collection of extensions and data-loaders for few-shot learning & meta-learning in PyTorch. Torchmeta contains popular meta-learning bench

Tristan Deleu 1.7k Jan 06, 2023
A code copied from google-research which named motion-imitation was rewrited with PyTorch

motor-system Introduction A code copied from google-research which named motion-imitation was rewrited with PyTorch. More details can get from this pr

NewEra 6 Jan 08, 2022
Training PyTorch models with differential privacy

Opacus is a library that enables training PyTorch models with differential privacy. It supports training with minimal code changes required on the cli

1.3k Dec 29, 2022
Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

PyTorch Implementation of Differentiable ODE Solvers This library provides ordinary differential equation (ODE) solvers implemented in PyTorch. Backpr

Ricky Chen 4.4k Jan 04, 2023
Riemannian Adaptive Optimization Methods with pytorch optim

geoopt Manifold aware pytorch.optim. Unofficial implementation for “Riemannian Adaptive Optimization Methods” ICLR2019 and more. Installation Make sur

642 Jan 03, 2023
3D-RETR: End-to-End Single and Multi-View3D Reconstruction with Transformers

3D-RETR: End-to-End Single and Multi-View 3D Reconstruction with Transformers (BMVC 2021) Zai Shi*, Zhao Meng*, Yiran Xing, Yunpu Ma, Roger Wattenhofe

Zai Shi 36 Dec 21, 2022
Code snippets created for the PyTorch discussion board

PyTorch misc Collection of code snippets I've written for the PyTorch discussion board. All scripts were testes using the PyTorch 1.0 preview and torc

461 Dec 26, 2022
An implementation of Performer, a linear attention-based transformer, in Pytorch

Performer - Pytorch An implementation of Performer, a linear attention-based transformer variant with a Fast Attention Via positive Orthogonal Random

Phil Wang 900 Dec 22, 2022
GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks

GPU-accelerated PyTorch implementation of Zero-shot User Intent Detection via Capsule Neural Networks This repository implements a capsule model Inten

Joel Huang 15 Dec 24, 2022
PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

PyTorch framework A simple and complete framework for PyTorch, providing a variety of data loading and simple task solutions that are easy to extend and migrate

Cong Cai 12 Dec 19, 2021
Learning Sparse Neural Networks through L0 regularization

Example implementation of the L0 regularization method described at Learning Sparse Neural Networks through L0 regularization, Christos Louizos, Max W

AMLAB 202 Nov 10, 2022
High-fidelity performance metrics for generative models in PyTorch

High-fidelity performance metrics for generative models in PyTorch

Vikram Voleti 5 Oct 24, 2021
A very simple and small path tracer written in pytorch meant to be run on the GPU

MentisOculi Pytorch Path Tracer A very simple and small path tracer written in pytorch meant to be run on the GPU Why use pytorch and not some other c

Matthew B. Mirman 222 Dec 01, 2022