A library to inspect itermediate layers of PyTorch models.

Last update: Dec 28, 2022

Overview

A library to inspect itermediate layers of PyTorch models.

Why?

It's often the case that we want to inspect intermediate layers of a model without modifying the code e.g. visualize attention matrices of language models, get values from an intermediate layer to feed to another layer, or applying a loss function to intermediate layers.

Install

$ pip install surgeon-pytorch

Usage

Inspect

Given a PyTorch model we can display all layers using get_layers:

import torch
import torch.nn as nn

from surgeon_pytorch import Inspect, get_layers

class SomeModel(nn.Module):

    def __init__(self):
        super().__init__()
        self.layer1 = nn.Linear(5, 3)
        self.layer2 = nn.Linear(3, 2)
        self.layer3 = nn.Linear(2, 1)

    def forward(self, x):
        x1 = self.layer1(x)
        x2 = self.layer2(x1)
        y = self.layer3(x2)
        return y


model = SomeModel()
print(get_layers(model)) # ['layer1', 'layer2', 'layer3']

Then we can wrap our model to be inspected using Inspect and in every forward call the new model we will also output the provided layer outputs (in second return value):

model_wrapped = Inspect(model, layer='layer2')
x = torch.rand(1, 5)
y, x2 = model_wrapped(x)
print(x2) # tensor([[-0.2726,  0.0910]], grad_fn=<AddmmBackward0>)

We can also provide a list of layers:

model_wrapped = Inspect(model, layer=['layer1', 'layer2'])
x = torch.rand(1, 5)
y, [x1, x2] = model_wrapped(x)
print(x1) # tensor([[ 0.1739,  0.3844, -0.4724]], grad_fn=<AddmmBackward0>)
print(x2) # tensor([[-0.2238,  0.0107]], grad_fn=<AddmmBackward0>)

Or a dictionary to get named outputs:

model_wrapped = Inspect(model, layer={'x1': 'layer1', 'x2': 'layer2'})
x = torch.rand(1, 5)
y, layers = model_wrapped(x)
print(layers)
"""
{
    'x1': tensor([[ 0.3707,  0.6584, -0.2970]], grad_fn=<AddmmBackward0>),
    'x2': tensor([[-0.1953, -0.3408]], grad_fn=<AddmmBackward0>)
}
"""

TODO

add extract function to get intermediate block

Comments

Use one backbone with different heads

Is it possible to save the results from the backbone and apply them on the heads of the all the other models. My goal was to try to save time by avoiding repeating the backbone part. Instead of running the 3 complete models (left), only run the backbone 1 time and switch only the heads for the 3 models (right), therefore not repeating executing the backbone every time in yolov5 model.

Thank you for the help!
question

opened by brunopatricio2012 4
Support for DataParallel?

Hi, I noticed that the current version does not support parallel models (at least those created using torch.nn.DataParallel) since the forward hook does not differentiate between the different copies of the model and a model wrapped with Inspect will just return the intermediate features of the last copy of the parallelized model to run.

Are you planning on fixing this issue/supporting this use case?

opened by zimmerrol 1

A library to inspect itermediate layers of PyTorch models.

Related tags

Overview

Why?

Install

Usage

Inspect

TODO

You might also like...

pyhsmm - library for approximate unsupervised inference in Bayesian Hidden Markov Models (HMMs) and explicit-duration Hidden semi-Markov Models (HSMMs), focusing on the Bayesian Nonparametric extensions, the HDP-HMM and HDP-HSMM, mostly with weak-limit approximations.

PyTorch implementation and pretrained models for XCiT models. See XCiT: Cross-Covariance Image Transformer

TorchGeo is a PyTorch domain library, similar to torchvision, that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data.

Pytorch library for end-to-end transformer models training and serving

This repository provides an efficient PyTorch-based library for training deep models.

TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

Pretrained SOTA Deep Learning models, callbacks and more for research and production with PyTorch Lightning and PyTorch

Comments

Use one backbone with different heads

Support for DataParallel?

Releases(0.0.4)

0.0.4(May 3, 2022)

0.0.3(May 3, 2022)

0.0.2(Apr 21, 2022)

0.0.1(Apr 21, 2022)

Owner

archinet.ai

PyTorch implementation of the implicit Q-learning algorithm (IQL)

Code for Environment Inference for Invariant Learning (ICML 2020 UDL Workshop Paper)

Supervised forecasting of sequential data in Python.

Code for ECCV 2020 paper "Contacts and Human Dynamics from Monocular Video".

The first dataset of composite images with rationality score indicating whether the object placement in a composite image is reasonable.

Semi-Supervised Learning for Fine-Grained Classification

O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysis

Chinese clinical named entity recognition using pre-trained BERT model

Img-process-manual - Utilize Python Numpy and Matplotlib to realize OpenCV baisc image processing function

CLUES: Few-Shot Learning Evaluation in Natural Language Understanding

Plugin adapted from Ultralytics to bring YOLOv5 into Napari

Rotation-Only Bundle Adjustment

IAUnet: Global Context-Aware Feature Learning for Person Re-Identification

Python implementation of cover trees, near-drop-in replacement for scipy.spatial.kdtree

Pyramid Scene Parsing Network, CVPR2017.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

The 3rd place solution for competition

A torch implementation of "Pixel-Level Domain Transfer"

TalkNet 2: Non-Autoregressive Depth-Wise Separable Convolutional Model for Speech Synthesis with Explicit Pitch and Duration Prediction.

Image De-raining Using a Conditional Generative Adversarial Network