This is a potpourri of fixes.

Dependencies Currently, we require very specific package versions for any install. This is too restrictive for a research package where researchers may have a variety of reasons to carefully tailor their environment. This PR resolves the issue by allowing general installation to have flexible packages. To maintain CI stability and functionality, the fixed version requirements have been moved to dev.

Build We build our CPP extension using load instead of setup.py. This removes PyTorch as a dependency at install time and removes the need for us to distribute binaries. This required adding ninja as a dependency. Hopefully we can get rid of it eventually, but this temporarily should fix distribution.

CI testing isort was misconfigured and missed a lot of import changes. I fixed the config and ran it on all files to update import sorting.

Other While implementing this I also found a few other minor issues with respect to versioning, tests, and formatting that I resolved.

This PR makes NeuralCompression require Python 3.8 due to its usage of importlib.

Testing Testing done via CI.

Bug

Documentation builds on ReadTheDocs are failing.

Steps

See here: https://readthedocs.org/projects/neuralcompression/builds/15172473/

Expected behavior

Docs should build without error.

Environment

ReadTheDocs

Context

See here: https://readthedocs.org/projects/neuralcompression/builds/15172473/

Abstract base class (ABC) for implementing continuous entropy layers.

The abstract class pre-computes integer probability tables based on a prior distribution, which can be used across different platforms by a range encoder and decoder. The class also provides abstract methods for compression, decompression, quantization, and reconstruction.

closes #86

Non-negative parameterization as required by generalized divisive normalization (GDN) activations. The parameter is subjected to an invertible transformation that slows down the learning rate for small values.

A brief usage example:

import torch
from torch.nn import Module, Parameter
import torch.nn.functional

from torch import Tensor

from ._non_negative_parameterization import NonNegativeParameterization


class GeneralizedDivisiveNormalization(Module):
    def __init__(
        self,
        in_channels: int,
        inverse: bool = False,
        beta_min: float = 1e-6,
        gamma_init: float = 0.1,
    ):
        super(GeneralizedDivisiveNormalization, self).__init__()

        self._inverse = inverse

        self._reparameterized_beta = NonNegativeParameterization(
            torch.ones(in_channels),
            minimum=beta_min,
        )

        self._beta = Parameter(
            self._reparameterized_beta.initialized,
        )

        self._reparameterized_gamma = NonNegativeParameterization(
            gamma_init * torch.eye(in_channels),
        )

        self._gamma = Parameter(
            self._reparameterized_gamma.initialized,
        )

    def forward(self, x: Tensor) -> Tensor:
        _, channels, _, _ = x.size()

        y = torch.nn.functional.conv2d(
            x ** 2,
            torch.reshape(
                self._reparameterized_gamma(self._gamma),
                (channels, channels, 1, 1)
            ),
            self._reparameterized_beta(self._beta),
        )

        if self._inverse:
            return x * torch.sqrt(y)

        return x * torch.rsqrt(y)

import torch
import torch.testing

from neuralcompression.layers import GeneralizedDivisiveNormalization


class TestGeneralizedDivisiveNormalization:
    def test_backward(self):
        x = torch.rand((1, 32, 16, 16), requires_grad=True)

        generalized_divisive_normalization = GeneralizedDivisiveNormalization(32)

        y = generalized_divisive_normalization(x)

        y.backward(x)

        assert y.shape == x.shape

        assert x.grad is not None

        assert x.grad.shape == x.shape

        torch.testing.assert_allclose(
            x / torch.sqrt(1 + 0.1 * (x ** 2)),
            y,
        )

        generalized_divisive_normalization = GeneralizedDivisiveNormalization(
            32,
            inverse=True,
        )

        y = generalized_divisive_normalization(x)

        y.backward(x)

        assert y.shape == x.shape

        assert x.grad is not None

        assert x.grad.shape == x.shape

        torch.testing.assert_allclose(
            x * torch.sqrt(1 + 0.1 * (x ** 2)),
            y,
        )

Changes

At inference time, pad the image instead of doing an interpolation-based resize, which gives poor results (e.g. on PSNR) when the input image heigh and/or width is not exactly divisible by the downsampling factor (=2^{number of downsampling layers}).

The intention was three-fold:

1. consolidate copyright formatting across .py sources
2. simplify the implementation of #100
3. remove copyright headers from module documentation

Changes

• [x] replaces license docstrings with comments

closes #75

Survival function of x. Generally defined as 1 - distribution.cdf(x).

Unit test tests whether the returned result matches the returned result of scipy.stats.norm.sf.

This PR alters the triggers for continuous integration. Previously, we triggered all tests on both pushes and pull requests. This meant that within a PR we would have "duplicate" (but not really duplicate) checks. What we really want on a PR is just the PR check, so we'll keep that.

The other thing that's nice to have is to trigger CI when pushing to a branch. That is what this PR will remove, but to replace it we add workflow_dispatch which allows a user to trigger CI with the GitHub UI. So we remove quite a bit of duplicated tests at the cost of making users click a button if they want to test their code before PR.

Note that this PR only applies to people who push to branches of the repository.

Implements the following modules from Mentzer, et al. (2020)

• HiFiCDiscriminator
• HiFiCEncoder
• HiFiCGenerator

@misc{mentzer2020highfidelity,
      title={High-Fidelity Generative Image Compression}, 
      author={Fabian Mentzer and George Toderici and Michael Tschannen and Eirikur Agustsson},
      year={2020},
      eprint={2006.09965},
      archivePrefix={arXiv},
      primaryClass={eess.IV}
}

Originally implemented in TensorFlow Compression (TFC) by the author (@relational).

Changes

The metadata special variables from neuralcompression.__init__ were removed.

This metadata was not currently used and is now available from setup.cfg

Closes #127.

Changes

• [x] Replaced torch.testing.assert_equal with torch.testing.assert_close
• [x] Updates torch to 1.10.0
• [x] Updates torchvision to 0.11.1

We would like to have an implementation of the following paper:

Image Compression with Product Quantized Masked Image Modeling Alaaeldin El-Nouby, Matthew J. Muckley, Karen Ullrich, Ivan Laptev, Jakob Verbeek, and Hervé Jégou

At the moment we have several model implementations that are already implemented in CompressAI (e.g., Scale Hyperprior, Mean-Scale Hyperprior). At this point CompressAI has pretty good adoption, so we should be able to remove these from our repository and upstream the dependency.

By default CompressAI doesn't handle reflective image padding for users, so if desired we could include wrappers like that in PR #185 to handle this for users unfamiliar with the functionality of these models.