efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

PyDEns is a framework for solving Ordinary and Partial Differential Equations (ODEs & PDEs) using neural networks

Implementation of the Swin Transformer in PyTorch.

Repository for RNNs using TensorFlow and Keras - LSTM and GRU Implementation from Scratch - Simple Classification and Regression Problem using RNNs

Implementing Graph Convolutional Networks and Information Retrieval Mechanisms using pure Python and NumPy

Python Tensorflow 2 scripts for detecting objects of any class in an image without knowing their label.

Code, final versions, and information on the Sparkfun Graphical Datasheets

A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.

A Simple LSTM-Based Solution for "Heartbeat Signal Classification and Prediction" in Tianchi

The implementation of "Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer"

Semi-supervised Representation Learning for Remote Sensing Image Classification Based on Generative Adversarial Networks

Implicit Graph Neural Networks

Industrial knn-based anomaly detection for images. Visit streamlit link to check out the demo.

Model parallel transformers in Jax and Haiku

Learning from Synthetic Shadows for Shadow Detection and Removal [Inoue+, IEEE TCSVT 2020].

KoRean based ELECTRA pre-trained models (KR-ELECTRA) for Tensorflow and PyTorch

Greedy Gaussian Segmentation

Employee-Managment - Company employee registration software in the face recognition system

Concept drift monitoring for HA model servers.

Bayesian Generative Adversarial Networks in Tensorflow

Segmentation models with pretrained backbones. PyTorch.