efficient neural audio synthesis in the waveform domain

Last update: Dec 23, 2022

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

by Ben Hayes, Charalampos Saitis, György Fazekas

This repository is the official implementation of Neural Waveshaping Synthesis.

Model Architecture

Requirements

To install:

pip install -r requirements.txt
pip install -e .

We recommend installing in a virtual environment.

Data

We trained our checkpoints on the URMP dataset. Once downloaded, the dataset can be preprocessed using scripts/create_urmp_dataset.py. This will consolidate recordings of each instrument within the dataset and preprocess them according to the pipeline in the paper.

python scripts/create_urmp_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/urmp \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Alternatively, you can supply your own dataset and use the general create_dataset.py script:

python scripts/create_dataset.py \
  --gin-file gin/data/urmp_4second_crepe.gin \ 
  --data-directory /path/to/dataset \
  --output-directory /path/to/output \
  --device cuda:0  # torch device string for CREPE model

Training

To train a model on the URMP dataset, use this command:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/urmp \
  --urmp \
  --instrument vn \  # select URMP instrument with abbreviated string
  --load-data-to-memory

Or to use a non-URMP dataset:

python scripts/train.py \
  --gin-file gin/train/train_newt.gin \
  --dataset-path /path/to/processed/data \
  --load-data-to-memory

efficient neural audio synthesis in the waveform domain

Related tags

Overview

neural waveshaping synthesis

real-time neural audio synthesis in the waveform domain

paper • website • colab • audio

Model Architecture

Requirements

Data

Training

Owner

Ben Hayes

Airborne magnetic data of the Osborne Mine and Lightning Creek sill complex, Australia

Tooling for GANs in TensorFlow

Code for GNMR in ICDE 2021

用强化学习DQN算法，训练AI模型来玩合成大西瓜游戏，提供Keras版本和PARL（paddle）版本

RoadMap and preparation material for Machine Learning and Data Science - From beginner to expert.

An easy way to build PyTorch datasets. Modularly build datasets and automatically cache processed results

MLP-Like Vision Permutator for Visual Recognition (PyTorch)

NeuralTalk is a Python+numpy project for learning Multimodal Recurrent Neural Networks that describe images with sentences.

Fast Neural Style for Image Style Transform by Pytorch

DynaTune: Dynamic Tensor Program Optimization in Deep Neural Network Compilation

SplineConv implementation for Paddle.

Segmentation in Style: Unsupervised Semantic Image Segmentation with Stylegan and CLIP

Solving Zero-Shot Learning in Named Entity Recognition with Common Sense Knowledge

Make your own game in a font!

Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

Semantic Image Synthesis with SPADE

Justmagic - Use a function as a method with this mystic script, like in Nim

Zero-Shot Text-to-Image Generation VQGAN+CLIP Dockerized

Studying Python release adoptions by looking at PyPI downloads

Official implementation of the paper DeFlow: Learning Complex Image Degradations from Unpaired Data with Conditional Flows