Video-Music Transformer

Last update: Jul 13, 2022

Related tags

Deep Learning VMT

Overview

VMT

Video-Music Transformer (VMT) is an attention-based multi-modal model, which generates piano music for a given video.

Paper

https://arxiv.org/abs/2112.15320

Demo

Here is an example video fragments from our dataset. Note that we do not do any post-production. Each file is made from the original video with a WAVE file converted from the MIDI of the model output.

Original

The original music of the videos.

100-001_original.mp4

VMT

The music generated by our VMT model.

100-001_vmt.mp4

Seq2Seq

The musics generated by the baseline Seq2Seq model.

100-001_seq2seq.mp4

Owner

Chin-Tung Lin

GitHub Repository

Supervised Classification from Text (P)

MSc-Thesis Module: Masters Research Thesis Language: Python Grade: 75 Title: An investigation of supervised classification of therapeutic process from

1 Nov 22, 2021

Clinica is a software platform for clinical research studies involving patients with neurological and psychiatric diseases and the acquisition of multimodal data

Clinica Software platform for clinical neuroimaging studies Homepage | Documentation | Paper | Forum | See also: AD-ML, AD-DL ClinicaDL About The Proj

165 Dec 29, 2022

PyTorch Implementation of PIXOR: Real-time 3D Object Detection from Point Clouds

PIXOR: Real-time 3D Object Detection from Point Clouds This is a custom implementation of the paper from Uber ATG using PyTorch 1.0. It represents the

270 Dec 14, 2022

Diverse Image Generation via Self-Conditioned GANs

Diverse Image Generation via Self-Conditioned GANs Project | Paper Diverse Image Generation via Self-Conditioned GANs Steven Liu, Tongzhou Wang, David

147 Dec 03, 2022

When BERT Plays the Lottery, All Tickets Are Winning

When BERT Plays the Lottery, All Tickets Are Winning Large Transformer-based models were shown to be reducible to a smaller number of self-attention h

16 Nov 10, 2022

BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer

BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer Project Page | Paper | Video State-of-the-art image-to-image translatio

47 Dec 06, 2022

Official repository for Automated Learning Rate Scheduler for Large-Batch Training (8th ICML Workshop on AutoML)

Automated Learning Rate Scheduler for Large-Batch Training The official repository for Automated Learning Rate Scheduler for Large-Batch Training (8th

35 Jan 04, 2023

Adversarial Self-Defense for Cycle-Consistent GANs

Adversarial Self-Defense for Cycle-Consistent GANs This is the official implementation of the CycleGAN robust to self-adversarial attacks used in pape

10 Oct 10, 2022

AugLy is a data augmentations library that currently supports four modalities (audio, image, text & video) and over 100 augmentations

AugLy is a data augmentations library that currently supports four modalities (audio, image, text & video) and over 100 augmentations. Each modality’s augmentations are contained within its own sub-l

4.6k Jan 09, 2023

An open source Python package for plasma science that is under development

PlasmaPy PlasmaPy is an open source, community-developed Python 3.7+ package for plasma science. PlasmaPy intends to be for plasma science what Astrop

444 Jan 07, 2023

A brand new hub for Scene Graph Generation methods based on MMdetection (2021). The pipeline of from detection, scene graph generation to downstream tasks (e.g., image cpationing) is supported. Pytorch version implementation of HetH (ECCV 2020) and TopicSG (ICCV 2021) is included.

MMSceneGraph Introduction MMSceneneGraph is an open source code hub for scene graph generation as well as supporting downstream tasks based on the sce

39 Dec 17, 2022

Video-Music Transformer

Related tags

Overview

VMT

Paper

Demo

Original

VMT

Seq2Seq

Owner

Chin-Tung Lin

Supervised Classification from Text (P)

Clinica is a software platform for clinical research studies involving patients with neurological and psychiatric diseases and the acquisition of multimodal data

PyTorch Implementation of PIXOR: Real-time 3D Object Detection from Point Clouds

Diverse Image Generation via Self-Conditioned GANs

When BERT Plays the Lottery, All Tickets Are Winning

BalaGAN: Image Translation Between Imbalanced Domains via Cross-Modal Transfer

Official repository for Automated Learning Rate Scheduler for Large-Batch Training (8th ICML Workshop on AutoML)

Adversarial Self-Defense for Cycle-Consistent GANs

AugLy is a data augmentations library that currently supports four modalities (audio, image, text & video) and over 100 augmentations

An open source Python package for plasma science that is under development

A brand new hub for Scene Graph Generation methods based on MMdetection (2021). The pipeline of from detection, scene graph generation to downstream tasks (e.g., image cpationing) is supported. Pytorch version implementation of HetH (ECCV 2020) and TopicSG (ICCV 2021) is included.

NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations

PyTorch implementations of the paper: "Learning Independent Instance Maps for Crowd Localization"

Realtime_Multi-Person_Pose_Estimation

N-HiTS: Neural Hierarchical Interpolation for Time Series Forecasting

audioLIME: Listenable Explanations Using Source Separation

PyTorch/GPU re-implementation of the paper Masked Autoencoders Are Scalable Vision Learners

👨‍💻 run nanosaur in simulation with Gazebo/Ingnition

[CVPR 2021] Involution: Inverting the Inherence of Convolution for Visual Recognition, a brand new neural operator

An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)