Contrastive Language-Audio Pretraining

Overview

CLAP

Contrastive Language-Audio Pretraining

In due time this repo will be full of lovely things, I hope.

Feel free to check out the Issues if you're interested in contributing. Leave a note saying what interests you. :)

from jax import random, numpy as np
from clap.clap import CLAP

key1, key2 = random.split(random.PRNGKey(0), 2)

text = random.randint(key1, (2, 16,), 0, 256)
audio = random.uniform(key1, (2, 8, 512))

text_mask = np.ones((2, 16), dtype = bool)
audio_mask = np.ones((2, 8), dtype = bool)

model = CLAP(
    text_vocab = 256,
    text_dim = 512,
    text_depth = 6,
    text_heads = 8,
    audio_dim = 512,
    audio_depth = 6,
    audio_heads = 8
)

params = model.init(key2, text, audio, text_mask, audio_mask)
loss = model.apply(params, text, audio, text_mask, audio_mask)

# after a lot of training

sim = model.apply(params, text, audio, text_mask, audio_mask, return_loss = False) # (2, 2)
Owner
Charles Foster
Charles Foster
Financial portfolio optimisation in python, including classical efficient frontier, Black-Litterman, Hierarchical Risk Parity

PyPortfolioOpt has recently been published in the Journal of Open Source Software 🎉 PyPortfolioOpt is a library that implements portfolio optimizatio

Robert Martin 3.2k Jan 02, 2023
This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;).

CSGO-Python-Cheat This python cheat utilizes PyMeow, PyMem, and others to enhance your CS:GO experience ;). Features Esp Tracers Chams (More to come)

Addi 1 Nov 30, 2021
Discord-disnake - This package allows to use disnake without changing the discord namespace

This package is a shim This module allows to use disnake using discord namespace. This is not an independent library. Installing Python 3.8 or higher

5 Dec 13, 2022
A web app via which users can buy and sell stocks using virtual money

finance Virtual Stock Trader. A web app via which users can buy and sell stocks using virtual money. All stock prices are real and provided by IEX. Fe

Kiron Deb 0 Jan 15, 2022
twitter bot tha uses tweepy library class to connect to TWITTER API

TWITTER-BOT-tweepy- twitter bot that uses tweepy library class to connect to TWITTER API replies to mentions automatically and follows the tweet.autho

Muziwandile Nkomo 2 Jan 08, 2022
⛑ REDCap API interface in Python

REDCap API in Python Description Supports structured data extraction for REDCap projects. The API module d3b_redcap_api.redcap.REDCapStudy can be logi

D3b 1 Nov 21, 2022
The Simple Google Colab Notebook to Download Files from Direct Link to Google Drive with custom name and bulk link support.

Direct Link to Google Drive (Advanced! 🔥 ) The Most Advanced yet Simple Google Colab Notebook to Download Files from Direct Link to Google Drive. 🆕

Dr.Caduceus 14 Jul 26, 2022
Mailjet API implementation in Python

READ THIS FIRST!! This repository isn't compatible with the current Mailjet API (v3) and, as a consequence, is considered deprecated and won't undergo

Rick van Hattem 18 Oct 21, 2022
A discord bot written in discord.py to manage custom roles assigned to boosters of your server.

BBotty A discord bot written in discord.py to manage custom roles assigned to boosters of your server. v0.0.1-alpha released! This version is incomple

Oui002 1 Nov 27, 2021
Un bot leggero basato su py-cord facile da hostare sul cloud

GalbiBot Un bot leggero basato su py-cord facile da hostare sul cloud Guida installazione su una macchina Per far funzionare il bot devi aver installa

Galbaninoh 2 Oct 21, 2022
CVE-2021-39685 Description and sample exploit for Linux USB Gadget overflow vulnerability

CVE-2021-39685 Description and sample exploit for Linux USB Gadget overflow vulnerability

8 May 25, 2022
Music bot for playing music on telegram voice chat group.

Somali X Music 🎵 Music bot for playing music on telegram voice chat group. Requirements FFmpeg NodeJS nodesource.com Python 3.8+ or Higher PyTgCalls

Abdisamad Omar Mohamed 4 Dec 01, 2021
A Discord bot for viewing any currency you want comfortably.

Dost Dost is a Discord bot for viewing currencies. Getting Started These instructions will get you a copy of the project up and running on your local

Baran Gökalp 2 Jan 18, 2022
HackZ-Token-Grabber-V2 - HackZ Token Grabber V2

HackZ-Token-Grabber-V2 was made by Love ❌ code ✅ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ ‎ 🌟

! ™NightMare 2 Mar 01, 2022
A powerful discord bot for forming team.

Discord_SquadBot A powerful discord bot for forming team. Pre-requirement Python 3.7 and latest Discord.py module is required. Installation guideline

Jacky Yu 2 Jan 29, 2022
Discord bot ( discord.py ), uses pandas library from python for data-management.

Discord_bot A Best and the most easy-to-use Discord bot !! Some simple basic auto moderations, Chat functions. It includes a game similar to Casino, g

Jaitej 4 Aug 30, 2022
微信支付接口V3版python库

wechatpayv3 介绍 微信支付接口V3版python库。 适用对象 wechatpayv3支持微信支付直连商户,接口说明详见 官网。 特性 平台证书自动更新,无需开发者关注平台证书有效性,无需手动下载更新; 支持本地缓存平台证书,初始化时指定平台证书保存目录即可。 适配进度 微信支付V3版A

chen gang 258 Jan 06, 2023
Maubot azuracast - A maubot to fetch data from your radio station

Maubot Azuracast A maubot to fetch data from your radio station Setup Configure

3 Mar 14, 2022
Python binding to the OneTimeSecret API

Thin Python binding for onetimesecret.com API. Unicode-safe. Description of API itself you can find here: https://onetimesecret.com/docs/api Usage:

Vladislav Stepanov 10 Jun 12, 2022
Brute force instagram account / actonetor, 2021

Brute force instagram account / actonetor, 2021

actonetor 6 Nov 16, 2022