A Python/Pytorch app for easily synthesising human voices

Overview

Voice Cloning App

CircleCI Discord codecov comment comment

A Python/Pytorch app for easily synthesising human voices

Preview

Documentation

Discord Server

Video guide

Voice Sharing Hub

FAQ's

System Requirements

  • Windows 10 or Ubuntu 20.04+ operating system
  • 5GB+ Disk space
  • NVIDIA GPU with at least 4GB of memory & driver version 456.38+ (optional)

Key features

  • Automatic dataset generation (with support for subtitles and audiobooks)
  • Additional language support
  • Local & remote training
  • Easy train start/stop
  • Data importing/exporting
  • Multi GPU support

Manual Guides

Future Improvements

  • Add support for Talknet
  • Add GTA alignment for Hifi-gan
  • Improved batch size estimation
  • AMD GPU support

Other resources

Acknowledgements

This project uses a reworked version of Tacotron2. All rights for belong to NVIDIA and follow the requirements of their BSD-3 licence.

Additionally, the project uses DSAlign, Silero, DeepSpeech & hifi-gan.

Thank you to Dr. John Bustard at Queen's University Belfast for his support throughout the project.

Supported by uberduck.ai, reach out to them for live model hosting.

Also a big thanks to the members of the VocalSynthesis subreddit for their feedback.

Finally thank you to everyone raising issues and contributing to the project.

Comments
  • Transcription error: wav file is empty

    Transcription error: wav file is empty

    Hello

    I am running the Voice-Cloning-App.exe on Windows 10. I have a GeForce RTX 2060 Graphics Card with the GeForce Game Ready Driver Version 461.92.

    When I attempt build the data set, the windows console stops after the following:

    [12644] WARNING: file already exists but should not: C:\Users\GREGOR~1\AppData\Local\Temp_MEI126442\torch_C.cp38-win_amd64.pyd Server initialized for threading. Server initialized for threading. pydub\utils.py:170: RuntimeWarning: Couldn't find ffmpeg or avconv - defaulting to ffmpeg, but may not work torchaudio\extension\extension.py:14: UserWarning: torchaudio C++ extension is not available. torchaudio\backend\utils.py:63: UserWarning: The interface of "soundfile" backend is planned to change in 0.8.0 to match that of "sox_io" backend and the current interface will be removed in 0.9.0. To use the new interface, do torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE = False before setting the backend to "soundfile". Please refer to https://github.com/pytorch/audio/issues/903 for the detail. INFO:matplotlib.font_manager:Generating new fontManager, this may take some time... [nltk_data] Downloading package wordnet to C:\Users\GREGOR~1\AppData\L [nltk_data] ocal\Temp_MEI126442\nltk_data... [nltk_data] Package wordnet is already up-to-date! WARNING:werkzeug:WebSocket transport not available. Install eventlet or gevent and gevent-websocket for improved performance.

    • Serving Flask app "main" (lazy loading)
    • Environment: production WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
    • Debug mode: off INFO:werkzeug: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:56:25] "GET / HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:56:57] "POST / HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:56:57] "GET /static/error.css HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:56:57] "GET /favicon.ico HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:11] "GET / HTTP/1.1" 200 - Starting Thread INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:42] "POST / HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet OPEN data {'sid': 'qINJoZN0iSsAW66FAAAA', 'upgrades': [], 'pingTimeout': 5000, 'pingInterval': 25000} INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet OPEN data {'sid': 'qINJoZN0iSsAW66FAAAA', 'upgrades': [], 'pingTimeout': 5000, 'pingInterval': 25000} INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:42] "GET /socket.io/?EIO=4&transport=polling&t=NXjKmkr HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Received packet MESSAGE data 0/voice, INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet MESSAGE data 0/voice, qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 0/voice,{"sid":"hvDlhnRAa1GAVtomAAAB"} INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 0/voice,{"sid":"hvDlhnRAa1GAVtomAAAB"} INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:42] "POST /socket.io/?EIO=4&transport=polling&t=NXjKmlA&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:42] "GET /socket.io/?EIO=4&transport=polling&t=NXjKmlB&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - emitting event "logs" to all [/voice] INFO:socketio.server:emitting event "logs" to all [/voice] qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading audio from data\datasets\JamesEarlJones\audio.mp3..."}] INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:57:47] "GET /socket.io/?EIO=4&transport=polling&t=NXjKmlb&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading audio from data\datasets\JamesEarlJones\audio.mp3..."}] INFO:voice:Loading audio from data\datasets\JamesEarlJones\audio.mp3... emitting event "logs" to all [/voice] INFO:socketio.server:emitting event "logs" to all [/voice] qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading script from data\datasets\JamesEarlJones\text.txt..."}] INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:02] "GET /socket.io/?EIO=4&transport=polling&t=NXjKnxd&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading script from data\datasets\JamesEarlJones\text.txt..."}] INFO:voice:Loading script from data\datasets\JamesEarlJones\text.txt... emitting event "logs" to all [/voice] INFO:socketio.server:emitting event "logs" to all [/voice] qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Fetching segments..."}] INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Fetching segments..."}] INFO:voice:Fetching segments... INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:02] "GET /socket.io/?EIO=4&transport=polling&t=NXjKrgH&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:07] "GET /socket.io/?EIO=4&transport=polling&t=NXjKrgS&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:07] "POST /socket.io/?EIO=4&transport=polling&t=NXjKsst&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - emitting event "logs" to all [/voice] INFO:socketio.server:emitting event "logs" to all [/voice] qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Transcribing segments..."}] INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:20] "GET /socket.io/?EIO=4&transport=polling&t=NXjKssu&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Transcribing segments..."}] INFO:voice:Transcribing segments... Using cache found in C:\Users\Gregory Betsey/.cache\torch\hub\snakers4_silero-models_master torchaudio\backend\utils.py:63: UserWarning: The interface of "soundfile" backend is planned to change in 0.8.0 to match that of "sox_io" backend and the current interface will be removed in 0.9.0. To use the new interface, do torchaudio.USE_SOUNDFILE_LEGACY_INTERFACE = False before setting the backend to "soundfile". Please refer to https://github.com/pytorch/audio/issues/903 for the detail. Exception in thread Thread-13: Traceback (most recent call last): File "application\utils.py", line 47, in background_task max_seqlength = max(max([len(_) for _ in batch]), 12800) File "application\utils.py", line 32, in create_dataset if wav.size(0) > 1: File "dataset\forced_alignment\align.py", line 123, in align File "dataset\transcribe.py", line 34, in stt File "dataset\transcribe.py", line 16, in transcribe File "torch\hub.py", line 370, in load File "torch\hub.py", line 399, in _load_local File "C:\Users\Gregory Betsey/.cache\torch\hub\snakers4_silero-models_master\hubconf.py", line 24, in silero_stt model, decoder = init_jit_model(model_url=models.stt_models.get(language).latest.jit, File "C:\Users\Gregory Betsey/.cache\torch\hub\snakers4_silero-models_master\utils.py", line 135, in init_jit_model model = torch.jit.load(model_path, map_location=device) File "torch\jit_serialization.py", line 161, in load RuntimeError: [enforce fail at ..\caffe2\serialize\inline_container.cc:145] . PytorchStreamReader failed reading zip archive: failed finding central directory

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "threading.py", line 932, in bootstrap_inner File "threading.py", line 870, in run File "application\utils.py", line 50, in background_task inputs[i, :len(wav)].copy(wav) NameError: name 'traceback' is not defined qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:32] "GET /socket.io/?EIO=4&transport=polling&t=NXjKvy4&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:32] "POST /socket.io/?EIO=4&transport=polling&t=NXjKyzw&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:57] "GET /socket.io/?EIO=4&transport=polling&t=NXjKyzw.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:58:57] "POST /socket.io/?EIO=4&transport=polling&t=NXjL358&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:59:22] "GET /socket.io/?EIO=4&transport=polling&t=NXjL358.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:59:22] "POST /socket.io/?EIO=4&transport=polling&t=NXjL9CA&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:59:47] "GET /socket.io/?EIO=4&transport=polling&t=NXjL9CB&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 00:59:47] "POST /socket.io/?EIO=4&transport=polling&t=NXjLFJ8&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:00:12] "GET /socket.io/?EIO=4&transport=polling&t=NXjLFJ8.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:00:12] "POST /socket.io/?EIO=4&transport=polling&t=NXjLLQ8&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:00:37] "GET /socket.io/?EIO=4&transport=polling&t=NXjLLQ9&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:00:37] "POST /socket.io/?EIO=4&transport=polling&t=NXjLRWz&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:02] "GET /socket.io/?EIO=4&transport=polling&t=NXjLRW-&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:02] "POST /socket.io/?EIO=4&transport=polling&t=NXjLXe3&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:27] "GET /socket.io/?EIO=4&transport=polling&t=NXjLXe4&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:27] "POST /socket.io/?EIO=4&transport=polling&t=NXjLdkv&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:52] "GET /socket.io/?EIO=4&transport=polling&t=NXjLdkv.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:01:52] "POST /socket.io/?EIO=4&transport=polling&t=NXjLjrp&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:02:17] "GET /socket.io/?EIO=4&transport=polling&t=NXjLjrq&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:02:17] "POST /socket.io/?EIO=4&transport=polling&t=NXjLpyh&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:02:42] "GET /socket.io/?EIO=4&transport=polling&t=NXjLpyh.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Sending packet PING data None qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:engineio.server:qINJoZN0iSsAW66FAAAA: Received packet PONG data INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:02:42] "POST /socket.io/?EIO=4&transport=polling&t=NXjLw3g&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1" 200 - qINJoZN0iSsAW66FAAAA: Sending packet PING data None INFO:werkzeug:127.0.0.1 - - [26/Mar/2021 01:03:07] "qINJoZN0iSsAW66FAAAA: Received packet CLOSE data GET /socket.io/?EIO=4&transport=polling&t=NXjLw3g.0&sid=qINJoZN0iSsAW66FAAAA HTTP/1.1qINJoZN0iSsAW66FAAAA: Client is gone, closing socket Error.txt

    bug 
    opened by GregoryBetsey 58
  • GPU Memory exhausted (training)

    GPU Memory exhausted (training)

    Label for this issue is help wanted.

    Hardware

    Processor: Intel Core i9 - 9900K CPU @ 3.60GHz Installed RAM: 64.0 GB GPU: NVIDIA GeForce RTX 2070 - 8GB


    Attempt

    command


    Failure

    gpu blasted


    Tried Steps

    1. Epoch selection:
      • 8000
      • 5000
      • 3000
      • 1000 and 10 iterations

    Steps that might help to resolve

    I found some articles that might be be helpful to resolve this:

    1. https://pytorch.org/docs/stable/notes/faq.html
    2. https://forums.fast.ai/t/clearing-gpu-memory-pytorch/14637/4

    Questions

    1. Would you suggest to reduce the iterator counter from 1000 to a smaller number? If yes then what number would be ideal.
    2. Would you suggest to reduce the epochs from 8000 to a smaller number? If yes then what number would be ideal.
    bug 
    opened by vinamramunot-tech 15
  • HTTPError

    HTTPError

    First time user - after uploading .txt and .wav I get the following error. Any guidance?

    Type: HTTPError Text: HTTP Error 503: Service Temporarily Unavailable Full: Traceback (most recent call last): File "flask\app.py", line 1950, in full_dispatch_request File "flask\app.py", line 1936, in dispatch_request File "application\views.py", line 99, in create_dataset_post File "dataset\transcribe.py", line 85, in create_transcription_model File "dataset\transcribe.py", line 58, in init File "torch\hub.py", line 364, in load model = _load_local(repo_or_dir, model, *args, **kwargs) File "torch\hub.py", line 393, in _load_local model = entry(*args, **kwargs) File "C:\Users\Sam London/.cache\torch\hub\snakers4_silero-models_master\hubconf.py", line 28, in silero_stt **kwargs) File "C:\Users\Sam London/.cache\torch\hub\snakers4_silero-models_master\utils.py", line 128, in init_jit_model progress=True) File "torch\hub.py", line 419, in download_url_to_file u = urlopen(req) File "urllib\request.py", line 223, in urlopen File "urllib\request.py", line 532, in open File "urllib\request.py", line 642, in http_response File "urllib\request.py", line 570, in error File "urllib\request.py", line 504, in _call_chain File "urllib\request.py", line 650, in http_error_default urllib.error.HTTPError: HTTP Error 503: Service Temporarily Unavailable

    opened by ManBearPig87 13
  • Training unexpectedly crashes

    Training unexpectedly crashes

    Okay I have my wavs folder and my metadata.csv, all is working good except that when I train it loads all data (I can see it on the "embedded console", and then after some seconds opens a new tab in my browser with the main page (localhost:5000). If I switch to the training tab and I wait for some time it doesn't make any progress. Can you help me with this please?

    bug 
    opened by gbh4x 12
  • Error fetching silero model

    Error fetching silero model

    Full: Traceback (most recent call last): File "C:\Users\superuser\Anaconda3\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "C:\Users\superuser\Anaconda3\lib\site-packages\flask\app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "C:\Users\superuser\Desktop\Voice-Cloning-App\application\views.py", line 249, in synthesis_post alertnative_words=get_alternative_word_suggestions(audio_path, text), File "C:\Users\superuser\Desktop\Voice-Cloning-App\synthesis\synonyms.py", line 29, in get_alternative_word_suggestions poor_words = evalulate_audio(audio, text) File "C:\Users\superuser\Desktop\Voice-Cloning-App\synthesis\synonyms.py", line 20, in evalulate_audio results = transcribe(audio) File "C:\Users\superuser\Desktop\Voice-Cloning-App\dataset\transcribe.py", line 27, in transcribe repo_or_dir="snakers4/silero-models", model="silero_stt", language="en", device=device File "C:\Users\superuser\Anaconda3\lib\site-packages\torch\hub.py", line 370, in load model = _load_local(repo_or_dir, model, *args, **kwargs) File "C:\Users\superuser\Anaconda3\lib\site-packages\torch\hub.py", line 399, in _load_local model = entry(*args, **kwargs) File "C:\Users\superuser/.cache\torch\hub\snakers4_silero-models_master\hubconf.py", line 25, in silero_stt **kwargs) File "C:\Users\superuser/.cache\torch\hub\snakers4_silero-models_master\utils.py", line 133, in init_jit_model progress=True) File "C:\Users\superuser\Anaconda3\lib\site-packages\torch\hub.py", line 445, in download_url_to_file unit='B', unit_scale=True, unit_divisor=1024) as pbar: File "C:\Users\superuser\Anaconda3\lib\site-packages\tqdm_tqdm.py", line 662, in init TqdmKeyError("Unknown argument(s): " + str(kwargs))) tqdm._tqdm.TqdmKeyError: "Unknown argument(s): {'unit_divisor': 1024}"

    bug 
    opened by CrazyPlaysHD 9
  • Force Align text and Audio (dataset)

    Force Align text and Audio (dataset)

    Hi @BenAAndrew I am on the step where I am trying to align the text and audio of an audiobook. I have acquired the audio and text from amazon audible. Unfortunately, I was not able to assign the help label to this issue. I don't think I have the permission for that.

    • [X] using virtualenv

    In order to work through align.py, I had to modify it. After modifying I was able to run the file. Below is the modified part of the file. Also in the screenshot category I have mentioned how I am trying to execute this file.

    import os
    import sys
    import json
    import logging
    import argparse
    from pydub import AudioSegment
    
    sys.path.append(".")
    
    from search import FuzzySearch
    from audio import DEFAULT_RATE, read_frames_from_file, vad_split
    from dataset.transcribe import stt
    

    Screenshots

    Failure Point

    failure point


    Questions

    1. Do you suggest to use a virtualenv?
    2. Do I need to reduce the quality of wav file or the mp3 file?

    Link to the dataset

    Audio Dataset

    I have the book.txt and the mp3 file. I have converted that mp3 to wav file when I am trying to use the align. Please let me know if you can try using my dataset. Thanks for the help in advance.

    bug question 
    opened by vinamramunot-tech 9
  • All unlabeled clips unplayable in the Manage datasets interface for a freshly built dataset.

    All unlabeled clips unplayable in the Manage datasets interface for a freshly built dataset.

    This is with VCA release 1.0.2 using the compiled executable on Windows 10. The play option is simply grayed out for all listed clips. Have checked, and all the listed clips are playable from within file explorer, and in the case of this dataset, ranging in length from 1-6 seconds.

    bug 
    opened by RayDAnt3D 8
  • Imported model cannot be loaded

    Imported model cannot be loaded

    Was exporting a model beforehand. (Using a google colab notebook.)

    Type: ValueError Text: invalid literal for int() with base 10: 'checkpoints' Full: Traceback (most recent call last): File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/dist-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/content/Voice-Cloning-App/application/views.py", line 391, in download_model model_path = get_latest_checkpoint(os.path.join(paths["models"], model_name)) File "/content/Voice-Cloning-App/training/checkpoint.py", line 27, in get_latest_checkpoint if int(checkpoint.split("")[1].split(".")[0]) > int(latest_checkpoint.split("")[1].split(".")[0]): ValueError: invalid literal for int() with base 10: 'checkpoints'

    bug 
    opened by ericstheguy 8
  • Hang-up when attempting to process dataset on Ampere card [dataset]

    Hang-up when attempting to process dataset on Ampere card [dataset]

    Description: Application freezes during dataset preparation process, I believe this is due to the bundled version of pytorch not supporting ampere cards.

    Version used: 0.6.1 (Windows Executable)

    OS: Win 10 21H1

    Device Specificiations: r9 3900x, dual 3090s, 32gb system memory

    Nvidia Driver version: 461.92

    Log:

    INFO:werkzeug: * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:24] "GET / HTTP/1.1" 200 -
    Starting Thread
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:48] "POST / HTTP/1.1" 200 -
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:48] "GET /static/application.js HTTP/1.1" 200 -
    PQVUzoQCJzc-rrA_AAAA: Sending packet OPEN data {'sid': 'PQVUzoQCJzc-rrA_AAAA', 'upgrades': [], 'pingTimeout': 5000, 'pingInterval': 25000}
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet OPEN data {'sid': 'PQVUzoQCJzc-rrA_AAAA', 'upgrades': [], 'pingTimeout': 5000, 'pingInterval': 25000}
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:48] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0WEU HTTP/1.1" 200 -
    PQVUzoQCJzc-rrA_AAAA: Received packet MESSAGE data 0/voice,
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Received packet MESSAGE data 0/voice,
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 0/voice,{"sid":"iBJGih1vKc84-OLJAAAB"}
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 0/voice,{"sid":"iBJGih1vKc84-OLJAAAB"}
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:48] "POST /socket.io/?EIO=4&transport=polling&t=NZQ0WII&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:48] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0WII.0&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Coverting data\\datasets\\Diana\\audio.mp3..."}]
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:53] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0WN2&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Coverting data\\datasets\\Diana\\audio.mp3..."}]
    INFO:voice:Coverting data\datasets\Diana\audio.mp3...
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading script from data\\datasets\\Diana\\text.txt..."}]
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:56] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0XNM&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Loading script from data\\datasets\\Diana\\text.txt..."}]
    INFO:voice:Loading script from data\datasets\Diana\text.txt...
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Searching text for matching fragments..."}]
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Searching text for matching fragments..."}]
    INFO:voice:Searching text for matching fragments...
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:56] "emitting event "logs" to all [/voice]
    GET /socket.io/?EIO=4&transport=polling&t=NZQ0YFA&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Changing sample rate..."}]
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Changing sample rate..."}]
    INFO:voice:Changing sample rate...
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:56] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0YFR&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Fetching segments..."}]
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:57] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0YKC&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Fetching segments..."}]
    INFO:voice:Fetching segments...
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Matching segments..."}]
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:57] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0YOL&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Matching segments..."}]
    INFO:voice:Matching segments...
    emitting event "logs" to all [/voice]
    INFO:socketio.server:emitting event "logs" to all [/voice]
    PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Generating segments..."}]
    INFO:engineio.server:PQVUzoQCJzc-rrA_AAAA: Sending packet MESSAGE data 2/voice,["logs",{"text":"Generating segments..."}]
    INFO:voice:Generating segments...
    INFO:werkzeug:127.0.0.1 - - [16/Apr/2021 10:27:57] "GET /socket.io/?EIO=4&transport=polling&t=NZQ0YYk&sid=PQVUzoQCJzc-rrA_AAAA HTTP/1.1" 200 -
    Using cache found in C:\Users\thefi/.cache\torch\hub\snakers4_silero-models_master
    torch\cuda\__init__.py:104: UserWarning:
    GeForce RTX 3090 with CUDA capability sm_86 is not compatible with the current PyTorch installation.
    The current PyTorch install supports CUDA capabilities sm_37 sm_50 sm_60 sm_61 sm_70 sm_75 compute_37.
    If you want to use the GeForce RTX 3090 GPU with PyTorch, please check the instructions at https://pytorch.org/get-started/locally/
    
    bug 
    opened by LexCybermac 8
  • cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

    cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

    i m trying use my own audio and text, this is the error i m having.

    cannot reshape tensor of 0 elements into shape [-1, 0] because the unspecified dimension size -1 can be any value and is ambiguous

    using windows version. python version is giving another error.

    File "main.py", line 4, in from engineio.async_drivers import threading ModuleNotFoundError: No module named 'engineio'

    I m new to both worlds, is there anything I m not doing?

    bug 
    opened by Syed044 8
  • Invalid characters in text

    Invalid characters in text

    I tried training with it and I got this: Invalid characters in text (for alphabet): ” (RIGHT DOUBLE QUOTATION MARK),­ (SOFT HYPHEN),’ (RIGHT SINGLE QUOTATION MARK),“ (LEFT DOUBLE QUOTATION MARK),— (EM DASH) and it refused to start. Does this mean the text shouldn't have punctuations anymore with this version? If I change to this version will I have to start all over?

    question 
    opened by LeeroyJenkinsss 7
  • Get Audio file via Command

    Get Audio file via Command

    I use this command to get an audio file from cli but here the audio file Im getting is not saying anything at all, via GUI it works perfectly. Im using a more extended alphabet than english python synthesis/synthesize.py -m data/models/..... -vm "data/hifigan/vocoder/model.pt" -hc "data/hifigan/vocoder/config.json" -t "test this is a test" -a audio.wav Any idea? Thanks in advance

    opened by miguelgh65 2
  • JSONDecodeError

    JSONDecodeError

    When I go to synthesize and submit, this is the error message that appears: Text: Expecting value: line 1 column 1 (char 0) Full: Traceback (most recent call last): File "flask\app.py", line 1950, in full_dispatch_request File "flask\app.py", line 1936, in dispatch_request File "application\views.py", line 302, in synthesis_setup_post File "synthesis\vocoders\hifigan.py", line 25, in init File "json_init_.py", line 354, in loads File "json\decoder.py", line 339, in decode File "json\decoder.py", line 357, in raw_decode json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

    opened by skydam 1
  • App is not starting.

    App is not starting.

    I installed all requirements with python 3.6.15. When i try to start app i am getting this error

    Traceback (most recent call last):
      File "main.py", line 52, in <module>
        from application.views import *  # noqa
      File "/home/libir/Desktop/voiceCloning/Voice-Cloning-App/application/views.py", line 11, in <module>
        from application.utils import (
      File "/home/libir/Desktop/voiceCloning/Voice-Cloning-App/application/utils.py", line 8, in <module>
        import librosa
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/librosa/__init__.py", line 12, in <module>
        from . import core
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/librosa/core/__init__.py", line 103, in <module>
        from .audio import *  # pylint: disable=wildcard-import
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/librosa/core/audio.py", line 12, in <module>
        import resampy
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/resampy/__init__.py", line 7, in <module>
        from .core import *
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/resampy/core.py", line 9, in <module>
        from .interpn import resample_f_s, resample_f_p
      File "/home/libir/.pyenv/versions/3.6.15/lib/python3.6/site-packages/resampy/interpn.py", line 75, in <module>
        nopython=True,
    TypeError: guvectorize() missing 1 required positional argument: 'signature'
    

    Also i am using EndeavourOS.

    opened by LibirSoft 3
  • Is it possible for me to extract a runtime/environment from this?

    Is it possible for me to extract a runtime/environment from this?

    I can't find a python runtime/environment in the temp folder for this program. This is important as installing a correct environment to run tacotron2 is the biggest pain in the world, and I want to use the voice-cloning-app environment to run tacotron2 in another program, without being restricted to using the app. I could not find a single trace of python.exe, but i did find python related files, so how is it able to run? What is it doing to call the synthesis script?

    opened by FlashlightET 1
Releases(v1.1.1)
Owner
Ben Andrew
Developer working on open source Machine Learning, IoT, CAD & electronics projects.
Ben Andrew
fastai ulmfit - Pretraining the Language Model, Fine-Tuning and training a Classifier

fast.ai ULMFiT with SentencePiece from pretraining to deployment Motivation: Why even bother with a non-BERT / Transformer language model? Short answe

Florian Leuerer 26 May 27, 2022
Pytorch NLP library based on FastAI

Quick NLP Quick NLP is a deep learning nlp library inspired by the fast.ai library It follows the same api as fastai and extends it allowing for quick

Agis pof 283 Nov 21, 2022
A versatile token stream for handwritten parsers.

Writing recursive-descent parsers by hand can be quite elegant but it's often a bit more verbose than expected, especially when it comes to handling indentation and reporting proper syntax errors. Th

Valentin Berlier 8 Nov 30, 2022
Long text token classification using LongFormer

Long text token classification using LongFormer

abhishek thakur 161 Aug 07, 2022
Mastering Transformers, published by Packt

Mastering Transformers This is the code repository for Mastering Transformers, published by Packt. Build state-of-the-art models from scratch with adv

Packt 195 Jan 01, 2023
Script and models for clustering LAION-400m CLIP embeddings.

clustering-laion400m Script and models for clustering LAION-400m CLIP embeddings. Models were fit on the first million or so image embeddings. A subje

Peter Baylies 22 Oct 04, 2022
Negative sampling for solving the unlabeled entity problem in NER. ICLR-2021 paper: Empirical Analysis of Unlabeled Entity Problem in Named Entity Recognition.

Negative Sampling for NER Unlabeled entity problem is prevalent in many NER scenarios (e.g., weakly supervised NER). Our paper in ICLR-2021 proposes u

Yangming Li 128 Dec 29, 2022
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Model

303 Dec 17, 2022
Sentiment-Analysis and EDA on the IMDB Movie Review Dataset

Sentiment-Analysis and EDA on the IMDB Movie Review Dataset The main part of the work focuses on the exploration and study of different approaches whi

Nikolas Petrou 1 Jan 12, 2022
Simple Text-To-Speech Bot For Discord

Simple Text-To-Speech Bot For Discord This is a very simple TTS bot for discord made with python. For this bot you need FFMPEG, see installation to se

1 Sep 26, 2022
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization (ACL 2021)

Structured Super Lottery Tickets in BERT This repo contains our codes for the paper "Super Tickets in Pre-Trained Language Models: From Model Compress

Chen Liang 16 Dec 11, 2022
Code for "Parallel Instance Query Network for Named Entity Recognition", accepted at ACL 2022.

README Code for Two-stage Identifier: "Parallel Instance Query Network for Named Entity Recognition", accepted at ACL 2022. For details of the model a

Yongliang Shen 45 Nov 29, 2022
AI and Machine Learning workflows on Anthos Bare Metal.

Hybrid and Sovereign AI on Anthos Bare Metal Table of Contents Overview Terraform as IaC Substrate ABM Cluster on GCE using Terraform TensorFlow ResNe

Google Cloud Platform 8 Nov 26, 2022
COVID-19 Chatbot with Rasa 2.0: open source conversational AI

COVID-19 chatbot implementation with Rasa open source 2.0, conversational AI framework.

Aazim Parwaz 1 Dec 23, 2022
Türkçe küfürlü içerikleri bulan bir yapay zeka kütüphanesi / An ML library for profanity detection in Turkish sentences

"Kötü söz sahibine aittir." -Anonim Nedir? sinkaf uygunsuz yorumların bulunmasını sağlayan bir python kütüphanesidir. Farkı nedir? Diğer algoritmalard

KaraGoz 4 Feb 18, 2022
Sentello is python script that simulates the anti-evasion and anti-analysis techniques used by malware.

sentello Sentello is a python script that simulates the anti-evasion and anti-analysis techniques used by malware. For techniques that are difficult t

Malwation 62 Oct 02, 2022
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism (SVS & TTS); AAAI 2022

DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism This repository is the official PyTorch implementation of our AAAI-2022 paper, in

Jinglin Liu 829 Jan 07, 2023
An example project using OpenPrompt under pytorch-lightning for prompt-based SST2 sentiment analysis model

pl_prompt_sst An example project using OpenPrompt under the framework of pytorch-lightning for a training prompt-based text classification model on SS

Zhiling Zhang 5 Oct 21, 2022
Translate - a PyTorch Language Library

NOTE PyTorch Translate is now deprecated, please use fairseq instead. Translate - a PyTorch Language Library Translate is a library for machine transl

775 Dec 24, 2022
Transformer related optimization, including BERT, GPT

This repository provides a script and recipe to run the highly optimized transformer-based encoder and decoder component, and it is tested and maintained by NVIDIA.

NVIDIA Corporation 1.7k Jan 04, 2023