This is an example of building a video Question-Answer system using Jina.

Related tags

Videoexample-video-qa
Overview

example-video-search

This is an example of building a video Question-Answer system using Jina.

The index data is subtitle files of YouTube videos. After indexing, you can query with questions in natural langurage and retrieve the related video together with the timestamp that the corresponding answer appears.

Prerequisites

pip install -r requirements.txt

Usage

By default, we index the subtitle file, toy-data/zvXkQkqd2I8.vtt

python app.py -m index

Query with questions,

python app.py -m query

To start the video UI, run the following codes and open http://localhost:3000/video/ in your browser.

git clone https://github.com/jina-ai/jina-ui.git
cd jina-ui
git checkout showcase-video-search
yarn install
yarn jinajs build
yarn showcases dev

How it works

The index flow is as below. The sentences are extracted from the subtitle file. The meta information of the sentences are stored in the LMDBStorage. In the other pathway, the sentences of the subtitles are encoded by the DPRTextEncoder and afterwards the result embeddings are stored in the SimpleIndexer.

The query flow is as shown below.

  1. The input query is a question which is encoded into embeddings by using DPRTextEncoder.
  2. The embedding of the query question is used to retrieve the sentences from SimpleIndexer.
  3. After retrieving the indexed sentences, we use MatchExpander to extend the questions so that the sentences from the neighboring timestamps are considered as candidate sentences as well.
  4. The candidate sentences retrieved from the above steps have no text or meta information. We retrieve the meta information from LMDBStorage.
  5. Rank the candidate sentences and extract the exact answers from the sentences by using DPRReaderRanker.
  6. Get the timestamp and video uri information about the answer candidates with Text2Frame

How to index my own data? [W.I.P.]

  1. download the subtitle files
youtube-dl --write-auto-sub --skip-download https://www.youtube.com/watch\?v\=zvXkQkqd2I8 -o data/zvXkQkqd2I8
  1. run the following
python app.py -m index -d data
python app.py -m query
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
Telegram Video Stream

Video Stream An Advanced VC Video Player created for playing video in the voice chats of Telegram Groups And Channel Configs TOKEN - Get bot token fro

mr_lokaman 46 Dec 25, 2022
I have baked a custom integration to control Eufy Security Cameras and access RTSP and P2P stream if possible.

I have baked a custom integration to control Eufy Security Cameras and access RTSP (real time streaming protocol) and P2P (peer to peer) stream if pos

Fuat Akgün 422 Jan 01, 2023
A platform which give you info about the newest video on a channel

youtube A platform which give you info about the newest video on a channel. This uses web scraping, a better implementation will be to use the API. BR

Custom components for Home Assistant 36 Sep 29, 2022
Your self hosted Youtube media server

The Tube Archivist Your self hosted Youtube media server Core functionality Subscribe to your favourite Youtube channels Download Videos using yt-dlp

Simon 2.1k Dec 31, 2022
A Advanced Anime Theme VC Video Player created for playing vidio in the voice chats of Telegram Groups

Yui Vidio Player A Advanced Anime Theme VC Video Player created for playing vidio in the voice chats of Telegram Groups Demo Setting up Add this Bot t

Achu biju 32 Sep 16, 2021
A way to run youtube videos in TTY

TTY youtube client its finally here, the one thing literally NO ONE ASKED FOR!! A way to run youtube videos in TTY Dependencies: (pip) yt-search (syst

1 Nov 28, 2021
Your own movie streaming service. Easy to install, easy to use. Download, manage and watch your favorite movies conveniently from your browser or phone. Install it on your server, access it anywhere and enjoy.

Vigilio Your own movie streaming service. Easy to install, easy to use. Download, manage and watch your favorite movies conveniently from your browser

Tugcan Olgun 141 Jan 06, 2023
pyffstream - A CLI frontend for streaming over SRT and RTMP specializing in sending off files

pyffstream - A CLI frontend for streaming over SRT and RTMP specializing in sending off files

Gregory Beauregard 3 Mar 04, 2022
Video-to-GIF-Converter - A small code snippet that can be used to convert any video to a gif

Video to GIF Converter Project Description: This is a small code snippet that ca

Hassan Shahzad 3 Jun 22, 2022
A tool to fuck a video/audio quality using FFmpeg

Media quality fucker A tool to fuck a video/audio quality using FFmpeg How to use Download the source Download Python Extract FFmpeg Put what you want

Maizena 8 Jan 25, 2022
Code for the manim-generated scenes used in 3blue1brown videos

Code for the manim-generated scenes used in 3blue1brown videos

Grant Sanderson 4.1k Jan 09, 2023
Video Editor for Linux

Project on break until late March. NEW RELEASE 2.8 IS OUT NOW. INSTALLING: see here. RELEASE NOTES AVAILABLE here. Introduction Features Releases Inst

1.9k Jan 07, 2023
This will help you study and avoid moving mouse coz u dont need mouse for watching youtube videos

This will help you study and avoid moving mouse coz u dont need mouse for watching youtube videos. Neither u need it for pdfs just use your keyboard

KorryKatti 5 Jan 07, 2022
Play Video & Music on Telegram Group Video Chat

🖤 DEMONGIRL 🖤 ʜᴇʟʟᴏ ❤️ 🇱🇰 Join us ᴠɪᴅᴇᴏ sᴛʀᴇᴀᴍ ɪs ᴀɴ ᴀᴅᴠᴀɴᴄᴇᴅ ᴛᴇʟᴇʀᴀᴍ ʙᴏᴛ ᴛʜᴀᴛ's ᴀʟʟᴏᴡ ʏᴏᴜ ᴛᴏ ᴘʟᴀʏ ᴠɪᴅᴇᴏ & ᴍᴜsɪᴄ ᴏɴ ᴛᴇʟᴇɢʀᴀᴍ ɢʀᴏᴜᴘ ᴠɪᴅᴇᴏ ᴄʜᴀᴛ 🧪 ɢ

Jonathan 5 Dec 31, 2021
Converts Betaflight blackbox gyro to MP4 GoPro Meta data so it can be used with ReelSteady GO

Here are a bunch of scripts that I created some time ago as a proof of concept that Betaflight blackbox gyro data can be converted to GoPro Metadata F

108 Oct 05, 2022
Komposition - The video editor built for screencasters

Komposition The video editor built for screencasters Tutorial Video | Introduction | Installation Documentation See the documentation and user guide.

Oskar Wickström 428 Jan 08, 2023
A simple Python Youtube Wachtime for YTbebot

Simple bot that was development in python 3.7, that automatically watch youtube videos. It can be used to give more views in your channel helping in the spread and increase the followers because your

Rian eka wiratma 1 Dec 05, 2021
Add a "flame" effect on each hand's index onto a video stream.

Add a "flame" effect on each hand's index onto a video stream. recording.webm.mov This script is just a quick hack, it's a bit of glue between mediapi

Paul Willot 7 Sep 15, 2022
A youtube video link or id to video thumbnail python package.

Youtube-Video-Thumbnail A youtube video link or id to video thumbnail python package. Made with Python3

Fayas Noushad 10 Oct 21, 2022
Wonkey - an open source programming language for the creation of cross-platform video games

Wonkey Programming Language Wonkey is an open source programming language for the creation of cross-platform video games, highly inspired by the “Blit

Wonkey Coders 110 Nov 09, 2022