A Celery application to collect data, download media and extract information from social media APIs

Overview

Project IBEX

A Celery application to collect data, download media and extract information from social media APIs.

Requirements

You must have a Redis DB instance running (which is currently used as the Celery broker).

Also, another DB will probably be used as the Celery backend in the future (if we ever need backend, that is).

You must also have a MongoDB instance running as the application's write layer.

Usage

  1. Run celery:
celery -A app.core.celery.worker worker -l info

Note, that Celery does not support Windows, so you should probably include

--pool=solo

to avoid any unnecessary errors (for testing only).

  1. Run python server:
python server.py

And make sure to include CROWDTANGLE_TOKEN and YOUTUBE_TOKEN as environment variables and set credentials into app/core/datasources/twitter/.twitter_keys.yaml file for the respective collectors to work properly.

Owner
ibex
ibex
Downloads files and folders

PyDownloader Downloads files and folders at high speed (based on your interent speed). This is very useful to transfer big files from one computer to

ArmenG 4 Feb 24, 2022
A股tick下载,自动判断交易日历,获取全市场level1数据

TickDown A股tick下载,自动判断交易日历,获取全市场level1数据 依赖项 func_timeout requests some_tool(仓库里) akshare 使用 定时任务在上午 09:07开始运行 参数调节 max_num 单批次提交的股票数,当前为800,可以自行尝试多个数

Demon Finch 7 Jul 06, 2022
📼Command line tool based on youtube-dl to easily download selected channels from your subscriptions.

youtube-cdl Command line tool based on youtube-dl to easily download selected channels from your subscriptions. This tool is very handy if you want to

Anatoly 64 Dec 25, 2022
You Can download any video/image in all social medias very easy and High Speed.

All-Downloader You Can download any video/image in all social medias very easy and High Speed. also you can easily download videos from web browsers s

Razor Kenway 14 Dec 16, 2022
Youtube Video Downloader Using Python Gui Appliction with progress Bar

Youtube-Video-Downloader Youtube Video Downloader Using Python Gui Appliction with progress Bar Module Used Pytube Tkinter Pil Urllib Bytes Io LICENSE

Community Programmer 6 Dec 19, 2022
This Program helps you download songs from the Spotify track's link you give in.

Spotify-Downloader-GUI This Program helps you download songs from the Spotify track's link you give in. It uses yt-dlp to download songs from Youtube.

Harish 12 Jun 14, 2022
Command-line program to download videos from YouTube.com and other video sites

youtube-dl - download videos from youtube.com or other video platforms

youtube-dl 116.4k Jan 07, 2023
Yahoo! Finance next gen python 3 / pandas market data downloader

Yahoo! Finance-ng python3 / pandas market data downloader Ever since Yahoo! finance decommissioned their historical data API, many programs that relie

Pedro Larroy 7 Dec 09, 2022
Parallels Desktop dmg downloader

parallelsdesktop-dl Parallels Desktop dmg file downloader Usage usage: pd-dl [-h] [--dlv [DLV]] [-v] Parallels Desktop downloader optional arguments

2 Sep 13, 2022
YoutubeDownloader - Download any public Playlist from Youtube

YoutubeDownloader Download any public Youtube Channel / Playlist Features Bulk d

17 Nov 12, 2022
Pytube ve tkinter kütüphanesi ile yapmış olduğum basit ve temel bir youtube video indirme programı.

PyTube Pytube ve tkinter kütüphanesi ile yapmış olduğum basit ve temel bir youtube video indirme programı. Videolar 720p çözünürlükte indirilmektedir.

1 Nov 12, 2021
Animoo - Python scraper made with BeautifulSoup4 that scrapes images from /c/.

Animoo - Python scraper made with BeautifulSoup4 that scrapes images from /c/. Features Scrapes 10 pages Scrapes each thread Downloads all the images

aether 1 Dec 29, 2021
A simple Python program which uses youtube-dl for downloading YouTube videos as mp3 files.

yt-mp3 converter This is a simple Python program which uses youtube-dl for downloading YouTube videos as mp3 files. This program is for you if you are

nostalgicnerdpenguin 1 Oct 24, 2021
Downloads separate (specified) file to a randomly generated folder in /TEMP then executes it.

PyTemp-1 A Python3 file downloader. What you do with this code / project / idea is non of my buisness or concern, and this was made for **educational*

NightTab 1 Aug 03, 2022
A Python script to download PDB files associated with a Portable Executable (PE)

A Python script to download PDB files associated with a Portable Executable (PE)

Podalirius 33 Jan 03, 2023
Download all your URI Online Judge source codes and upload to GitHub with simple steps.

URI-Code-Downloader Download all your URI Online Judge source codes and upload to GitHub with simple steps. Prerequisites Python 3.x Installing Downlo

Luan Simões 9 Mar 23, 2022
MMDL (Mega Music Downloader) - A tool to easily download music.

mmdl - Mega Music Downloader What is mmdl ❓ TLDR: MMDL is a cli app which allows you to quickly and efficiently download one or multiple songs from Yo

techboy-coder 30 Dec 13, 2022
A downloader for Cave Story written in Python

Cave Story Downloader This is a downloader for Cave Story written in Python. Thi

Imsad2 2 Feb 16, 2022
A web app for downloading Facebook comments as a csv file

Facebook Comment Downloader A small web app for downloading comments from a public facebook page post. Comment downloading from https://github.com/min

WSDOT 23 Jan 04, 2023
Tool To Get Downloads up to 4k from Paramount+

Paramount 4K Downloader Tool To Get Downloads up to 4k from Paramount+ 😄 Hello Fellow Developers/ ! Hi! My name is WVDUMP. I am Leaking the script

2 Dec 25, 2021