ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

Yes, it's true :orange_heart: This repository has 346 stars.

Yes, it's true! Inspired by a similar repository from @RealPeha, but implemented using a webhook on AWS Lambda and API Gateway, so it's serverless! If

512 Jan 01, 2023
Go-cqhttp Plugin for EFB QQ Slave.

efb-qq-plugin-go-cqhttp efb-qq-plugin-go-cqhttp 是 efb-qq-slave 的插件,需要配合 efb-qq-slave 使用,使用前请先阅读 efb-qq-slave 的文档。

XYenon 26 Dec 11, 2022
DeleteAllBot - Telegram bot to delete all messages in a group

Delete All Bot A star ⭐ from you means a lot to me ! Telegram bot to delete all

Stark Bots 15 Dec 26, 2022
Google Drive, OneDrive and Youtube as covert-channels - Control systems remotely by uploading files to Google Drive, OneDrive, Youtube or Telegram

covert-control Control systems remotely by uploading files to Google Drive, OneDrive, Youtube or Telegram using Python to create the files and the lis

Ricardo Ruiz 52 Dec 06, 2022
Discord raiding tool. Made in python 3.9

XSpammer Discord raiding tool with 20 features. YT Showcase Requirements/Installation Python 3.7+ [https://python.org] Run setup.bat to install the es

Tiie 6 Oct 24, 2022
WhatsApp Multi Device Client

WhatsApp Multi Device Client

23 Nov 18, 2022
Simple Discord Nuke Bot.

Discord-Nuke-Bot Simple Discord Nuke Bot. Simple Discord Nuke Bot Python 3.6 - 3.8 Features Delete Channels Ban All Members Delete Roles Create Channe

9X4N 6 Aug 16, 2022
Free and Open Source Machine Translation API. 100% self-hosted, no limits, no ties to proprietary services. Built on top of Argos Translate.

LibreTranslate Try it online! | API Docs Free and Open Source Machine Translation API, entirely self-hosted. Unlike other APIs, it doesn't rely on pro

UAV4GEO 3.5k Jan 03, 2023
Blankly - 🚀 💸 Trade stocks, cryptos, and forex w/ one package. Easily build, backtest, trade, and deploy across exchanges in a few lines of code.

💨 Rapidly build and deploy quantitative models for stocks, crypto, and forex 🚀 View Docs · Our Website · Join Our Newsletter · Getting Started Why B

Blankly Finance 1.4k Jan 03, 2023
Python script for download course from platzi.com

Platzi Downloader Tool Esta es una pequeña herramienta que hace mucho y que te ahorra una gran cantidad de trabajo a la hora de descargar cursos de Pl

Devil64-Dev 21 Sep 22, 2022
A fun hangman style game to guess random movie names with a short summary about the movie.

hang-movie-man Hangman but for movies 😉 This is a fun hangman style game to guess random movie names from the local database and show some summary ab

Ankit Josh 10 Sep 07, 2022
Telegram Remote Administration Tool

Telegram Remote Administration Tool DISCLAIMER | Telegram Remote Administration Tool can only be used at your PC. Do not be evil! Читайте на Русском |

13 Nov 12, 2022
A Python wrapper around the Twitter API.

Python Twitter A Python wrapper around the Twitter API. By the Python-Twitter Developers Introduction This library provides a pure Python interface fo

Mike Taylor 3.4k Jan 01, 2023
Status-embed - Cool open source profile embed for Discord

Current Status : Incomplete Status Embed Status Embed is an awesome open source

Ritabrata Das 2 Feb 17, 2022
企业微信消息推送的python封装接口,让你轻松用python实现对企业微信的消息推送

👋 corpwechat-bot是一个python封装的企业机器人&应用消息推送库,通过企业微信提供的api实现。 利用本库,你可以轻松地实现从服务器端发送一条文本、图片、视频、markdown等等消息到你的微信手机端,而不依赖于其他的第三方应用,如ServerChan。 如果喜欢该项目,记得给个

Chaopeng 161 Jan 06, 2023
A Django-style ORM idea for manipulating Google Datastore entities

No SeiQueLa ORM EM DESENVOLVIMENTO Uma ideia de ORM no estilo do Django para manipular entidades do Google Datastore. Montando seu modelo: from noseiq

Geraldo Castro 16 Nov 01, 2022
NMux is the version of the NMscript in termux

NMscript-termux-version Termux-Version NMux is the termux version of NMscript which is NMscript? NMscript is a simple script written in Python that he

cabeson sin z 5 Apr 23, 2022
Wrapper for shh/rsync for use with OpenFOAM and blue bear

bbsync wrapper for shh/rsync for use with OpenFOAM and blue bear About The Project bbsync is a wrapper for shh/rsync for use with OpenFOAM and blue be

1 Dec 10, 2021
A Python AWS Lambda Webhook listener that generates a permanent URL when an asset is created in Contentstack.

Webhook Listener A Python Lambda Webhook Listener - Generates a permanent URL on created assets. See doc on Generating a Permanent URL: https://www.co

Contentstack Solutions 1 Nov 04, 2021
ВКонтакте бот для управления Sugar кошельком

Sugarchain VK ВКонтакте бот для управления Sugar кошельком Установка Установить зависимости можно командой: pip install -r requirements.txt Запуск (из

Vladimir 4 Jun 06, 2021