ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete

Overview

ShadowClone

ShadowClone allows you to distribute your long running tasks dynamically across thousands of serverless functions and gives you the results within seconds where it would have taken hours to complete.

You can make full use of the Free Tiers provided by cloud providers and supercharge your mundane cli tools with shadow clone jutsu (Naruto style)!

Installation

Please visit the wiki for installation and intial configuration instructions

Usage

⚡ python shadowclone.py -h
usage: shadowclone.py [-h] -i INPUT [-s SPLITNUM] [-o OUTPUT] -c COMMAND

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
  -s SPLITNUM, --split SPLITNUM
                        Number of lines per chunk of file
  -o OUTPUT, --output OUTPUT
  -c COMMAND, --command COMMAND
                        command to execute

How it works

We create a container image during the initial setup and register it as a runtime for our function in AWS/GCP/Azure whatever. When you execute ShadowClone on your computer, instances of that container are activated automatically and are only active for the duration of its execution. How many instances to activate is dynamically decided at runtime depending on the size of the input file provided and the split factor. The input is then split into chunks and equally distributed between all the instances to execute in parallel. For example, if your input file has 10,000 lines and you set the split factor to 100 lines, then it will be split into 100 chunks of 100 lines each and 100 instances will be run in parallel!

Features

  • Extremely fast
  • No need to maintain a VPS (or a fleet of it :))
  • Costs almost nothing per month
    • Compatible with free tiers of most cloud services
  • Cloud agnostic
    • Same script works with AWS, GCP, Azure etc.
  • Supports upto 1000 parallel invocations
  • Dynamically decide the number of invocations
  • Run any tool in parallel on the cloud
  • Pipe output to other tools

Comparison

This tool was inspired by the awesome Axiom and Fleex projects and goes beyond the concept of VPS for running the tools by using serverless functions and containers.

Features Axiom/Fleex ShadowClone
Instances 10-100s* 1000s
Cost Per instance/per minute Mostly Free**
Startup Time 4-5 minutes 2-3 seconds
Idle Cost $++ Free
On Demand Scalability No

*Most cloud providers do not allow spinning up too many instances by default, so you are limited to around 10-15 instances at max. You have to make a request to the support to increase this number.

** AWS & Azure allow 1 million invocations per month for free. Google allows 2 million invocations per month for free. You will be charged only if you go above these limits

Demo

DNS Bruteforcing using a 43mb file - 34 seconds

asciicast

Running httpx on 94K subdomains in 1 min

asciicast

References

Lithops documentation

Free Tiers

Cloud Provider Free Allowance Link
Google Functions 2 Million invocations, 400,000 GB-seconds per month Google Cloud Free Program
AWS Lambda 1 Million invocations, Up to 3.2 million seconds of compute time per month Free Cloud Computing Services - AWS Free Tier
Azure Functions 1 Million invocations Microsoft Azure Free Services

Obviously, you can make any number of function invocations per month. The table above only shows how many invocations are free.

Similar Tools

Disclaimer

This tool is designed as a proof-of-concept implementation and it's usage is intended for educational purpose only. Usage for attacking targets without prior mutual consent is illegal. It's the end user's responsibility to obey all applicable local, state and federal laws. Developers assume no liability and are not responsible for any misuse or damage caused by this program.

Ross Virtual Assistant is a programme which can play Music, search Wikipedia, open Websites and much more.

Ross-Virtual-Assistant Ross Virtual Assistant is a programme which can play Music, search Wikipedia, open Websites and much more. Installation Downloa

Jehan Patel 4 Nov 08, 2021
SMAM2 is a package manager built specifically for SourceMod.

SourceMod Addon Manager 2 (SMAM2) SMAM2 is a package manager built specifically for SourceMod. This was heavily inspired by Phil25's SMAM. I thought t

John Mascagni 6 Sep 16, 2022
VC-Music , Playing music without bot.

VC-Userbot A Telegram Userbot to play or streaming Audio and Video songs / files in Telegram Voice Chats. It's made with PyTgCalls and Pyrogram Requir

RioProjectX 8 Aug 04, 2022
Catinthebox - Awesome bot for Mastodon

Cat In The Box :3 Description Awesome bot for Mastodon Requirements python pip g

satanist 0 Jan 19, 2022
The Most advanced and User-Friendly Google Collab NoteBook to download Torrent directly to Google Drive with File or Magnet Link support and with added protection of Timeout Preventer.

Torrent To Google Drive (UI Added! 😊 ) A Simple and User-Friendly Google Collab Notebook with UI to download Torrent to Google Drive using (.Torrent)

Dr.Caduceus 33 Aug 16, 2022
Telegram Bot to Filter posts in Bot Inline search

Inline-Filter-Bot A Telegram Bot for filter in Inline Features Unlimited Filters Supports all type of filters Supports Alert Button Using Common Marku

Code X Botz 67 Dec 26, 2022
Checks instagram names to see if they're available

How to install You must have python 3.7.6 installed and make sure you click the 'ADD TO PATH' option when installing Open cmd and type pip install aio

2 Oct 20, 2021
Online Marketplace API

Online Marketplace API Table of Contents Setup Instructions Documentation Setup instructions Make sure you have python installed Clone the repository

Kanat 3 Jul 13, 2022
Telegram Google Translater Bot Can Translate Any Language To Your Selected Language

🔰 TELEGRAM GOOGLE TRANSLATER 🔰 • ⚡ INSTALLING ⚡ • • ✅ OFFICIAL SUPPORTS ✅ •

⚝ANKIT KUMAR⚝ 2 Jan 16, 2022
Send GitHub Issues, PRs or Discussions Updates to Wechat

Send GitHub Issues, PRs or Discussions Updates to Wechat

Hollow Man 2 Jul 12, 2022
Process your transactions from etherscan (and other forks) into excel file for easier manipulation.

DEGEN TRACKER Read first This is my first Python open source project and it is very likely full of bad practices and security issues. You should not u

1 Oct 13, 2022
Maestral is an open-source Dropbox client written in Python.

Maestral - A light-weight and open-source Dropbox client for macOS and Linux

2.6k Jan 03, 2023
A PowerFull Telegram Mirror Bot.......

- [ DEAD REPO AND NO MORE UPDATE ] Slam Mirror Bot Slam Mirror Bot is a multipurpose Telegram Bot written in Python for mirroring files on the Interne

αвιנтн 2 Nov 09, 2021
🤟The VC Music Source code of @DaisyXBot ❤️ v3 Out now

DAISYXMUSIC V3 🎵 A bot that can play music on telegram group's voice call Available on telegram as @DaisyXbot Whats new 🔥 Thumbnail Support Playlist

TeamDaisyX 207 Dec 05, 2022
A way to export your saved reddit posts to a Notion table.

reddit-saved-to-notion A way to export your saved reddit posts and comments to a Notion table.Uses notion-sdk-py and praw for interacting with Notion

19 Sep 12, 2022
Its The Basic Commands Of Termux

Its The Basic Commands Of Termux

ANKIT KUMAR 1 Dec 27, 2021
Discord Bot for League of Legends live match tracker

SABot Dicord Bot for League of Legends match auto tracker Features: Search Summoners statistics in League of Legends. Auto-notifications provide when

Jungyu Choi 4 Sep 27, 2022
Get notifications in your Discord server of any software releases from Apple.

Apple Releases Get notifications in your Discord server of any software releases from Apple. Running To locally host your own instance, create a Disco

adam 17 Oct 22, 2022
Example code for interacting with solana anchor programs - candymachine

candypy example code for interacting with solana anchor programs - candymachine THIS IS PURELY SAMPLE CODE TO FORK, MODIFY, UNDERSTAND AND INTERACT WI

dubbelosix 3 Sep 18, 2022
A media upload to telegraph module

A media upload to telegraph module

Fayas Noushad 5 Dec 01, 2021