Automatically block traffic on Cloudflare's side based on Nginx Log parsing.

Overview

AutoRL

This is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.

It will evaluate Nginx access.log and find potential CC pattern, and block them on Cloudflare's side and send a message to Telegram Group.

Topology

With Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: 使用 Cloudflare Argo Tunnel(cloudflared) 来加速和保护你的网站), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.

Prerequisites

Since this is only a PoC, the following condition must be met to use AutoRL.

  • Python 3 installed on Host

  • Nginx used for Reverse proxy and all the logs are logged into one access.log file.

  • Nginx has the following log format (in /etc/nginx/nginx.conf)

    log_format  main  '$remote_addr $time_iso8601 "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
    

    On this condition, the raw log should look like this:

    172.70.211.101 2022-05-02T10:44:16+08:00 "GET /grafana/api/live/ws HTTP/1.1" 400 12 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5028.0 Safari/537.36" "145.xx.xx.xxx"
    

    Where, 172.70.211.101 is Cloudflare's IP, 2022-05-02T10:44:16+08:00 stands for request datetime and "145.xx.xx.xxx" is the real visitor IP.

Usage

  1. Download the autorl.py to your host

  2. Edit the following variable in the autorl.py

    • CF_EMAIL (Your Cloudflare login email)
    • CF_AUTH_KEY (Your Cloudflare Global API Key)
    • ACCESS_LOG_PATH (Default is /var/log/nginx/access.log)
    • INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)
    • RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and INTERVAL_MIN is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)
    • TG_CHAT_ID (Your Telegram Chat Group ID)
    • TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here)
    • IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)
  3. Create a crontab for this script, example:

    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py & sleep 10; done
    

Demo

On Telegram side:

On Cloudflare side:

Notes

  • Blocked IP address will never gets unblocked.
  • If logrotate is not setup correctly, then parsing the whole access.log might consume a lot of system resources.
  • The attack pattern/sample is not stored so we have no idea how the attack is conducted.
Owner
Nova Kwok
43EC 6073 0BFF A16C 34BB 9EF2 8D42 A0E6 99E5 0639
Nova Kwok
Ping IP addresses and domains in parallel to find the accessible and inaccessible ones.

🚀 IPpy Parallel testing of IP addresses and domains in python. Reads IP addresses and domains from a CSV file and gives two lists of accessible and i

Shivam Mathur 54 May 21, 2022
A fire and forget command-line tool to allow for easy transitions of VPN connections between a pool of AWS machines.

VPN Swapper A fire and forget command-line tool to allow for easy transitions of VPN connections between a pool of AWS machines. Dependencies poetry -

Workday 5 Jul 07, 2022
With the use of this tool, you can change your MAC address

Akshat0404/MAC_CHANGER This tool has to be used on linux kernel. Now o

1 Jan 25, 2022
A Python module that allows you to create and use simple sockets.

EasySockets A Python module that allows you to create and use simple sockets. Installation The easysockets module can be installed using pip. pip inst

Matthias Wijnsma 2 Jan 16, 2022
🌐 Tools for Networking

🌐 Network Tools Tools for Networking This repository contains the tools needed to make networking easier. Make sure to download all of the requiremen

Tornaido 1 Jan 15, 2022
syncio: asyncio, without await

syncio: asyncio, without await asyncio can look very intimidating to newcomers, because of the async/await syntax. Even experienced programmers can ge

David Brochart 10 Nov 21, 2022
Godzilla traffic decoder Godzilla Decoder 是一个用于 哥斯拉Godzilla 加密流量分析的辅助脚本。

Godzilla Decoder 简介 Godzilla Decoder 是一个用于 哥斯拉Godzilla 加密流量分析的辅助脚本。 Godzilla Decoder 基于 mitmproxy,是mitmproxy的addon脚本。 目前支持 哥斯拉3.0.3 PhpDynamicPayload的

He Ruiliang 40 Dec 25, 2022
Terminal based chat - networking project with sockets in python

Terminal based chat - networking project with sockets in python

2 Jan 24, 2022
This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

Njomza Rexhepi 0 May 29, 2022
Network-Shredder is a python based NIDS.

Network-Shredder is a python based NIDS.

Oussama RAHALI 9 Dec 13, 2022
Best discord webhook spammer using proxy (support all proxy type)

Best discord webhook spammer using proxy (support all proxy type)

Iтѕ_Ѵιcнч#1337 25 Nov 01, 2022
Python module to interface with Tuya WiFi smart devices

TinyTuya Python module to interface with Tuya WiFi smart devices Description This python module controls and monitors Tuya compatible WiFi Smart Devic

Jason Cox 365 Dec 26, 2022
Tool written on Python that locate all up host on your subnet

HOSTSCAN Easy to use command line network host scanner. From noob to noobs. Dependencies Nmap 7.92 or superior Python 3.9 or superior All requirements

NexCreep 4 Feb 27, 2022
test whether http(s) proxies actually hide your ip

Proxy anonymity I made this for other projects, to find working proxies. If it gets enough support and if i have time i might make it into a gui Repos

gxzs1337 1 Nov 09, 2021
Start a simple TCP Listener on a specified IP Address and Port Number and receive incoming connections.

About Start a simple TCP Listener on a specified IP Address and Port Number and receive incoming connections. Download Clone using git in terminal(git

AgentGeneric 5 Feb 24, 2022
Truetool - A TrueCharts automatic and bulk update utility

truetool A easy tool for frequently used TrueNAS SCALE CLI utilities. Previously

TrueCharts 125 Jan 04, 2023
PoC code for stealing the WiFi password of a network with a Lovebox IOT device connected

LoveBoxer PoC code for stealing the WiFi password of a network with a Lovebox IOT device connected. This PoC was is what I used in this blogpost Usage

Graham Helton 10 May 24, 2022
A simple tcpdump sidecar injector to demonstrate Kubernetes's Mutating Webhook

k8s-tcpdump-webhook A simple tcpdump sidecar injector to demonstrate Kubernetes's Mutating Webhook Build and Deploy Build docker image; docker build -

Bilal Ünal 2 Sep 01, 2022
Automatic Proxy scraper and Proxy-rotating Nitro Generator.

Automatic Proxy scraper and Proxy-rotating Nitro Generator.

Tawren007 2 Nov 08, 2021
openPortScanner is a port scanner made with Python!

Port Scanner made with python • Installation • Usage • Commands Installation Run this to install: $ git clone https://github.com/Miguel-Galdin0/openPo

Miguel Galdino 7 Jan 09, 2022