Automatically block traffic on Cloudflare's side based on Nginx Log parsing.

Overview

AutoRL

This is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.

It will evaluate Nginx access.log and find potential CC pattern, and block them on Cloudflare's side and send a message to Telegram Group.

Topology

With Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: 使用 Cloudflare Argo Tunnel(cloudflared) 来加速和保护你的网站), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.

Prerequisites

Since this is only a PoC, the following condition must be met to use AutoRL.

  • Python 3 installed on Host

  • Nginx used for Reverse proxy and all the logs are logged into one access.log file.

  • Nginx has the following log format (in /etc/nginx/nginx.conf)

    log_format  main  '$remote_addr $time_iso8601 "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
    

    On this condition, the raw log should look like this:

    172.70.211.101 2022-05-02T10:44:16+08:00 "GET /grafana/api/live/ws HTTP/1.1" 400 12 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5028.0 Safari/537.36" "145.xx.xx.xxx"
    

    Where, 172.70.211.101 is Cloudflare's IP, 2022-05-02T10:44:16+08:00 stands for request datetime and "145.xx.xx.xxx" is the real visitor IP.

Usage

  1. Download the autorl.py to your host

  2. Edit the following variable in the autorl.py

    • CF_EMAIL (Your Cloudflare login email)
    • CF_AUTH_KEY (Your Cloudflare Global API Key)
    • ACCESS_LOG_PATH (Default is /var/log/nginx/access.log)
    • INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)
    • RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and INTERVAL_MIN is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)
    • TG_CHAT_ID (Your Telegram Chat Group ID)
    • TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here)
    • IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)
  3. Create a crontab for this script, example:

    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py & sleep 10; done
    

Demo

On Telegram side:

On Cloudflare side:

Notes

  • Blocked IP address will never gets unblocked.
  • If logrotate is not setup correctly, then parsing the whole access.log might consume a lot of system resources.
  • The attack pattern/sample is not stored so we have no idea how the attack is conducted.
Owner
Nova Kwok
43EC 6073 0BFF A16C 34BB 9EF2 8D42 A0E6 99E5 0639
Nova Kwok
Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing

Using AWS's API Gateway + Lambda + Python to run a simple websocket application. For learning/testing. The AWS Resources seemed overly complex and were missing some critical gotchas in setting up a s

Seth Miller 15 Dec 23, 2022
An open source bias lighting program which syncs up colored lights to the contents of your screen.

About Firelight Firelight is an open source bias lighting program which syncs up colored lights to the contents of your screen or TV, providing an imm

Roshan 18 Dec 18, 2022
ProtOSINT is a Python script that helps you investigate Protonmail accounts and ProtonVPN IP addresses

ProtOSINT ProtOSINT is a Python script that helps you investigate ProtonMail accounts and ProtonVPN IP addresses. Description This tool can help you i

pixelbubble 249 Dec 23, 2022
Get Your Localhost Online - Ngrok Alternative

Get Your Localhost Online - Ngrok Alternative

Azimjon Pulatov 442 Jan 04, 2023
A collection of domains, wildcards and substrings designed for dnscrypt-proxy filter method.

A collection of domains, wildcards and substrings designed for dnscrypt-proxy filter method.

3 Oct 25, 2022
TunnelProxy 是一个本地隧道代理,可以从fofa爬取免费的socks代理,然后构建代理池,如果一个代理失效,会自动切换

TunnelProxy 是一个本地隧道代理,可以从fofa爬取免费的socks代理,然后构建代理池,如果一个代理失效,会自动切换。 应用场景 渗透测试需要访问某些国内网站(比如edu的),想要隐藏自己,但是国外代理不能访问,也没有稳定的可用代理的时候。 之后,可能我会增加国外代理,实现白嫖科学上网。

urdr-gungnir 45 Nov 17, 2022
Roadster - Distance to Closest Road Feature Server

Roadster: Distance to Closest Road Feature Server Milliarium Aerum, the zero of

Textualization Software Ltd. 4 May 23, 2022
Equibles Stocks API for Python

Equibles Stocks API for Python Requirements. Python 2.7 and 3.4+ Installation & Usage pip install If the python package is hosted on Github, you can i

Equibles 3 Apr 15, 2022
IP-Escaner - A Python Tool to obtain information from an IP address

IP-Escaner Herramienta para obtener informacion de una direccion IP Opciones de

4 Apr 09, 2022
Script and library to wait for a DNS authority server to get its configuration.

DNSWait dnswait is a small script to wait for the "propagation" of a namserver configuration. Installing It's as easy as: python -m pip install dnswai

Julien Palard 14 Jan 17, 2022
Lightweight asyncio compatible utilities for consuming broker messages.

A simple asyncio compatible consumer for handling amqp messages.

Mehdi Kamani 3 Apr 10, 2022
wg-exporter is a simple yet effective Prometheus exporter for Wireguard

wg-exporter wg-exporter is a simple yet effective Prometheus exporter for Wireguard. What are the collected metrics ? General: wg_connected_peers: num

men1n2 3 May 20, 2022
This is simple script that changes the config register of a cisco router over serial so that you can reset the password

Cisco-router-config-bypass-tool- This is simple script that changes the config register of a cisco router over serial so that you can bypass the confi

James 1 Jan 02, 2022
A website to list Shadowsocks proxies and check them periodically

Shadowmere An automatically tested list of Shadowsocks proxies. Motivation Collecting proxies around the internet is fun, but what if they stop workin

Jorge Alberto Díaz Orozco (Akiel) 29 Dec 21, 2022
Multi-vendor library to simplify CLI connections to network devices

Netmiko Multi-vendor library to simplify CLI connections to network devices Why Netmiko? Network automation to screen-scraping devices is primarily co

Kirk Byers 3k Jan 01, 2023
Free,Cross-platform,Single-file mass network protocol server simulator

FaPro Free,Cross-platform,Single-file mass network protocol server simulator 中文Readme Description FaPro is a Fake Protocol Server tool, Can easily sta

FOFA Pro 1.4k Jan 06, 2023
A pretty quick and simple interface to paramiko SFTP

A pretty quick and simple interface to paramiko SFTP. Provides multi-threaded routines with progress notifications for reliable, asynchronous transfers. This is a Python3 optimized fork of pysftp wit

14 Dec 21, 2022
This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID )

This Tool can help enginners and biggener in network, the tool help you to find of any ip with subnet mask that can calucate them and show you ( Availble IP's , Subnet Mask, Network-ID, Broadcast-ID

12 Dec 13, 2022
Distribute a portion of your yield to other addresses 💙

YSHARE Distribute a portion of your yield to other addresses. How does it work Desposit your yToken or tokens into this contract Set the benificiaries

11 Nov 24, 2021
🔥 Minimal performant package to asynchronously make GET requests.

Minimal performant package to asynchronously make GET requests without any dependencies other than asyncio.

Yannick Perrenet 1 Jun 01, 2022