Automatically block traffic on Cloudflare's side based on Nginx Log parsing.

Overview

AutoRL

This is a PoC of automatically block traffic on Cloudflare's side based on Nginx Log parsing.

It will evaluate Nginx access.log and find potential CC pattern, and block them on Cloudflare's side and send a message to Telegram Group.

Topology

With Cloudflare Argo Tunnel, we can set security group to allow inbound traffic for SSH only, this can guarantee the Host's IP will not be exposed to the Internet (ref: 使用 Cloudflare Argo Tunnel(cloudflared) 来加速和保护你的网站), however, attackers can still CC your website by sending enormous requests cocurrently, AutoRL is here trying to mitigate this problem.

Prerequisites

Since this is only a PoC, the following condition must be met to use AutoRL.

  • Python 3 installed on Host

  • Nginx used for Reverse proxy and all the logs are logged into one access.log file.

  • Nginx has the following log format (in /etc/nginx/nginx.conf)

    log_format  main  '$remote_addr $time_iso8601 "$request" '
                        '$status $body_bytes_sent "$http_referer" '
                        '"$http_user_agent" "$http_x_forwarded_for"';
    

    On this condition, the raw log should look like this:

    172.70.211.101 2022-05-02T10:44:16+08:00 "GET /grafana/api/live/ws HTTP/1.1" 400 12 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.5028.0 Safari/537.36" "145.xx.xx.xxx"
    

    Where, 172.70.211.101 is Cloudflare's IP, 2022-05-02T10:44:16+08:00 stands for request datetime and "145.xx.xx.xxx" is the real visitor IP.

Usage

  1. Download the autorl.py to your host

  2. Edit the following variable in the autorl.py

    • CF_EMAIL (Your Cloudflare login email)
    • CF_AUTH_KEY (Your Cloudflare Global API Key)
    • ACCESS_LOG_PATH (Default is /var/log/nginx/access.log)
    • INTERVAL_MIN (Default is 1, then this script will evaluate for 1min's traffic)
    • RATE_PER_MINUTE (How many requests are allowed for single IP, e,g, when this is set to 600 and INTERVAL_MIN is 1, then one IP can send at most 600 requests, after that, this IP will be blocked.)
    • TG_CHAT_ID (Your Telegram Chat Group ID)
    • TG_BOT_TOKEN (You should invite a bot to your group, and fillin the bot token here)
    • IP_WHITE_LIST (If you'd like to whitelist some IP, fillin here)
  3. Create a crontab for this script, example:

    * * * * * for i in {1..6}; do /usr/bin/python3 /path/to/autorl.py & sleep 10; done
    

Demo

On Telegram side:

On Cloudflare side:

Notes

  • Blocked IP address will never gets unblocked.
  • If logrotate is not setup correctly, then parsing the whole access.log might consume a lot of system resources.
  • The attack pattern/sample is not stored so we have no idea how the attack is conducted.
Owner
Nova Kwok
43EC 6073 0BFF A16C 34BB 9EF2 8D42 A0E6 99E5 0639
Nova Kwok
Takes a file of hosts or domains and outputs the IP address of each host/domain in the file.

Takes a file of hosts or domains and outputs the IP address of each host/domain in the file. Installation $ git clone https://github.com/whoamisec75/i

whoami security 2 May 10, 2022
Rufus is a Dos tool written in Python3.

🦎 Rufus 🦎 Rufus is a simple but powerful Denial of Service tool written in Python3. The type of the Dos attack is TCP Flood, the power of the attack

Billy 88 Dec 20, 2022
pyWhisker is a Python equivalent of the original Whisker made by Elad Shamir and written in C#.

PyWhisker pyWhisker is a Python equivalent of the original Whisker made by Elad Shamir and written in C#. This tool allows users to manipulate the msD

Shutdown 325 Jan 08, 2023
批量检查目标是否为cdn

🐸 Frog For Automatic Scan 🐶 Doge For Defense Evasion&Offensive Security Frog-checkCDN 批量检查目标是否为cdn Usage: python3 checkCDN.py list.txt list内可以为ip或者d

TimWhite 119 Dec 27, 2022
FramIp - it a framework for work at IP and domain

FramIp FramIp - it a framework for work with IP and domain Installation (termux) $ pkg install git && pkg install python && git clone https://github.c

1 Jan 28, 2022
A Python server and client app that tracks player session times and server status

MC Outpost A Python server and client application that tracks player session times and server status About MC Outpost provides a session graph and ser

Grant Scrits 0 Jul 23, 2021
Monitoring plugin to check network interfaces with Icinga, Nagios and other compatible monitoring solutions

check_network_interface - Monitor network interfaces This is a monitoring plugin for Icinga, Nagios and other compatible monitoring solutions to check

DinoTools 3 Nov 15, 2022
Raspberry Pi Based Serial Console Server, with PushBullet Notification of IP changes, Automatic VPN termination, custom menu, Power Outlet Control, and a lot more

ConsolePi Acts as a serial Console Server, allowing you to remotely connect to ConsolePi via Telnet/SSH/bluetooth to gain Console Access to devices co

120 Jan 05, 2023
Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7.

Medusa Medusa is a cross-platform agent compatible with both Python 3.8 and Python 2.7. Installation To install Medusa, you'll need Mythic installed o

Mythic Agents 123 Nov 09, 2022
A powerful framework for decentralized federated learning with user-defined communication topology

Scatterbrained Decentralized Federated Learning Scatterbrained makes it easy to build federated learning systems. In addition to traditional federated

Johns Hopkins Applied Physics Laboratory 7 Sep 26, 2022
A live streaming chatroom involving multiple modalities, such as voice, gesture, and facial expression

HiLive A live streaming chatroom involving multiple modalities, such as voice, gesture, and facial expression. Introduction We focus on demonstrating

Ryan Yen 2 Dec 02, 2021
This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

This application aims to read all wifi passwords and visualizes the complexity in graph formation by taking into account several criteria and help you generate new random passwords.

Njomza Rexhepi 0 May 29, 2022
A python socket.io client for Roboteur

Roboteur Client Example TODO Basic setup Install the requirements: $ pip install -r requirements.txt Run the application: $ python -m roboteur_client

Barry Buck 1 Oct 13, 2021
A protocol or procedure that connects an ever-changing IP address to a fixed physical machine address

p0znMITM ARP Poisoning Tool What is ARP? Address Resolution Protocol (ARP) is a protocol or procedure that connects an ever-changing IP address to a f

Furkan OZKAN 9 Sep 18, 2022
Nexum is an open-source, remote administration tool written in Python 3

A full-featured remote administration tool written in Python 3. The goal of this project is to make the use of a remote administration tool as simple

z3phyrus 2 Nov 26, 2021
Tool that creates a complete copy of your server

Discord-Server-Cloner Tool that creates a complete copy of your server Setup: Open run.bat If the file closes, open cmd And write: pip install -r requ

DEEM 3 Dec 13, 2021
Nyx-Net: Network Fuzzing with Incremental Snapshots

Nyx-Net: Network Fuzzing with Incremental Snapshots Nyx-Net is fast full-VM snapshot fuzzer for complex network based targets. It's built upon kAFL, R

Chair for Sys­tems Se­cu­ri­ty 146 Dec 16, 2022
An curated collection of awesome resources about networking in cybersecurity

An ongoing curated collection of awesome software, libraries, frameworks, talks & videos, best practices, learning tutorials and important practical resources about networking in cybersecurity

Paul Veillard, P. Eng 7 Nov 30, 2022
IoT owl is light face detection and recognition system made for small IoT devices like raspberry pi.

IoT Owl IoT owl is light face detection and recognition system made for small IoT devices like raspberry pi. Versions Heavy with mask detection withou

Ret2Me 6 Jun 06, 2022
Building a Robust IOT device which is customizable, encrypted, secure and user friendly

Building a Robust IOT device which is customizable, encrypted, secure and user friendly, which uses a single GPIO pin to extract multiple sensor values

1 Jan 03, 2022