Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
A Multipurpose bot with many Commands made using Pycord

This repo has all of the commands you will ever need in a discord bot. a Multipurpose discord bot

Pogrammar 42 Dec 18, 2022
Colors in Terminal - Python Lang

🎨 Colorate - Python 🎨 About Colorate is an Open Source project that makes it easy to use Python color coding in your projects. After downloading the

0110 Henrique 1 Dec 01, 2021
Official AIdea command line tool

AIdea CLI Official AIdea command line tool for https://aidea-web.tw. Installation Make sure you have installed both Python 3 and pip package manager.

AIdea 5 Dec 15, 2021
Spongebob-cli - Watch classic spongebob from the terminal

spongebob-cli Watch classic spongebob from the terminal! Thanks to everyone that

523 Jan 02, 2023
Easily handle day to day CLI operation via Python instead of regular Bash programs.

pz Ever wished to use Python in Bash? Would you choose the Python syntax over sed, awk, ...? Should you exactly know what command would you use in Pyt

CZ.NIC 697 Jan 03, 2023
A python CLI app that converts a mp4 file into a gif with ASCII effect added.

Video2ASCIIgif This CLI app takes in a mp4 format video, converts it to a gif with ASCII effect applied. This also includes full control over: backgro

Sriram R 6 Dec 31, 2021
A simple python implementation of a reverse shell

llehs A python implementation of a reverse shell. Note for contributors The project is open for contributions and is hacktoberfest registered! llehs u

Archisman Ghosh 2 Jul 05, 2022
Tiny command-line utility for mapping broken keys to other positions.

brokenkey Tiny command-line utility for mapping broken keys to other positions. Installation Clone this repository using git: git clone https://github

0 Oct 04, 2021
Jupyter notebook client in neovim

🪐 Jupyter-Nvim Read jupyter notebooks in neovim Note: The plugin is still in alpha stage 👾 Usage Just open any *.ipynb file and voila! ✨ Contributin

Ahmed Khalf 85 Dec 29, 2022
Shellmon is a tool used to create and control a webshell remotely, created using the Python3

An Simple PHP Webshell Manager Description Shellmon is a tool used to create and control a webshell remotely, created using the Python3 programming la

22XploiterCrew 12 Dec 30, 2022
Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network)

Phil Wang 4.4k Jan 09, 2023
liquidctl – liquid cooler control Cross-platform tool and drivers for liquid coolers and other devices

Cross-platform CLI and Python drivers for AIO liquid coolers and other devices

1.7k Jan 08, 2023
A communist shell written in Python

kash A communist shell written in Python It doesn't support escapes, quotes, comment lines, |, &&, , or similar yet. If you need help, get it from

Çınar Yılmaz 1 Dec 10, 2021
Create animated ASCII-art for the command line almost instantly!

clippy Create and play colored 🟥 🟩 🟦 or colorless ⬛️ ⬜️ animated, or static, ASCII-art in the command line! clippy can help if you are wanting to;

Connor 10 Jun 26, 2022
jrnl is a simple journal application for the command line.

jrnl To get help, submit an issue on Github. jrnl is a simple journal application for the command line. You can use it to easily create, search, and v

jrnl 5.7k Dec 31, 2022
Play videos in the terminal.

Termvideo Play videos in the terminal (stdout). python main.py /path/to/video.mp4 Terminal size: -x output_width, -y output_height. Default autodetect

Patrick 11 Jun 13, 2022
🌌 A Python script to generate blog banners from command line.

Auto Blog Banner Generator A Python script to generate blog banners. This script is used at RavSam. The following image is an example of the blog bann

RavSam 10 Sep 20, 2022
An interactive cheatsheet tool for the command-line

navi An interactive cheatsheet tool for the command-line. navi allows you to browse through cheatsheets (that you may write yourself or download from

Denis Isidoro 12.2k Dec 31, 2022
Conditional Transformer Language Model for Controllable Generation

CTRL - A Conditional Transformer Language Model for Controllable Generation Authors: Nitish Shirish Keskar, Bryan McCann, Lav Varshney, Caiming Xiong,

Salesforce 1.7k Jan 05, 2023
A tool to automatically convert old string literal formatting to f-strings

flynt - string formatting converter flynt is a command line tool to automatically convert a project's Python code from old "%-formatted" and .format(.

Elijah K 551 Jan 06, 2023