Command line parser for common log format (Nginx default).

Related tags

Command-line Toolspythonnginxcommonlogging
Overview

Logparser

Command line parser for common log format (Nginx default).

Usage

It counts most important data: referrers, operating systems, browsers and daily unique visitors (IPs).

# Console output
python parse.py sitename.log.gz

# HTML output
python parse.py sitename.log.gz --html ~/sitename/logs.html

Install and update PIP packages.

pip install -U -r requirements.txt

Speed

  • logparser 24,249/s
  • GoAccess 6,234/s

Outputs

  • HTML output is based on Jinja2 templates. It can be improved as you see fit.
  • Console output for Subreply for a fews days in Sep, 2021:
----- Days
 1402 2021-09-14 Tuesday
  893 2021-09-15 Wednesday
  797 2021-09-16 Thursday
  857 2021-09-17 Friday
  951 2021-09-18 Saturday
  945 2021-09-19 Sunday
  869 2021-09-20 Monday

----- Browsers
    2 Chrome Mobile iOS
    2 Bytespider
    2 HeadlessChrome
    3 Links
    4 Firefox iOS
    4 UC Browser
    6 IE
    7 Go-http-client
    7 Opera
    8 Other
   20 Chrome Mobile WebView
   20 Edge
   28 Firefox Mobile
   30 Safari
   42 Chrome Mobile
   56 Mobile Safari
  111 Firefox
 3468 Chrome

----- Operating Systems
   15 Other
   22 Ubuntu
   39 Linux
   63 iOS
   91 Android
  101 Mac OS X
 3474 Windows

----- Referrers
    2 twtxt.net
    2 l.instagram.com
    2 lobste.rs/s/bffayk/what_are_you_doing_this_weekend
    3 ro.linkedin.com
    3 199.247.2.88:80
    3 t.co/sv4adihlka
    3 lobste.rs/s/fe2eph/what_are_you_doing_this_weekend
    4 news.ycombinator.com
    4 1mb.club
    4 old.reddit.com/r/internetisbeautiful
    4 baidu.com
    4 t.co
    5 out.reddit.com/t3_pl9hiu
    5 sjmulder.nl/en/textonly.html
    8 nt
   10 199.247.2.88
   11 lobste.rs
   13 sjmulder.nl
   15 google.com
   16 lucianmarin.com
   49 reddit.com
   85 lobste.rs/s/8fzzmz/what_are_you_doing_this_week
Owner
Lucian Marin
Python developer and all things design.
Lucian Marin
GitHub Repository
CLI tool to show the current crypto balance

CryptoBoard The simple python CLI tool for one currency to show the current crypto balance yours purchases. That's all. Data source is from https://ww

John 2 Nov 18, 2021
A tool to automatically convert old string literal formatting to f-strings

flynt - string formatting converter flynt is a command line tool to automatically convert a project's Python code from old "%-formatted" and .format(.

Elijah K 551 Jan 06, 2023
Execute shell command lines in parallel on Slurm, S(on) of Grid Engine (SGE), PBS/Torque clusters

qbatch Execute shell command lines in parallel on Slurm, S(on) of Grid Engine (SGE), PBS/Torque clusters qbatch is a tool for executing commands in pa

Jon Pipitone 26 Dec 12, 2022
Easily handle day to day CLI operation via Python instead of regular Bash programs.

pz Ever wished to use Python in Bash? Would you choose the Python syntax over sed, awk, ...? Should you exactly know what command would you use in Pyt

CZ.NIC 697 Jan 03, 2023
Python3 parser for Apple's crash reports

pyCrashReport in intended for analyzing crash reports from Apple devices into a clearer view, without all the thread listing and loaded images, just the actual data you really need to debug the probl

7 Aug 19, 2022
Centauro - a command line tool with some network management functionality

Centauro Ferramenta de rede O Centauro é uma ferramenta de linha de comando com

1 Jan 01, 2022
PathPicker accepts a wide range of input -- output from git commands, grep results, searches -- pretty much anything.After parsing the input, PathPicker presents you with a nice UI to select which files you're interested in. After that you can open them in your favorite editor or execute arbitrary commands.

PathPicker Facebook PathPicker is a simple command line tool that solves the perpetual problem of selecting files out of bash output. PathPicker will:

Facebook 4.8k Dec 30, 2022
Notion-cli-list-manager - A simple command-line tool for managing Notion databases

A simple command-line tool for managing Notion List databases. ✨

Giacomo Salici 75 Dec 04, 2022
Simple Tool To Grab Like-Card Coupon

Simple Tool To Grab Like-Card Coupon

Soud 10 Jan 30, 2022
🕰 The command line tool for scheduling Python scripts

hickory is a simple command line tool for scheduling Python scripts.

Max Humber 146 Dec 07, 2022
A simple command for converting and processing data from your clipboard.

A generic text conversion/processing tool

3DigitDev 29 Jan 17, 2022
pls is a better ls for developers, pronounced /pliːz/ as in 'please'

pls is a better ls for developers. The "p" stands for ("pro" as in "professional"/"programmer") or "prettier". It works in a manner similar to ls, in

Dhruv Bhanushali 572 Dec 28, 2022
Skiller - With this payload you can control the target computer with (cmd)

Skiller - With this payload you can control the target computer with (cmd)

1 Jan 02, 2022
Command-line program for organizing and managing ebook collections

Command-line program for organizing and managing ebook collections. It is a Python port from the original shell scripts ebook-tools

Raul 14 Nov 12, 2022
🗃️ Fileio-cli wrapper for fileioapi.py with fire.py, inspiration DOS

🗃️ File.io File.io simply upload a file, share the link, and after it is downloaded, the file is completely deleted. An API wrapper for the file.io w

nkot56297 2 May 12, 2022
A CLI framework based on asyncio

asynccli A CLI framework based on asyncio. Note This is still in active development. Things will change. For now, the basic framework is operational.

Adam Hopkins 6 Nov 13, 2022
commandline version of wordle game and my auto solver.

Wordle Machine (and Wordle Game) (in commandline) My implementation of the Wordle game (inspired by https://www.powerlanguage.co.uk/wordle/) and my in

Kevin Xu 11 Jan 03, 2023
GoogleFormSpammer - A simple CLI script to spam Google Forms used by Crypto Wallet scammers to collect stolen data

GoogleFormSpammer - A simple CLI script to spam Google Forms used by Crypto Wallet scammers to collect stolen data

14 Dec 17, 2022
Management commands to help backup and restore your project database and media files

Django Database Backup This Django application provides management commands to help backup and restore your project database and media files with vari

687 Jan 04, 2023
Tstock - Check stocks from the terminal

tstock - Check stocks from the terminal! 📈 tstock is a tool to easily generate stock charts from the command line. Just type tstock aapl to get a 3 m

Gabe Banks 502 Dec 30, 2022