Yaml - Loggers are like print() statements

Overview

Upgrade your print statements

print statement meme

Loggers are like print() statements

except they also include loads of other metadata:

  • timestamp
  • msg (same as print!)
  • args (values or variables put into functions)
  • function name
  • level (e.g. DEBUG)
  • line number (e.g line 42)
  • module (which python script it came from)
  • processes (getting fancy)
  • threads (getting very fancy)

I want it all WITHOUT CONFIGURATION!

loguru logo

No problems. Check out the Loguru repo!
You can pretty much run with minimal config and loads of features. Loguru has all the features of this repo and more, along with a badass logo.

Where do I start?

This repo comes bundled with tidy logging configuration files saved in src/logconfig/.
There is also a timing decorator in src/utils/ so you can optimize your code by simply decorating your function with @timing above it. There are many online articles but very few clearly explain how to configure complex loggers with filters and yaml files.
To see loggers in action, setup a virtual environment and then run src/mainmodule.py

Requirements

  • Python 3.6+

(Feel free to submit earlier versions that work)

Quickstart - Git clone and virtual env setup

Quickstart Instructions

Windows using powershell or CMD

cd to clone directory. Create virtual env with pip + venv:

git clone https://github.com/izzley/loggerexamples
cd loggerexamples\
py -0p # Optional: check your version and python path
py -m venv .venv
.venv\Scripts\activate
pip install --upgrade pip
pip install -r requirements.txt

# run main script
.\src\main.py

Linux/Mac

cd /to/clone/location
git clone https://github.com/izzley/loggerexamples
cd loggerexamples/
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt

If your relative imports aren't working, create `.pth' and add the parent folder/s to the file:

$ echo $(pwd) >> .venv/lib/python3.8/site-packages/my_p_ext.pth

Play with the logger

Walkthroughs

@TODO add list of suggestions to inspect

Try changing the main logger level from main_module.py

The main logger is first setup from main_module.py. Notice how the level is set to DEBUG?

logger level change


  • Run main_module.py with this DEBUG setting and take a look at src/logconfig/root.log.
  • Now change the level to WARNING and spot the difference in src/logconfig/root.log. Remember this is the __main__ logger. Notice any changes?

logger level change


  • Notice everything from __main__ is greater or equal to WARNING? The logs running from other_module.py are still showing because we haven't changed its log level. Go ahead and change the loglevel in other_module.py to CRITICAL and spot the difference.

Whats in the YAML file??

YAML parts

In short, the conf.YAML file contains all of the instructions for how the logger should behave. Below breaks down the conf yaml file into its parts:

formatters

formatters:
    standard:
        format: "%(asctime)s %(levelname)s - [%(filename)s: line %(lineno)s] - %(funcName)s - %(message)s"

Take this logger for example:

def funccalc(n):
    logger.debug("something executed")
    for _ in range(n):
        i = 0
    return

The output reflects the yaml file format settings:

2021-11-21 15:43:47,689 DEBUG - [module01.py: line 17] - funccalc - something executed

loggers

@TODO describe root loggers and their inheritance

root:
  level: DEBUG
  handlers: [console, debug_file_handler, info_file_handler, warn_file_handler, error_file_handler, critical_file_handler, root_file_handler]

root logger yaml

handlers

@TODO describe handlers and how they redirect bytes

filters

@TODO describe how filters only allow bytes to handlers if a condition is true. reference filter classes in logconfig.py

References

Owner
isaac peterson
Open source || Awesome sauce
isaac peterson
A cool logging replacement for Python.

Welcome to Logbook Travis AppVeyor Supported Versions Latest Version Test Coverage Logbook is a nice logging replacement. It should be easy to setup,

1.4k Nov 11, 2022
This open-source python3 script is a builder to the very popular token logger that is on my github that many people use.

Discord-Logger-Builder This open-source python3 script is a builder to the very popular token logger that is on my github that many people use. This i

Local 4 Nov 17, 2021
GTK and Python based, system performance and usage monitoring tool

System Monitoring Center GTK3 and Python 3 based, system performance and usage monitoring tool. Features: Detailed system performance and usage usage

Hakan Dündar 649 Jan 03, 2023
Beautifully colored, quick and simple Python logging

Python Quick Logging | QLogging Beautifully colored, quick and simple Python logging. This logger is based on Python logging package Screenshots: Term

45 Sep 25, 2022
A Fast, Extensible Progress Bar for Python and CLI

tqdm tqdm derives from the Arabic word taqaddum (تقدّم) which can mean "progress," and is an abbreviation for "I love you so much" in Spanish (te quie

tqdm developers 23.7k Jan 01, 2023
Outlog it's a library to make logging a simple task

outlog Outlog it's a library to make logging a simple task!. I'm a lazy python user, the times that i do logging on my apps it's hard to do, a lot of

ZSendokame 2 Mar 05, 2022
Small toolkit for python multiprocessing logging to file

Small Toolkit for Python Multiprocessing Logging This is a small toolkit for solving unsafe python mutliprocess logging (file logging and rotation) In

Qishuai 1 Nov 10, 2021
Espion is a mini-keylogger tool that keeps track of all keys a user presses on his/her keyboard

Espion is a mini-keylogger tool that keeps track of all keys a user presses on his/her keyboard. The details get displayed on the terminal window and also stored in a log file.

Anurag.R.Simha 1 Apr 24, 2022
Python logging package for easy reproducible experimenting in research

smilelogging Python logging package for easy reproducible experimenting in research. Why you may need this package This project is meant to provide an

Huan Wang 20 Dec 23, 2022
pyEventLogger - a simple Python Library for making customized Logs of certain events that occur in a program

pyEventLogger is a simple Python Library for making customized Logs of certain events that occur in a program. The logs can be fully customized and can be printed in colored format or can be stored i

Siddhesh Chavan 2 Nov 03, 2022
Vibrating-perimeter - Simple helper mod that logs how fast you are mining together with a simple buttplug.io script to control a vibrator

Vibrating Perimeter This project consists of a small minecraft helper mod that writes too a log file and a script that reads said log. Currently it on

Heart[BOT] 0 Nov 20, 2022
Debugging-friendly exceptions for Python

Better tracebacks This is a more helpful version of Python's built-in exception message: It shows more code context and the current values of nearby v

Clemens Korndörfer 1.2k Dec 28, 2022
Monitoring plugin to check disk io with Icinga, Nagios and other compatible monitoring solutions

check_disk_io - Monitor disk io This is a monitoring plugin for Icinga, Nagios and other compatible monitoring solutions to check the disk io. It uses

DinoTools 3 Nov 15, 2022
A small utility to pretty-print Python tracebacks. ⛺

TBVaccine TBVaccine is a utility that pretty-prints Python tracebacks. It automatically highlights lines you care about and deemphasizes lines you don

Stavros Korokithakis 365 Nov 11, 2022
A demo of Prometheus+Grafana for monitoring an ML model served with FastAPI.

ml-monitoring Jeremy Jordan This repository provides an example setup for monitoring an ML system deployed on Kubernetes.

Jeremy Jordan 176 Jan 01, 2023
Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

David Puerta Martín 1 Nov 11, 2021
Ransomware leak site monitoring

RansomWatch RansomWatch is a ransomware leak site monitoring tool. It will scrape all of the entries on various ransomware leak sites, store the data

Zander Work 278 Dec 31, 2022
The new Python SDK for Sentry.io

sentry-python - Sentry SDK for Python This is the next line of the Python SDK for Sentry, intended to replace the raven package on PyPI. from sentry_s

Sentry 1.4k Dec 31, 2022
Greppin' Logs: Leveling Up Log Analysis

This repo contains sample code and example datasets from Jon Stewart and Noah Rubin's presentation at the 2021 SANS DFIR Summit titled Greppin' Logs. The talk was centered around the idea that Forens

Stroz Friedberg 20 Sep 14, 2022
APT-Hunter is Threat Hunting tool for windows event logs

APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover susp

824 Jan 08, 2023