A Python package which supports global logfmt formatted logging.

Related tags

Loggingpythonlogginglogfmt
Overview

Python Logfmter

pre-commit test

A Python package which supports global logfmt formatted logging.

Install

$ pip install logfmter

Usage

Before integrating this library, you should be familiar with Python's logging functionality. I recommend reading the Basic Logging Tutorial.

This package exposes a single Logfmter class that can be integrated into the standard library logging system similar to any logging.Formatter.

The provided formatter will logfmt encode all logs. Key value pairs are provided via the extra keyword argument or by passing a dictionary as the log message.

Basic

import logging
from logfmter import Logfmter

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(Logfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error("hello", extra={"alpha": 1}) # at=ERROR msg=hello alpha=1
logger.error({"token": "Hello, World!"}) # at=ERROR token="Hello, World!"

Customize

You can subclass the formatter to change its behavior.

import logging
from logfmter import Logfmter


class CustomLogfmter(Logfmter):
    """
    Provide a custom logfmt formatter which formats
    booleans as "yes" or "no" strings.
    """

    @classmethod
    def format_value(cls, value):
        if isinstance(value, bool):
            return "yes" if value else "no"

	return super().format_value(value)

handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(CustomLogfmter())

logger = logging.getLogger()
logger.addHandler(handler)

logger.error({"example": True}) # at=ERROR example=yes

Development

Required Software

Refer to the links provided below to install these development dependencies:

Getting Started

Setup

$ <runtimes.txt xargs -n 1 pyenv install -s
$ direnv allow
$ pip install -r requirements/dev.txt
$ pre-commit install
$ pip install -e .

Tests

Run the test suite against the active python environment.

$ pytest

Run the test suite against the active python environment and watch the codebase for any changes.

$ ptw

Run the test suite against all supported python versions.

$ tox

Publishing

Build

$ python -m build

Upload

$ twine upload dist/*
Comments
  • Question: How to include default / reserved attributes in the log message?

    Question: How to include default / reserved attributes in the log message?

    Thank you for making this library available. I have been looking at the tests and docs but could not find out how I can include some of the keys you defined as "RESERVED" in my message, such as asctime. I would like to say::

    >>> logger.info("hello")
"asctime=2022-04-20 msg='hello'"

    without having to pass asctime in as an extra. I have tried the below in my dictconfig but did not work::

            'logfmt': {
            'format': 'on=%(asctime)s level=%(levelname)s module=%(name)s msg=%(message)s',
            'class': 'logfmter.Logfmter',
        },

    Is this possible?

    opened by meitham 3
  • Include Native Log Record Attributes

    Include Native Log Record Attributes

    We should be able to configure the logger to always include some default keys. For example, a user may want to always include a time or logger key.

    This functionality is seen in the python-json-logger.

    opened by jteppinette 1
  • Add Python 11 Official Support / Testing

    Add Python 11 Official Support / Testing

    I assume this will work on Python 11 just fine, but let's add the Python 11 image to the test scripts and see if there are any improvements that can be made.

    opened by jteppinette 0
  • Update Structure

    Update Structure

    We should move the logfmter directory inside src as suggested by:

    • https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#choosing-a-test-layout-import-rules
    • https://blog.ionelmc.ro/2014/05/25/python-packaging/#the-structure%3E

    This is highly recommend and will also support the usage of importlib import mode as recommended here.

    opened by jteppinette 0
  • Exclude Keys

    Exclude Keys

    Some third party libraries may make log calls with keys which we don't want to log. We should be able to exclude these keys from the final log line.

    For example, when overriding uvicorn logging. The color_message key is currently printed (it is used by their custom color formatter). If we could exclude this key, then we could prevent the following output:

    at=INFO msg="Started server process [97819]" color_message="Started server process [%d]"
at=INFO msg="Waiting for application startup."
at=INFO msg="Application startup complete."
at=INFO msg="Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)" color_message="Uvicorn running on %s://%s:%d (Press CTRL+C to quit)"
    opened by jteppinette 0
Releases(v0.0.6)

  • v0.0.6(Apr 22, 2022)

  • v0.0.5(Apr 20, 2022)

    • Add support for include native log record attributes in the final log output.
    • Add support for overriding the date format used when formatting the asctime attribute.
    Source code(tar.gz)
    Source code(zip)

  • v0.0.4(Mar 30, 2022)

    • Fix the usage documentation
    • Escape newline characters in all logged values. Previously, you could generate multi-line log statements. This should never be the case.
    • Add support for auto-generating exc_info parameters. If the log record is generated with exc_info, as in when using logging.exception(...), the log message will contain properly formatted exception and traceback information.
    • Add support for logging with an empty message dictionary.
    Source code(tar.gz)
    Source code(zip)

  • v0.0.3(Feb 12, 2022)

  • v0.0.1(Feb 6, 2022)

Owner
Joshua Taylor Eppinette
Joshua Taylor Eppinette
GitHub Repository https://pypi.org/project/logfmter/
pyEventLogger - a simple Python Library for making customized Logs of certain events that occur in a program

pyEventLogger is a simple Python Library for making customized Logs of certain events that occur in a program. The logs can be fully customized and can be printed in colored format or can be stored i

Siddhesh Chavan 2 Nov 03, 2022
Key Logger - Key Logger using Python

Key_Logger Key Logger using Python This is the basic Keylogger that i have made

Mudit Sinha 2 Jan 15, 2022
Colored terminal output for Python's logging module

coloredlogs: Colored terminal output for Python's logging module The coloredlogs package enables colored terminal output for Python's logging module.

Peter Odding 496 Dec 30, 2022
Log4j alternative for Python

Log4p Log4p is the most secure logging library ever created in this and all other universes. Usage: import log4p log4p.log('"Wow, this library is sec

Isaak Uchakaev 15 Dec 16, 2022
Simple and versatile logging library for python 3.6 above

Simple and versatile logging library for python 3.6 above

Miguel 1 Nov 23, 2022
giving — the reactive logger

giving is a simple, magical library that lets you log or "give" arbitrary data throughout a program and then process it as an event stream.

Olivier Breuleux 0 May 24, 2022
Summarize LSF job properties by parsing log files.

Summarize LSF job properties by parsing log files of workflows executed by Snakemake.

Kim 4 Jan 09, 2022
APT-Hunter is Threat Hunting tool for windows event logs

APT-Hunter is Threat Hunting tool for windows event logs which made by purple team mindset to provide detect APT movements hidden in the sea of windows event logs to decrease the time to uncover susp

824 Jan 08, 2023
A Python package which supports global logfmt formatted logging.

Python Logfmter A Python package which supports global logfmt formatted logging. Install $ pip install logfmter Usage Before integrating this library,

Joshua Taylor Eppinette 15 Dec 29, 2022
Logging system for the TPC software.

tpc_logger Logging system for the TPC software. The TPC Logger class provides a singleton for logging information within C++ code or in the python API

UC Davis Machine Learning 1 Jan 10, 2022
Discord-Image-Logger - Discord Image Logger With Python

Discord-Image-Logger A exploit I found in discord. Working as of now. Explanatio

111 Dec 31, 2022
A watchdog and logger to Discord for hosting ScPrime servers.

ScpDog A watchdog and logger to Discord for hosting ScPrime servers. Designed to work on Linux servers. This is only capable of sending the logs from

Keagan Landfried 3 Jan 10, 2022
A python logging library

logi v1.3.4 instolation the lib works on python 3x versions pip install logi examples import import logi log = logger(path='C:/file path', timestamp=T

2 Jul 06, 2022
Json Formatter for the standard python logger

This library is provided to allow standard python logging to output log data as json objects. With JSON we can make our logs more readable by machines and we can stop writing custom parsers for syslo

Zakaria Zajac 1.4k Jan 04, 2023
Python bindings for g3log

g3logPython Python bindings for g3log This library provides python3 bindings for g3log + g3sinks (currently logrotate, syslog, and a color-terminal ou

4 May 21, 2021
蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品，基于业界主流的全文检索引擎

蓝鲸日志平台(BK-LOG)是为解决分布式架构下日志收集、查询困难的一款日志产品，基于业界主流的全文检索引擎，通过蓝鲸智云的专属 Agent 进行日志采集，提供多种场景化的采集、查询功能。

腾讯蓝鲸 102 Dec 22, 2022
Progressbar 2 - A progress bar for Python 2 and Python 3 - "pip install progressbar2"

Text progress bar library for Python. Travis status: Coverage: Install The package can be installed through pip (this is the recommended method): pip

Rick van Hattem 795 Dec 18, 2022
Track Nano accounts and notify via log file or email

nano-address-notifier Track accounts and notify via log file or email Required python libs

Joohansson (Json) 4 Nov 08, 2021
A new kind of Progress Bar, with real time throughput, eta and very cool animations!

alive-progress :) A new kind of Progress Bar, with real-time throughput, eta and very cool animations! Ever found yourself in a remote ssh session, do

Rogério Sampaio de Almeida 4k Dec 30, 2022
Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

Log processor for nginx or apache that extracts user and user sessions and calculates other types of useful data for bot detection or traffic analysis

David Puerta Martín 1 Nov 11, 2021