HiQ - A Modern Observability System

Overview

🦉 A Modern Observability System

Documentation Status CodeCov Github release lic

HiQ is a declarative, non-intrusive, dynamic and transparent tracking system for both monolithic application and distributed system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications.

To explain the four features, declarative means you can declare the things you want to track in a text file, which could be a json, yaml or even csv,and no need to change program code. Non-intrusive means HiQ doesn't requires to modify original python code. Dynamic means HiQ supports tracing metrics featuring at run time, which can be used for adaptive tracing. Transparent means HiQ provides the tracing overhead and doesn't hide it no matter it is huge or tiny.

In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.

HiQ's philosophy is to decouple observability logic from business logic. We don't have to enter the black hole to observe it. Do you like the idea? Leave a if you enjoy the project and welcome to say Hi to us on Slack 👋

Observability of Blackhole, NASA 2019

Installation

pip install hiq-python

Get Started

Let start with a simplest example by running HiQ against a simple monolithic python code 📄 main.py:

# this is the main.py python source code
import time

def func1():
    time.sleep(1.5)
    print("func1")
    func2()

def func2():
    time.sleep(2.5)
    print("func2")

def main():
    func1()

if __name__ == "__main__":
    main()

In this code, there is a simple chain of function calls: main() -> func1 -> func2.

Now we want to trace the functions without modifying its code. Let's run the following:

git clone https://github.com/oracle-samples/hiq.git
cd hiq/examples/quick_start
python main_driver.py

If everything is fine, you should be able to see the output like this:

HiQ Simplest Example

From the screenshot we can see the timestamp and the latency of each function:

main func1 func2 tracing overhead
latency(second) 4.0045 4.0044 2.5026 0.0000163

HiQ just traced the main.py file running without touching one line of its code.

Documentation

HTML: 🔗 HiQ Online Documents | PDF: Please check 🔗 HiQ User Guide.


Logging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king
Tracing: https://hiq.readthedocs.io/en/latest/5_distributed.html

Metrics:

Streaming:

Jupyter NoteBook

Add Observability to PaddlePaddle (PaddleOCR)

Add Observability to Onnxruntime (AlexNet)

Examples

Please check 🔗 examples for usage examples.

Contributing

HiQ welcomes contributions from the community. Before submitting a pull request, please review our 🔗 contribution guide.

Security

Please consult the 🔗 security guide for our responsible security vulnerability disclosure process.

Change Log

dev

  • add non-intrusive auto instrumentation for flask: HiQFlaskLatencyOtel ( 🔗 example)

v1.0.2

  • add non-intrusive auto instrumentation for Onnxruntime, Paddlepaddle, PaddleOCR

License

Copyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/.

Presentation and Demos

Owner
Oracle Sample Code
Oracle Sample Code
Oracle Sample Code
Python wrapper around Apple App Store Api

App Store Connect Api This is a Python wrapper around the Apple App Store Api : https://developer.apple.com/documentation/appstoreconnectapi So far, i

123 Jan 06, 2023
tool to automate exploitation of android degubg bridge vulnerability

DISCLAIMER DISCLAIMER: ANY MALICIOUS USE OF THE CONTENTS FROM THIS ARTICLE WILL NOT HOLD THE AUTHOR RESPONSIBLE HE CONTENTS ARE SOLELY FOR EDUCATIONAL

6 Feb 12, 2022
Automatically give thanks to Pypi packages you use in your project!

Automatically give thanks to Pypi packages you use in your project!

Ward 25 Dec 20, 2021
Modern robots.txt Parser for Python

Robots Exclusion Protocol Parser for Python Robots.txt parsing in Python. Goals Fetching -- helper utilities for fetching and parsing robots.txts, inc

Moz 176 Dec 16, 2022
A water drinking notification every hour to keep you healthy while coding :)

Water_Notification A water drinking notification every hour to keep you healthy while coding. 💧 💧 Stay Hydrated Stay Healthy 💧 💧 Authors @CrazyCat

Arghya Banerjee 1 Dec 22, 2021
📽 Streamlit application powered by a PyScaffold project setup

streamlit-demo Streamlit application powered by a PyScaffold project setup. Work in progress: The idea of this repo is to demonstrate how to package a

PyScaffold 2 Oct 10, 2022
A program to generate random numbers b/w 0 to 10 using time

random-num-using-time A program to generate random numbers b/w 0 to 10 using time it uses python's in-built module datetime and an equation which retu

Atul Kushwaha 1 Oct 01, 2022
Painel simples com consulta de cep,CNPJ,placa e ip

Painel mpm Um painel simples com consultas de IP, CNPJ, CEP e PLACA Início 🌐 apt update && apt upgrade -y pkg i python git pip install requests Insta

8 Feb 27, 2022
I³ Tracker for Essential Open Innovation Datasets

I³ Tracker for Essential Open Innovation Datasets This repository is set up to track, version, and contribute updates to the I³ Essential Open Innovat

1 Feb 08, 2022
Bu repoda python ile CAN-Bus çalışmalarını nasıl gerçekleyeceğiniz anlatılmaktadır.

CAN-Bus-with-Python "CAN Bus 1980'li yıllarda Robert BOSCH tarafından geliştirilmiş bir iletişim protokoldür. Hızlı ve hata oranının çok düşük olması

Yunus Emre Coşkun 16 Aug 29, 2022
Always fill your package requirements without the user having to do anything! Simple and easy!

WSL Should now work always-fill-reqs-python3 Always fill your package requirements without the user having to do anything! Simple and easy! Supported

Hashm 7 Jan 19, 2022
A python package template that can be adapted for RAP projects

Warning - this repository is a snapshot of a repository internal to NHS Digital. This means that links to videos and some URLs may not work. Repositor

NHS Digital 3 Nov 08, 2022
NYCU(NCTU)-差勤-助教

NCTU-TA-fill 填寫 差勤-助教時數 有沒有覺得在差勤系統填助教時數有點浪費生命? 今天有個懶鬼浪費好多時間幫大家寫了code 只要填好的必要的資料,就可以讓電腦自動幫你完成差勤助教的時數填寫喔! https://pt-attendance.nctu.edu.tw/verify/userL

14 Dec 21, 2021
A collection of Workflows samples for various use cases

Workflows Samples Workflows allow you to orchestrate and automate Google Cloud and HTTP-based API services with serverless workflows.

Google Cloud Platform 76 Jan 07, 2023
SEH-Helper - Binary Ninja plugin for exploring Structured Exception Handlers

SEH Helper Author: EliseZeroTwo A Binary Ninja helper for exploring structured e

Elise 74 Dec 26, 2022
JSEngine is a simple wrapper of Javascript engines.

JSEngine This is a simple wrapper of Javascript engines, it wraps the Javascript interpreter for Python use. There are two ways to call interpreters,

11 Dec 18, 2022
Algorand Python API examples

Algorand-Py Algorand Python API examples This repo will hold example scripts to monitor activities on Algorand main net. You can: Monitor your assets

Karthik Dutt 2 Jan 23, 2022
The functions we created are included in a script. The necessary parts for pre-processing were taken. Analysis complete.

Feature-Engineering The functions we created are included in a script. The necessary parts for pre-processing were taken. Analysis complete. Business

Ayşe Nur Türkaslan 4 Oct 17, 2021
Launcher program to select which version of the Q-Sys software to launch.

QSC-QSYS Launcher Launcher program to select which version of the Q-Sys software to launch. Instructions To use the application simply save the "Q-Sys

Zach Lisko 2 Sep 28, 2022
Fast STL (ASCII & Binary) importer for Blender

blender-fast-stl-importer Fast STL (ASCII & Binary) importer for Blender based on https://en.wikipedia.org/wiki/STL_(file_format) Technical notes: flo

Iyad Ahmed 7 Apr 17, 2022