HiQ - A Modern Observability System

Overview

🦉 A Modern Observability System

Documentation Status CodeCov Github release lic

HiQ is a declarative, non-intrusive, dynamic and transparent tracking system for both monolithic application and distributed system. It brings the runtime information tracking and optimization to a new level without compromising with speed and system performance, or hiding any tracking overhead information. HiQ applies for both I/O bound and CPU bound applications.

To explain the four features, declarative means you can declare the things you want to track in a text file, which could be a json, yaml or even csv,and no need to change program code. Non-intrusive means HiQ doesn't requires to modify original python code. Dynamic means HiQ supports tracing metrics featuring at run time, which can be used for adaptive tracing. Transparent means HiQ provides the tracing overhead and doesn't hide it no matter it is huge or tiny.

In addition to latency tracking, HiQ provides memory, disk I/O and Network I/O tracking out of the box. The output can be saved in form of normal line by line log file, or HiQ tree, or span graph.

HiQ's philosophy is to decouple observability logic from business logic. We don't have to enter the black hole to observe it. Do you like the idea? Leave a if you enjoy the project and welcome to say Hi to us on Slack 👋

Observability of Blackhole, NASA 2019

Installation

pip install hiq-python

Get Started

Let start with a simplest example by running HiQ against a simple monolithic python code 📄 main.py:

# this is the main.py python source code
import time

def func1():
    time.sleep(1.5)
    print("func1")
    func2()

def func2():
    time.sleep(2.5)
    print("func2")

def main():
    func1()

if __name__ == "__main__":
    main()

In this code, there is a simple chain of function calls: main() -> func1 -> func2.

Now we want to trace the functions without modifying its code. Let's run the following:

git clone https://github.com/oracle-samples/hiq.git
cd hiq/examples/quick_start
python main_driver.py

If everything is fine, you should be able to see the output like this:

HiQ Simplest Example

From the screenshot we can see the timestamp and the latency of each function:

main func1 func2 tracing overhead
latency(second) 4.0045 4.0044 2.5026 0.0000163

HiQ just traced the main.py file running without touching one line of its code.

Documentation

HTML: 🔗 HiQ Online Documents | PDF: Please check 🔗 HiQ User Guide.


Logging: https://hiq.readthedocs.io/en/latest/4_o_advanced.html#log-monkey-king
Tracing: https://hiq.readthedocs.io/en/latest/5_distributed.html

Metrics:

Streaming:

Jupyter NoteBook

Add Observability to PaddlePaddle (PaddleOCR)

Add Observability to Onnxruntime (AlexNet)

Examples

Please check 🔗 examples for usage examples.

Contributing

HiQ welcomes contributions from the community. Before submitting a pull request, please review our 🔗 contribution guide.

Security

Please consult the 🔗 security guide for our responsible security vulnerability disclosure process.

Change Log

dev

  • add non-intrusive auto instrumentation for flask: HiQFlaskLatencyOtel ( 🔗 example)

v1.0.2

  • add non-intrusive auto instrumentation for Onnxruntime, Paddlepaddle, PaddleOCR

License

Copyright (c) 2022 Oracle and/or its affiliates. Released under the Universal Permissive License v1.0 as shown at https://oss.oracle.com/licenses/upl/.

Presentation and Demos

Owner
Oracle Sample Code
Oracle Sample Code
Oracle Sample Code
Transform your boring distro into a hacking powerhouse.

Pentizer Transform your boring distro into a hacking powerhouse. Pentizer is a personal project that imports Kali and Parrot repositories in any Debia

Michail Tsimpliarakis 2 Nov 05, 2021
dotfiles - Cristian Valero Abundio

In this repository you can find various configurations to configure your Linux operating system, preferably ArchLinux and its derivatives.

Cristian Valero Abundio 1 Jan 09, 2022
Open-source data observability for modern data teams

Use cases Monitor your data warehouse in minutes: Data anomalies monitoring as dbt tests Data lineage made simple, reliable, and automated dbt operati

889 Jan 01, 2023
Audio-analytics for music-producers! Automate tedious tasks such as musical scale detection, BPM rate classification and audio file conversion.

Click here to be re-directed to the Beat Inspect Streamlit Web-App You are a music producer? Let's get in touch via LinkedIn Fundamental Analytics for

Stefan Rummer 11 Dec 27, 2022
Build your own Etherscan with web3.py

Build your own Etherscan with web3.py Video Tutorial: Run it pip3 install -r requirements.txt export FLASK_APP=app export FLASK_ENV=development flask

35 Jan 02, 2023
Verification of Monty Hall problem by experimental simulation.

Verification of Monty Hall problem by experimental simulation. |中文|English| In the process of learning causal inference, I learned about the Monty Hal

云端听茗 1 Nov 22, 2022
How to access and display MyEnergi data

MyEnergi-Python-Example How to access and display MyEnergi data Windows PC Install a version of Python typically 3.10 The Python code here needs addit

G6EJD 8 Nov 28, 2022
Ingest openldap data into bloodhound

Bloodhound for Linux Ingest a dumped OpenLDAP ldif into neo4j to be visualized in Bloodhound. Usage: ./ldif_to_neo4j.py ./sample.ldif | cypher-shell -

Guillaume Quéré 71 Nov 09, 2022
The next generation Canto RSS daemon

Canto Daemon This is the RSS backend for Canto clients. Canto-curses is the default client at: http://github.com/themoken/canto-curses Requirements De

Jack Miller 155 Dec 28, 2022
Construção de um jogo Dominó na linguagem python com base em algoritmos personalizados.

Domino (projecto-python) Construção de um jogo Dominó na linguaguem python com base em algoritmos personalizados e na: Monografia apresentada ao curso

Nuninha-GC 1 Jan 12, 2022
Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just about anything.

Retrying Retrying is an Apache 2.0 licensed general-purpose retrying library, written in Python, to simplify the task of adding retry behavior to just

Ray Holder 1.9k Dec 29, 2022
VirtualBox Power Driver for MAAS (Metal as a Service)

vboxpower VirtualBox Power Driver for MAAS (Metal as a Service) A way to manage the power of VirtualBox virtual machines via the MAAS webhook driver.

Saeid Bostandoust 131 Dec 17, 2022
Generating rent availability info from Effort rent

Rent-info Generating rent availability info from Effort rent Pre-Installation Latest version of python Pip module json, os, requests, datetime, time i

Laixuan 1 Oct 20, 2021
Opensource Desktop application for kenobi.

Kenobi-Server WIP Opensource desktop application for Kenobi. Download the apple watch app to get started. What is this repo? It's repo for the opensou

Aayush 9 Oct 08, 2022
TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virtual Summer Internship

💻 Welcome to Team Fleming's Repo! #TeamFleming is a multicultural group of 20 young bioinformatics enthusiasts participating in the 2021 HackBio Virt

3 Aug 08, 2021
NotesToCommands - a fully customizable notes / command template program, allowing users to instantly execute terminal commands

NotesToCommands is a fully customizable notes / command template program, allowing users to instantly execute terminal commands with dynamic arguments grouped into sections in their notes/files. It w

zxro 5 Jul 02, 2022
Chemical Analysis Calculator, with full solution display.

Chemicology Chemical Analysis Calculator, to solve problems efficiently by displaying whole solution. Go to releases for downloading .exe, .dmg, Linux

Muhammad Moazzam 2 Aug 06, 2022
Automatización del proceso Inmofianza

Selenium Inmofianza Proyecto de pruebas automatizadas con selenium webdriver para el aplicativo Omnicanalidad Pre-requisitos 📋 Componentes que deben

Natalia Narváez 1 Jan 07, 2022
Open source stenotype engine

Plover Bringing stenography to everyone. Homepage Releases Wiki Blog Google Group Discord Chat About Installation Getting help Contributing Donations

Open Steno Project 2k Jan 09, 2023
Twikoo自定义表情列表 | HexoPlusPlus自定义表情列表(其实基于OwO的项目都可以用的啦)

Twikoo-Magic 更新说明 2021/1/15 基于2021/1/14 Twikoo 更新1.1.0-beta,所有表情都将以缩写形式(如:[ text ]:)输出。1/14之前本仓库有部分表情text缺失及重复, 导致无法正常使用表情 1/14后的所有表情json列表已全部更新

noionion 90 Jan 05, 2023