CRISP: Critical Path Analysis of Microservice Traces

This repo contains code to compute and present critical path summary from Jaeger microservice traces. To use first collect the microservice traces of a specific endpoint in a directory (say traces). Let the traces be for OP operation and SVC service (these are Jaeger termonologies). python3 process.py --operationName OP --serviceName SVC -t <path to trace> -o . --parallelism 8 will produce the critical path summary using 8 concurrent processes. The summary will be output in the current directory as an HTML file with a heatmap, flamegraph, and summary text in criticalPaths.html. It will also produce three flamegraphs flame-graph-*.svg for three different percentile values.

The script accepts the following options:

python3 process.py --help
usage: process.py [-h] -a OPERATIONNAME -s SERVICENAME [-t TRACEDIR] [--file FILE] -o OUTPUTDIR
                  [--parallelism PARALLELISM] [--topN TOPN] [--numTrace NUMTRACE] [--numOperation NUMOPERATION]

optional arguments:
  -h, --help            show this help message and exit
  -a OPERATIONNAME, --operationName OPERATIONNAME
                        operation name
  -s SERVICENAME, --serviceName SERVICENAME
                        name of the service
  -t TRACEDIR, --traceDir TRACEDIR
                        path of the trace directory (mutually exclusive with --file)
  --file FILE           input path of the trace file (mutually exclusivbe with --traceDir)
  -o OUTPUTDIR, --outputDir OUTPUTDIR
                        directory where output will be produced
  --parallelism PARALLELISM
                        number of concurrent python processes.
  --topN TOPN           number of services to show in the summary
  --numTrace NUMTRACE   number of traces to show in the heatmap
  --numOperation NUMOPERATION
                        number of operations to show in the heatmap

CRISP: Critical Path Analysis of Microservice Traces

Related tags

Overview

CRISP: Critical Path Analysis of Microservice Traces

Owner

Uber Research

A Python 3 library making time series data mining tasks, utilizing matrix profile algorithms

talkbox is a scikit for signal/speech processing, to extend scipy capabilities in that domain.

GWpy is a collaboration-driven Python package providing tools for studying data from ground-based gravitational-wave detectors

t-SNE and hierarchical clustering are popular methods of exploratory data analysis, particularly in biology.

Functional Data Analysis, or FDA, is the field of Statistics that analyses data that depend on a continuous parameter.

Implementation in Python of the reliability measures such as Omega.

ETL flow framework based on Yaml configs in Python

This creates a ohlc timeseries from downloaded CSV files from NSE India website and makes a SQLite database for your research.

Pandas and Dask test helper methods with beautiful error messages.

This is an example of how to automate Ridit Analysis for a dataset with large amount of questions and many item attributes

Integrate bus data from a variety of sources (batch processing and real time processing).

follow-analyzer helps GitHub users analyze their following and followers relationship

Extract Thailand COVID-19 Cluster data from daily briefing pdf.

PrimaryBid - Transform application Lifecycle Data and Design and ETL pipeline architecture for ingesting data from multiple sources to redshift

Utilize data analytics skills to solve real-world business problems using Humana’s big data

Big Data & Cloud Computing for Oceanography

ASTR 302: Python for Astronomy (Winter '22)

Useful tool for inserting DataFrames into the Excel sheet.

BioMASS - A Python Framework for Modeling and Analysis of Signaling Systems

An ETL Pipeline of a large data set from a fictitious music streaming service named Sparkify.