A RESTful API for creating and monitoring resource components of a hypothetical build system. Built with FastAPI and pydantic. Complete with testing and CI.

Overview

diskspace-monitor-CRUD

Diskspace Monitor Test Suite

Background

The build system is part of a large environment with a multitude of different components. Many of the components have some sort of storage (examples: crash dump handler, versioning system, build distribution). To ensure none of the services go down due to a lack of available storage, the systems have an agent that reports disk usage back to a central monitoring facility, which evaluates the collected data against preset rules and provides status and warnings through API endpoints.

The current project implements this central monitoring facility.

To read API Documentation, see API_Documentation.md.

Getting Started Without Docker

Prerequisites

Python >= 3.8 and pip are the only prerequisite. I personally use Pipenv but have provided a requirements.txt file for pip.

pip install --upgrade pip

Installation

  1. Clone the repo
git clone  https://github.com/NHopewell/diskspace-monitor-CRUD

cd diskspace-monitor-CRUD
  1. Create a virtual environment of your choice (in this example, venv)
python -m venv env
  1. Activate the virtual environment
source env/bin/activate
  1. Install the source package in the virtual environment
pip install -e .
  1. Install requirements in virtual environment. If you would like to run tests and add onto the project, install the requirements_dev file instead.
# prod requirments
pip install -r requirements.txt

# dev requirements
pip install -r requirements_dev.txt

Usage

The application code which powers the API can be found in src/diskspacemonitor/. To run the webserver:

cd src/diskspacemonitor/

uvicorn main:app --reload

Now our monitoring system is being served over localhost. You can run my test script which automates sending requests to each end point:

python scripts/example_automated_api_calls.py

This script posts some system components and events (some of which triggered warnings in the system), we can also curl these endpoints to see:

# events
curl http://127.0.0.1:8000/v1/component_events | python -m json.tool

# warnings triggered
curl http://127.0.0.1:8000/v1/resource_warnings | python -m json.tool

To see documentation auto-generated by FastAPI, go to: http://127.0.0.1:8000/docs

Getting Started With Docker

  1. Clone the repo
git clone  https://github.com/NHopewell/diskspace-monitor-CRUD

cd diskspace-monitor-CRUD
  1. Build the Dockerfile:
docker build . -t diskspace-monitor
  1. You'll notice in the Dockerfile that we are using the port 8000. Now run the Docker image with port forwarding:
docker run -p 8000:8000 diskspace-monitor

The application is now accessible over localhost http://127.0.0.1:8000/docs

Testing and CI

This project is setup with the following things to ensure PEP8 compliance and proper builds:

  • pre-commit hooks including black, Flake8, and other hooks.
  • local tests for both the API and underlying models with Pytest.
  • virtual env management with tox to run pytests on different Python versions and environments.
  • github actions to automatically run tox with these different Python versions across different operating systems when changes are made to the repo.

To execute all tests manually in your virtualenv, run:

pytest

To execute all tests on your system in multiple virtual environments with different configurations, run:

tox

This will run the test suite in 6 different virtural environments using ubuntu and Windows, each with Python versions 3.8, 3.9. and 3.10.

On git pushes to master or pull requests, tox will be run in these 6 environments concurrently on separate machines.

Owner
Nick Hopewell
Nick Hopewell
Socket.IO integration for Flask applications.

Flask-SocketIO Socket.IO integration for Flask applications. Installation You can install this package as usual with pip: pip install flask-socketio

Miguel Grinberg 4.9k Jan 03, 2023
FastAPI-Amis-Admin is a high-performance, efficient and easily extensible FastAPI admin framework. Inspired by django-admin, and has as many powerful functions as django-admin.

简体中文 | English 项目介绍 FastAPI-Amis-Admin fastapi-amis-admin是一个拥有高性能,高效率,易拓展的fastapi管理后台框架. 启发自Django-Admin,并且拥有不逊色于Django-Admin的强大功能. 源码 · 在线演示 · 文档 · 文

AmisAdmin 318 Dec 31, 2022
Simple notes app backend using Python's FastAPI framework.

my-notes-app Simple notes app backend using Python's FastAPI framework. Route "/": User login (GET): return 200, list of all of their notes; User sign

José Gabriel Mourão Bezerra 2 Sep 17, 2022
A rate limiter for Starlette and FastAPI

SlowApi A rate limiting library for Starlette and FastAPI adapted from flask-limiter. Note: this is alpha quality code still, the API may change, and

Laurent Savaete 562 Jan 01, 2023
Easy and secure implementation of Azure AD for your FastAPI APIs 🔒

FastAPI-Azure-auth Azure AD Authentication for FastAPI apps made easy. 🚀 Description FastAPI is a modern, fast (high-performance), web framework for

Intility 216 Dec 27, 2022
Prometheus integration for Starlette.

Starlette Prometheus Introduction Prometheus integration for Starlette. Requirements Python 3.6+ Starlette 0.9+ Installation $ pip install starlette-p

José Antonio Perdiguero 229 Dec 21, 2022
Keycloak integration for Python FastAPI

FastAPI Keycloak Integration Documentation Introduction Welcome to fastapi-keycloak. This projects goal is to ease the integration of Keycloak (OpenID

Code Specialist 113 Dec 31, 2022
Local Telegram Bot With FastAPI & Ngrok

An easy local telegram bot server with python, fastapi and ngrok.

Ömer Faruk Özdemir 7 Dec 25, 2022
This is an API developed in python with the FastApi framework and putting into practice the recommendations of the book Clean Architecture in Python by Leonardo Giordani,

This is an API developed in python with the FastApi framework and putting into practice the recommendations of the book Clean Architecture in Python by Leonardo Giordani,

0 Sep 24, 2022
Sample-fastapi - A sample app using Fastapi that you can deploy on App Platform

Getting Started We provide a sample app using Fastapi that you can deploy on App

Erhan BÜTE 2 Jan 17, 2022
Repository for the Demo of using DVC with PyCaret & MLOps (DVC Office Hours - 20th Jan, 2022)

Using DVC with PyCaret & FastAPI (Demo) This repo contains all the resources for my demo explaining how to use DVC along with other interesting tools

Tezan Sahu 6 Jul 22, 2022
Run your jupyter notebooks as a REST API endpoint. This isn't a jupyter server but rather just a way to run your notebooks as a REST API Endpoint.

Jupter Notebook REST API Run your jupyter notebooks as a REST API endpoint. This isn't a jupyter server but rather just a way to run your notebooks as

Invictify 54 Nov 04, 2022
A rate limiter for Starlette and FastAPI

SlowApi A rate limiting library for Starlette and FastAPI adapted from flask-limiter. Note: this is alpha quality code still, the API may change, and

Laurent Savaete 562 Jan 01, 2023
A minimal Streamlit app showing how to launch and stop a FastAPI process on demand

Simple Streamlit + FastAPI Integration A minimal Streamlit app showing how to launch and stop a FastAPI process on demand. The FastAPI /run route simu

Arvindra 18 Jan 02, 2023
Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automatically use request headers such as x-request-id or x-correlation-id.

starlette context Middleware for Starlette that allows you to store and access the context data of a request. Can be used with logging so logs automat

Tomasz Wójcik 300 Dec 26, 2022
CLI and Streamlit applications to create APIs from Excel data files within seconds, using FastAPI

FastAPI-Wrapper CLI & APIness Streamlit App Arvindra Sehmi, Oxford Economics Ltd. | Website | LinkedIn (Updated: 21 April, 2021) fastapi-wrapper is mo

Arvindra 49 Dec 03, 2022
Docker image with Uvicorn managed by Gunicorn for high-performance FastAPI web applications in Python 3.6 and above with performance auto-tuning. Optionally with Alpine Linux.

Supported tags and respective Dockerfile links python3.8, latest (Dockerfile) python3.7, (Dockerfile) python3.6 (Dockerfile) python3.8-slim (Dockerfil

Sebastián Ramírez 2.1k Dec 31, 2022
fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability

FastAPI2 Admin Introduction fastapi-admin2 is an upgraded fastapi-admin, that supports ORM dialects, true Dependency Injection and extendability. Now

Glib 14 Dec 05, 2022
A dynamic FastAPI router that automatically creates CRUD routes for your models

⚡ Create CRUD routes with lighting speed ⚡ A dynamic FastAPI router that automatically creates CRUD routes for your models Documentation: https://fast

Adam Watkins 943 Jan 01, 2023
Simple example of FastAPI + Celery + Triton for benchmarking

You can see the previous work from: https://github.com/Curt-Park/producer-consumer-fastapi-celery https://github.com/Curt-Park/triton-inference-server

Jinwoo Park (Curt) 37 Dec 29, 2022