Deploy a ML inference service on a budget in less than 10 lines of code.

Overview

BudgetML: Deploy ML models on a budget

InstallationQuickstartCommunityDocs

PyPI - ZenML Version PyPI - Python Version PyPI Status GitHub

Give us a Slack GitHub star to show your love!

Why

BudgetML is perfect for practitioners who would like to quickly deploy their models to an endpoint, but not waste a lot of time, money, and effort trying to figure out how to do this end-to-end.

We built BudgetML because it's hard to find a simple way to get a model in production fast and cheaply.

  • Cloud functions are limited in memory and cost a lot at scale.

  • Kubernetes clusters are an overkill for one single model.

  • Deploying from scratch involves learning too many different concepts like SSL certificate generation, Docker, REST, Uvicorn/Gunicorn, backend servers etc., that are simply not within the scope of a typical data scientist.

BudgetML is our answer to this challenge. It is supposed to be fast, easy, and developer-friendly. It is by no means meant to be used in a full-fledged production-ready setup. It is simply a means to get a server up and running as fast as possible with the lowest costs possible.

BudgetML lets you deploy your model on a Google Cloud Platform preemptible instance (which is ~80% cheaper than a regular instance) with a secured HTTPS API endpoint. The tool sets it up in a way that the instance autostarts when it shuts down (at least once every 24 hours) with only a few minutes of downtime. BudgetML ensures the cheapest possible API endpoint with the lowest possible downtime.

Key Features

Cost comparison

BudgetML uses Google Cloud Preemptible instances under-the-hood to reduce costs by 80%. This can potentially mean hundreds of dollars worth of savings. Here is a screenshot of the e2-highmem GCP series, which is regular family of instances to be using for memory intense tasks like ML model inference functions. See the following price comparison (as of Jan 31, 2021 [source])

GCP costs

Even with the lowest machine_type, there is a $46/month savings, and with the highest configuration this is $370/month savings!

Installation

BudgetML is available for easy installation into your environment via PyPI:

pip install budgetml

Alternatively, if you’re feeling brave, feel free to install the bleeding edge:

NOTE: Do so on your own risk, no guarantees given!

pip install git+https://github.com/ebhy/[email protected] --upgrade

Quickstart

BudgetML aims for as simple a process as possible. First set up a predictor:

# predictor.py
class Predictor:
    def load(self):
        from transformers import pipeline
        self.model = pipeline(task="sentiment-analysis")

    async def predict(self, request):
        # We know we are going to use the `predict_dict` method, so we use
        # the request.payload pattern
        req = request.payload
        return self.model(req["text"])[0]

Then launch it with a simple script:

# deploy.py
import budgetml
from predictor import Predictor

# add your GCP project name here.
budgetml = budgetml.BudgetML(project='GCP_PROJECT')

# launch endpoint
budgetml.launch(
    Predictor,
    domain="example.com",
    subdomain="api",
    static_ip="32.32.32.322",
    machine_type="e2-medium",
    requirements=['tensorflow==2.3.0', 'transformers'],
)

For a deeper dive, check out the detailed guide in the examples directory. For more information about the BudgetML API, refer to the docs.

Screenshots

Interactive docs to test endpoints. Support for Images. Interactive docs

Password-protected endpoints: Password protected endpoints

Simple prediction interface: Simple Prediction Interface of BudgetML

Projects using BudgetML

We are proud that BudgetML is actively being used in the following live products:

ZenML: For production scenarios

BudgetML is for users on a budget. If you're working in a more serious production environment, then consider using ZenML as the perfect open-source MLOPs framework for ML production needs. It does more than just deployments, and is more suited for professional workplaces.

Proudly built by two brothers

We are two brothers who love building products, especially ML-related products that make life easier for people. If you use this tool for any of your products, we would love to hear about it and potentially add it to this space. Please get in touch via email.

Oh and please do consider giving us a GitHub star if you like the repository - open-source is hard, and the support keeps us going.

Comments
  • Extra files/scripts in Docker container

    Extra files/scripts in Docker container

    Hi @htahir1 , thanks for the super handy library !

    I am wondering whether or not it is possible to include some extra python file when creating the Docker container? I am attempting to infer a custom model and thus I need a bunch of files like: checkpoint, model file, config and so on.. I couldn't find anything mentioning this in the docs.

    Thanks for your help 😄

    opened by JulesBelveze 4
  • [FEATURE] Quickstart example for sockeye

    [FEATURE] Quickstart example for sockeye

    Is your feature request related to a problem? Please describe. I'm not sure how to run a sockeye (https://github.com/awslabs/sockeye) model with budgetml

    Describe the solution you'd like A quickstart example to run a sockeye model. For example the model built in https://awslabs.github.io/sockeye/tutorials/wmt.html .

    Describe alternatives you've considered Using https://github.com/jamesewoo/sockeye-serving/tree/master/src/sockeye_serving or writing FastAPI endpoints that import sockeye.

    Additional context https://github.com/jamesewoo/sockeye-serving/tree/master/src/sockeye_serving does not seem to be in active development.

    opened by michaelhochleitner 3
  • [BUG]

    [BUG]

    Describe the bug A clear and concise description of what the bug is.

    To Reproduce Steps to reproduce the behavior:

    1. Go to '...'
    2. Click on '....'
    3. Scroll down to '....'
    4. See error

    Expected behavior A clear and concise description of what you expected to happen.

    Screenshots If applicable, add screenshots to help explain your problem.

    Stack Trace If applicable, add the error stack trace to help explain your problem.

    ** Context (please complete the following information):**

    • OS: [e.g. Ubuntu 18.04]
    • Python Version: [e.g. 3.6.6]
    • BudgetML Version: [e.g. 0.1.0]

    Additional information Add any other context about the problem here.

    opened by aniket23456 2
  • Location error

    Location error

    Describe the bug As a newbie in GCP, I'm trying to run BudgetML with the "getting started" code shared. After setting up GCP, and running run_budget_ml.py (which contains the budget_ml.launch() call), I get the following error:

    Traceback (most recent call last): File "run_budget_ml.py", line 24, in budgetml.launch( File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/budgetml/main.py", line 321, in launch self.create_scheduler_job( File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/budgetml/main.py", line 266, in create_scheduler_job create_gcp_scheduler_job(project_id, topic, schedule, region) File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/budgetml/gcp/scheduler.py", line 30, in create_scheduler_job response = client.create_job( File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/google/cloud/scheduler_v1/services/cloud_scheduler/client.py", line 595, in create_job response = rpc(request, retry=retry, timeout=timeout, metadata=metadata,) File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/google/api_core/gapic_v1/method.py", line 145, in call return wrapped_func(*args, **kwargs) File "/Users/yadapruksachatkun/opt/anaconda3/lib/python3.8/site-packages/google/api_core/grpc_helpers.py", line 75, in error_remapped_callable six.raise_from(exceptions.from_grpc_error(exc), exc) File "", line 3, in raise_from google.api_core.exceptions.InvalidArgument: 400 Location must equal us-west2 because the App Engine app that is associated with this project is located in us-west2

    My app engine region is us-west-2, and I also set my project region to us-west-2. What region should I be setting? Thank you!

    opened by pruksmhc 1
  • [BUG] Better alignment with REST API: send 500 not 400 if predictor couldn't get loaded

    [BUG] Better alignment with REST API: send 500 not 400 if predictor couldn't get loaded

    Describe the bug Hi! first of all, thanks for such a neat tool! :tada:

    It's not a bug, I just thought that sending HTTP 400 is not good when the predictor couldn't get loaded (all /predict* routes):

    https://github.com/ebhy/budgetml/blob/7ade99c795451656401b3abdbd088b87eb8538eb/server/app/main.py#L96-L105

    I think, it's better to use a 5XX server-side error:

    • HTTP 400 means that there was a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing).
    • HTTP 500 means that the server encountered an unexpected condition that prevented it from fulfilling the request. This error response is a generic "catch-all" response. Usually, this indicates the server cannot find a better 5xx error code to response.
    opened by atemate 1
  • Bump fastapi from 0.63.0 to 0.65.2 in /server

    Bump fastapi from 0.63.0 to 0.65.2 in /server

    Bumps fastapi from 0.63.0 to 0.65.2.

    Release notes

    Sourced from fastapi's releases.

    0.65.2

    Security fixes

    This change fixes a CSRF security vulnerability when using cookies for authentication in path operations with JSON payloads sent by browsers.

    In versions lower than 0.65.2, FastAPI would try to read the request payload as JSON even if the content-type header sent was not set to application/json or a compatible JSON media type (e.g. application/geo+json).

    So, a request with a content type of text/plain containing JSON data would be accepted and the JSON data would be extracted.

    But requests with content type text/plain are exempt from CORS preflights, for being considered Simple requests. So, the browser would execute them right away including cookies, and the text content could be a JSON string that would be parsed and accepted by the FastAPI application.

    See CVE-2021-32677 for more details.

    Thanks to Dima Boger for the security report! 🙇🔒

    Internal

    0.65.1

    Security fixes

    0.65.0

    Breaking Changes - Upgrade

    • ⬆️ Upgrade Starlette to 0.14.2, including internal UJSONResponse migrated from Starlette. This includes several bug fixes and features from Starlette. PR #2335 by @​hanneskuettner.

    Translations

    Internal

    0.64.0

    Features

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Improve HTTP status codes

    Improve HTTP status codes

    Submitting this PR in hopes of making the HTTP Status codes more consistent through the project.

    • HTTP 401 Unauthorized (https://tools.ietf.org/html/rfc7235#section-3.1) for when authentication fails
    • HTTP 500 when the Predictor is not initialized correctly

    Feel free to reject this PR if it is not large enough, but just wanted to bring awareness to consistency in the HTTP Status codes your API is sending

    opened by bradleybonitatibus 0
Releases(0.1.0)
  • 0.1.0(Jan 31, 2021)

    Launch Release

    First release for the public!

    Features

    • Integration with Google Cloud Platform.
    • Auto-start orchestration automation.
    • Easy SSL certificate generation via LetsEncrypt.
    • FastAPI server with predict, predict_dict, and predict_image endpoints supported.
    • Custom requirements support.
    • Custom Docker image support.
    • Bare-bones docs and examples.
    Source code(tar.gz)
    Source code(zip)
Efficient face emotion recognition in photos and videos

This repository contains code of face emotion recognition that was developed in the RSF (Russian Science Foundation) project no. 20-71-10010 (Efficien

Andrey Savchenko 239 Jan 04, 2023
PyTorch implementation of the WarpedGANSpace: Finding non-linear RBF paths in GAN latent space (ICCV 2021)

Authors official PyTorch implementation of the "WarpedGANSpace: Finding non-linear RBF paths in GAN latent space" [ICCV 2021].

Christos Tzelepis 100 Dec 06, 2022
This repository compare a selfie with images from identity documents and response if the selfie match.

aws-rekognition-facecompare This repository compare a selfie with images from identity documents and response if the selfie match. This code was made

1 Jan 27, 2022
Tzer: TVM Implementation of "Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation (OOPSLA'22)“.

Artifact • Reproduce Bugs • Quick Start • Installation • Extend Tzer Coverage-Guided Tensor Compiler Fuzzing with Joint IR-Pass Mutation This is the s

12 Dec 29, 2022
An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

An algorithm that handles large-scale aerial photo co-registration, based on SURF, RANSAC and PyTorch autograd.

Luna Yue Huang 41 Oct 29, 2022
This repo is developed for Strong Baseline For Vehicle Re-Identification in Track 2 Ai-City-2021 Challenges

A STRONG BASELINE FOR VEHICLE RE-IDENTIFICATION This paper is accepted to the IEEE Conference on Computer Vision and Pattern Recognition Workshop(CVPR

Cybercore Co. Ltd 78 Dec 29, 2022
A free, multiplatform SDK for real-time facial motion capture using blendshapes, and rigid head pose in 3D space from any RGB camera, photo, or video.

mocap4face by Facemoji mocap4face by Facemoji is a free, multiplatform SDK for real-time facial motion capture based on Facial Action Coding System or

Facemoji 591 Dec 27, 2022
Simple SN-GAN to generate CryptoPunks

CryptoPunks GAN Simple SN-GAN to generate CryptoPunks. Neural network architecture and training code has been modified from the PyTorch DCGAN example.

Teddy Koker 66 Dec 15, 2022
Architecture Patterns with Python (TDD, DDD, EDM)

architecture-traning Architecture Patterns with Python (TDD, DDD, EDM) Chapter 5. 높은 기어비와 낮은 기어비의 TDD 5.2 도메인 계층 테스트를 서비스 계층으로 옮겨야 하는가? 도메인 계층 테스트 def

minsung sim 2 Mar 04, 2022
Official PyTorch implementation of MAAD: A Model and Dataset for Attended Awareness

MAAD: A Model for Attended Awareness in Driving Install // Datasets // Training // Experiments // Analysis // License Official PyTorch implementation

7 Oct 16, 2022
Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency[ECCV 2020]

Self-Supervised Monocular 3D Face Reconstruction by Occlusion-Aware Multi-view Geometry Consistency(ECCV 2020) This is an official python implementati

304 Jan 03, 2023
Torchreid: Deep learning person re-identification in PyTorch.

Torchreid Torchreid is a library for deep-learning person re-identification, written in PyTorch. It features: multi-GPU training support both image- a

Kaiyang 3.7k Jan 05, 2023
Pytorch implementation for the EMNLP 2020 (Findings) paper: Connecting the Dots: A Knowledgeable Path Generator for Commonsense Question Answering

Path-Generator-QA This is a Pytorch implementation for the EMNLP 2020 (Findings) paper: Connecting the Dots: A Knowledgeable Path Generator for Common

Peifeng Wang 33 Dec 05, 2022
FastyAPI is a Stack boilerplate optimised for heavy loads.

FastyAPI A FastAPI based Stack boilerplate for heavy loads. Explore the docs » View Demo · Report Bug · Request Feature Table of Contents About The Pr

Ali Chaayb 47 Dec 27, 2022
Implementation of DropLoss for Long-Tail Instance Segmentation in Pytorch

[AAAI 2021]DropLoss for Long-Tail Instance Segmentation [AAAI 2021] DropLoss for Long-Tail Instance Segmentation Ting-I Hsieh*, Esther Robb*, Hwann-Tz

Tim 37 Dec 02, 2022
[CIKM 2019] Code and dataset for "Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction"

FiGNN for CTR prediction The code and data for our paper in CIKM2019: Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Predicti

Big Data and Multi-modal Computing Group, CRIPAC 75 Dec 30, 2022
retweet 4 satoshi ⚡️

rt4sat retweet 4 satoshi This bot is the codebase for https://twitter.com/rt4sat please feel free to create an issue if you saw any bugs basically thi

6 Sep 30, 2022
Implementation of Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning

advantage-weighted-regression Implementation of Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learning, by Peng et al. (

Omar D. Domingues 1 Dec 02, 2021
Galaxy images labelled by morphology (shape). Aimed at ML development and teaching

Galaxy images labelled by morphology (shape). Aimed at ML debugging and teaching.

Mike Walmsley 14 Nov 28, 2022
SCAAML is a deep learning framwork dedicated to side-channel attacks run on top of TensorFlow 2.x.

SCAAML (Side Channel Attacks Assisted with Machine Learning) is a deep learning framwork dedicated to side-channel attacks. It is written in python and run on top of TensorFlow 2.x.

Google 69 Dec 21, 2022