Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Last update: Jul 27, 2022

Overview

OpenVINO Model Server wrapper API for Python

Description

This project provides a Python wrapper class for OpenVINO Model Server.
User can submit DL inference request to OVMS with just a few lines of code.

Sample code

from ovms_wrapper.ovms_wrapper import OpenVINO_Model_Server

import cv2
import numpy as np

ovms = OpenVINO_Model_Server()
ovms.connect('127.0.0.1', 9000)
#print(ovms.get_model_status('resnet_50'))
model = ovms.open_model('resnet_50')
print(model.inputs, model.outputs)

image_file  = 'ovms/lib/python3.8/site-packages/skimage/data/rocket.jpg'
img = cv2.imread(image_file)                # Read an image
res = model.single_image_infer(img)         # Infer
result = res[model.outputs[0]['name']]

# display result
nu = np.array(result)
ma = np.argmax(nu)
print("Result:", ma)

How to setup OpenVINO Model Server (Ubuntu)

Note: OVMS can run on Windows too. Please refer to the official OVMS document for details.

sudo apt update && sudo apt install -y python3-venv
python -m pip install tensorflow tensorflow-serving-api

Create Python virtual env, install OpenVINO, and prepare an IR model
Installing OpenVINO just for downloading a DL model and converting it into OpenVINO IR model. This is not required if you already have the IR models.

python3 -m venv ovms
. ovms/bin/activate
python -m pip install openvino-dev
omz_downloader --name resnet-50-tf
omz_converter --name resnet-50-tf --precisions FP16
deactivate

Start OpenVINO Model Server as Docker container

docker run -d --rm \
  -v $PWD/public/resnet-50-tf/FP16:/models/resnet50/1 \
  -p 9000:9000 \
  openvino/model_server:latest \
  --model_path /models/resnet50 \
  --model_name resnet_50 \
  --port 9000

OVMS will start serving the Resnet-50 model as model-name='resnet_50', model-version=1, and gRPC-port=9000.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Related tags

Overview

OpenVINO Model Server wrapper API for Python

Description

Sample code

How to setup OpenVINO Model Server (Ubuntu)

Owner

Yasunori Shimura

A big endian Gentoo port developed on a Pine64.org RockPro64

Official repository of DeMFI (arXiv.)

Drslmarkov - Distributionally Robust Structure Learning for Discrete Pairwise Markov Networks

HistoSeg : Quick attention with multi-loss function for multi-structure segmentation in digital histology images

A curated list and survey of awesome Vision Transformers.

Compact Bilinear Pooling for PyTorch

Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Wider or Deeper: Revisiting the ResNet Model for Visual Recognition

Semantic Segmentation for Aerial Imagery using Convolutional Neural Network

Graph Robustness Benchmark: A scalable, unified, modular, and reproducible benchmark for evaluating the adversarial robustness of Graph Machine Learning.

Do you like Quick, Draw? Well what if you could train/predict doodles drawn inside Streamlit? Also draws lines, circles and boxes over background images for annotation.

A deep learning CNN model to identify and classify and check if a person is wearing a mask or not.

Introduction to AI assignment 1 HCM University of Technology, term 211

Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filtering

An AutoML Library made with Optuna and PyTorch Lightning

Pytorch Implementation of "Desigining Network Design Spaces", Radosavovic et al. CVPR 2020.

[ACM MM 2021] Joint Implicit Image Function for Guided Depth Super-Resolution

Deep Anomaly Detection with Outlier Exposure (ICLR 2019)

Code for paper "Vocabulary Learning via Optimal Transport for Neural Machine Translation"

This is a vision-based 3d model manipulation and control UI