Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Last update: Jul 27, 2022

Overview

OpenVINO Model Server wrapper API for Python

Description

This project provides a Python wrapper class for OpenVINO Model Server.
User can submit DL inference request to OVMS with just a few lines of code.

Sample code

from ovms_wrapper.ovms_wrapper import OpenVINO_Model_Server

import cv2
import numpy as np

ovms = OpenVINO_Model_Server()
ovms.connect('127.0.0.1', 9000)
#print(ovms.get_model_status('resnet_50'))
model = ovms.open_model('resnet_50')
print(model.inputs, model.outputs)

image_file  = 'ovms/lib/python3.8/site-packages/skimage/data/rocket.jpg'
img = cv2.imread(image_file)                # Read an image
res = model.single_image_infer(img)         # Infer
result = res[model.outputs[0]['name']]

# display result
nu = np.array(result)
ma = np.argmax(nu)
print("Result:", ma)

How to setup OpenVINO Model Server (Ubuntu)

Note: OVMS can run on Windows too. Please refer to the official OVMS document for details.

sudo apt update && sudo apt install -y python3-venv
python -m pip install tensorflow tensorflow-serving-api

Create Python virtual env, install OpenVINO, and prepare an IR model
Installing OpenVINO just for downloading a DL model and converting it into OpenVINO IR model. This is not required if you already have the IR models.

python3 -m venv ovms
. ovms/bin/activate
python -m pip install openvino-dev
omz_downloader --name resnet-50-tf
omz_converter --name resnet-50-tf --precisions FP16
deactivate

Start OpenVINO Model Server as Docker container

docker run -d --rm \
  -v $PWD/public/resnet-50-tf/FP16:/models/resnet50/1 \
  -p 9000:9000 \
  openvino/model_server:latest \
  --model_path /models/resnet50 \
  --model_name resnet_50 \
  --port 9000

OVMS will start serving the Resnet-50 model as model-name='resnet_50', model-version=1, and gRPC-port=9000.

Python wrapper class for OpenVINO Model Server. User can submit inference request to OVMS with just a few lines of code

Related tags

Overview

OpenVINO Model Server wrapper API for Python

Description

Sample code

How to setup OpenVINO Model Server (Ubuntu)

Owner

Yasunori Shimura

Automatic deep learning for image classification.

Pytorch implementation of the paper Improving Text-to-Image Synthesis Using Contrastive Learning

Curated list of awesome GAN applications and demo

An unopinionated replacement for PyTorch's Dataset and ImageFolder, that handles Tar archives

Rethinking Transformer-based Set Prediction for Object Detection

Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

Random Walk Graph Neural Networks

CLOOB: Modern Hopfield Networks with InfoLOOB Outperform CLIP

NEG loss implemented in pytorch

IGCN : Image-to-graph convolutional network

Official Keras Implementation for UNet++ in IEEE Transactions on Medical Imaging and DLMIA 2018

A PyTorch-based R-YOLOv4 implementation which combines YOLOv4 model and loss function from R3Det for arbitrary oriented object detection.

Live training loss plot in Jupyter Notebook for Keras, PyTorch and others

Key information extraction from invoice document with Graph Convolution Network

Differentiable simulation for system identification and visuomotor control

Using deep learning model to detect breast cancer.

Using knowledge-informed machine learning on the PRONOSTIA (FEMTO) and IMS bearing data sets. Predict remaining-useful-life (RUL).

codes for Image Inpainting with External-internal Learning and Monochromic Bottleneck

Diabetes-Feature-Engineering - A machine learning model that can predict whether people have diabetes when their characteristics are specified

A static analysis library for computing graph representations of Python programs suitable for use with graph neural networks.