Experimental Python implementation of OpenVINO Inference Engine (very slow, limited functionality). All codes are written in Python. Easy to read and modify.

Related tags

Deep Learningpythonaideep-learningexperimentalinferencemnistmnist-classificationconvolutionconvolutional-neural-networkinference-engineinference-libraryopenvino
Overview

PyOpenVINO - An Experimental Python Implementation of OpenVINO Inference Engine (minimum-set)

Description

The PyOpenVINO is a spin-off product from my deep learning algorithm study work. This project is aiming at neither practical performance nor rich functionalities. PyOpenVINO can load an OpenVINO IR model (.xml/.bin) and run it. The implementation is quite straightforward and naive. No Optimization technique is used. Thus, the code is easy to read and modify. Supported API is quite limited, but it mimics OpenVINO IE Python API. So, you can easily read and modify the sample code too.

  • Developed as a spin-off from my deep learning study work.
  • Very slow and limited functionality. Not a general DL inference engine.
  • Naive and straightforward code: (I hope) This is a good reference for learning deep-learning technology.
  • Extensible ops: Ops are implemented as plugins. You can easily add your ops as needed.

How to run

Steps 1 and 2 are optional since the converted MNIST IR model is provided.

  1. (Optional) Train a model and generate a 'saved_model' with TensorFlow
python mnist-tf-training.py

The trained model data will be created under ./mnist-savedmodel directory.

  1. (Optional) Convert TF saved_model into OpenVINO IR model
    Prerequisite: You need to have OpenVINO installed (Model Optimizer is required).
convert-model.bat

Converted IR model (.xml/.bin) will be generated in ./models directory.

  1. Run pyOpenVINO sample program
python test_pyopenvino.py

You'll see the output like this.

pyopenvino>python test_pyopenvino.py
inputs: [{'name': 'conv2d_input', 'type': 'Parameter', 'version': 'opset1', 'data': {'element_type': 'f32', 'shape': (1, 1, 28, 28)}, 'output': {0: {'precision': 'FP32', 'dims': (1, 1, 28, 28)}}}]
outputs: [{'name': 'Func/StatefulPartitionedCall/output/_11:0', 'type': 'Result', 'version': 'opset1', 'input': {0: {'precision': 'FP32', 'dims': (1, 10)}}}]
# node_name, time (sec)
conv2d_input Parameter, 0.0
conv2d_input/scale_copy Const, 0.0
StatefulPartitionedCall/sequential/conv2d/Conv2D Convolution, 0.11315417289733887
StatefulPartitionedCall/sequential/conv2d/BiasAdd/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/conv2d/BiasAdd/Add Add, 0.0
StatefulPartitionedCall/sequential/conv2d/Relu ReLU, 0.0010142326354980469
StatefulPartitionedCall/sequential/max_pooling2d/MaxPool MaxPool, 0.020931482315063477
StatefulPartitionedCall/sequential/conv2d_1/Conv2D/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/conv2d_1/Conv2D Convolution, 0.04333162307739258
StatefulPartitionedCall/sequential/conv2d_1/BiasAdd/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/conv2d_1/BiasAdd/Add Add, 0.0
StatefulPartitionedCall/sequential/conv2d_1/Relu ReLU, 0.0
StatefulPartitionedCall/sequential/max_pooling2d_1/MaxPool MaxPool, 0.006029367446899414
StatefulPartitionedCall/sequential/target_conv_layer/Conv2D/ReadVariableOp Const, 0.0010688304901123047
StatefulPartitionedCall/sequential/target_conv_layer/Conv2D Convolution, 0.004073381423950195
StatefulPartitionedCall/sequential/target_conv_layer/BiasAdd/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/target_conv_layer/BiasAdd/Add Add, 0.0
StatefulPartitionedCall/sequential/target_conv_layer/Relu ReLU, 0.0
StatefulPartitionedCall/sequential/target_conv_layer/Relu/Transpose/value6071024 Const, 0.0
StatefulPartitionedCall/sequential/target_conv_layer/Relu/Transpose Transpose, 0.0
StatefulPartitionedCall/sequential/flatten/Const Const, 0.0
StatefulPartitionedCall/sequential/flatten/Reshape Reshape, 0.0
StatefulPartitionedCall/sequential/dense/MatMul/ReadVariableOp Const, 0.0010004043579101562
StatefulPartitionedCall/sequential/dense/MatMul MatMul, 0.0013704299926757812
StatefulPartitionedCall/sequential/dense/BiasAdd/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/dense/BiasAdd/Add Add, 0.0
StatefulPartitionedCall/sequential/dense/Relu ReLU, 0.0
StatefulPartitionedCall/sequential/dense_1/MatMul/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/dense_1/MatMul MatMul, 0.0
StatefulPartitionedCall/sequential/dense_1/BiasAdd/ReadVariableOp Const, 0.0
StatefulPartitionedCall/sequential/dense_1/BiasAdd/Add Add, 0.0
StatefulPartitionedCall/sequential/dense_1/Softmax SoftMax, 0.0009992122650146484
Func/StatefulPartitionedCall/output/_11:0 Result, 0.0
@TOTAL_TIME, 0.21120882034301758
0.21120882034301758 sec/inf
Raw result: {'Func/StatefulPartitionedCall/output/_11:0': array([[7.8985136e-07, 2.0382247e-08, 9.9999917e-01, 1.0367385e-10,
        1.0184062e-10, 1.6024957e-12, 2.0729640e-10, 1.6014919e-08,
        6.5354638e-10, 9.5946295e-14]], dtype=float32)}
Result: [2 0 1 7 8 6 3 4 5 9]
  1. Run Draw-and-Inter demo
python draw-and-infer.py

How to Operate

  • Left click to draw points.
  • Right click to clear the canvas.
    This demo program is using 'numpy' kernels for performance.
    draw-and-infer

A Littile Description of the Implementation

IR model internal representation

This inference engine uses networkx.DiGraph as the internal representation of the IR model. IR model will be translated into nodes and edges.
The nodes represent the ops, and it holds the attributes of the ops (e.g., strides, dilations, etc.).
The edges represent the connection between the nodes. The edges hold the port number for both ends.
The intermediate output from the nodes (feature maps) will be stored in the data attributes in the output port of the node (G.nodes[node_id_num]['output'][port_num]['data'] = feat_map)

An example of the contents (attributes) of a node

node id= 14
 name : StatefulPartitionedCall/sequential/target_conv_layer/Conv2D
 type : Convolution
 version : opset1
 data :
     auto_pad : valid
     dilations : 1, 1
     pads_begin : 0, 0
     pads_end : 0, 0
     strides : 1, 1
 input :
     0 :
         precision : FP32
         dims : (1, 64, 5, 5)
     1 :
         precision : FP32
         dims : (64, 64, 3, 3)
 output :
     2 :
         precision : FP32
         dims : (1, 64, 3, 3)

An example of the contents of an edge

format = (from-layer, from-port, to-layer, to-port)

edge_id= (0, 2)
   {'connection': (0, 0, 2, 0)}

Ops plugins

Operators are implemented as plugins. You can develop an Op in Python and place the file in the op_plugins directory. The inference_engine of pyOpenVINO will search the Python source files in the op_plugins directory at the start time and register them as the Ops plugin.
The file name of the Ops plugin will be treated as the Op name, so it must match the layer type attribute field in the IR XML file.
The inference engine will call the compute() function of the plugin to perform the calculation. The compute() function is the only API between the inference engine and the plugin. The inference engine will collect the required input data and pass it to the compute() function. The input data is in the form of Python dict. ({port_num:data[, port_num:data[, ...]]})
The op needs to calculate the result from the input data and return it as a Python dict. ({port_num:result[, port_num:result[, ...]]})

Kernel implementation: NumPy version and Naive version

Not all, but some Ops have dual kernel implementation, a naive implementation (easy to read), and a NumPy version implementation (a bit faster).
The NumPy version might be x10+ faster than the naive version.
The kernel type can be specified with Executable_Network.kernel_type attribute. You can specify eitgher one of 'naive' (default) or 'numpy'. Please refer to the sample program test_pyopenvino.py for the details.

END

Owner
Yasunori Shimura
Yasunori Shimura
GitHub Repository
Neural style in TensorFlow! 🎨

neural-style An implementation of neural style in TensorFlow. This implementation is a lot simpler than a lot of the other ones out there, thanks to T

Anish Athalye 5.5k Dec 29, 2022
Repositório para arquivos sobre o Módulo 1 do curso Top Coders da Let's Code + Safra

850-Safra-DS-ModuloI Repositório para arquivos sobre o Módulo 1 do curso Top Coders da Let's Code + Safra Para aprender mais Git https://learngitbranc

Brian Nunes 7 Dec 10, 2022
Unofficial pytorch implementation of 'Image Inpainting for Irregular Holes Using Partial Convolutions'

pytorch-inpainting-with-partial-conv Official implementation is released by the authors. Note that this is an ongoing re-implementation and I cannot f

Naoto Inoue 525 Jan 01, 2023
FAST Aiming at the problems of cumbersome steps and slow download speed of GNSS data

FAST Aiming at the problems of cumbersome steps and slow download speed of GNSS data, a relatively complete set of integrated multi-source data download terminal software fast is developed. The softw

ChangChuntao 23 Dec 31, 2022
Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

1.1k Jan 03, 2023
An implementation of a discriminant function over a normal distribution to help classify datasets.

CS4044D Machine Learning Assignment 1 By Dev Sony, B180297CS The question, report and source code can be found here. Github Repo Solution 1 Based on t

Dev Sony 6 Nov 09, 2021
Official Pytorch implementation of 'GOCor: Bringing Globally Optimized Correspondence Volumes into Your Neural Network' (NeurIPS 2020)

Official implementation of GOCor This is the official implementation of our paper : GOCor: Bringing Globally Optimized Correspondence Volumes into You

Prune Truong 71 Nov 18, 2022
Canonical Appearance Transformations

CAT-Net: Learning Canonical Appearance Transformations Code to accompany our paper "How to Train a CAT: Learning Canonical Appearance Transformations

STARS Laboratory 54 Dec 24, 2022
A package to predict protein inter-residue geometries from sequence data

trRosetta This package is a part of trRosetta protein structure prediction protocol developed in: Improved protein structure prediction using predicte

Ivan Anishchenko 185 Jan 07, 2023
Simple and Distributed Machine Learning

Synapse Machine Learning SynapseML (previously MMLSpark) is an open source library to simplify the creation of scalable machine learning pipelines. Sy

Microsoft 3.9k Dec 30, 2022
DumpSMBShare - A script to dump files and folders remotely from a Windows SMB share

DumpSMBShare A script to dump files and folders remotely from a Windows SMB shar

Podalirius 178 Jan 06, 2023
Bare bones use-case for deploying a containerized web app (built in streamlit) on AWS.

Containerized Streamlit web app This repository is featured in a 3-part series on Deploying web apps with Streamlit, Docker, and AWS. Checkout the blo

Collin Prather 62 Jan 02, 2023
Dynamic Graph Event Detection

DyGED Dynamic Graph Event Detection Get Started pip install -r requirements.txt TODO Paper link to arxiv, and how to cite. Twitter Weather dataset tra

Mert Koşan 3 May 09, 2022
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

RETRO - Pytorch (wip) Implementation of RETRO, Deepmind's Retrieval based Attent

Phil Wang 556 Jan 04, 2023
Code for testing convergence rates of Lipschitz learning on graphs

📈 LipschitzLearningRates The code in this repository reproduces the experimental results on convergence rates for k-nearest neighbor graph infinity L

2 Dec 20, 2021
A PyTorch-based R-YOLOv4 implementation which combines YOLOv4 model and loss function from R3Det for arbitrary oriented object detection.

R-YOLOv4 This is a PyTorch-based R-YOLOv4 implementation which combines YOLOv4 model and loss function from R3Det for arbitrary oriented object detect

94 Dec 03, 2022
WatermarkRemoval-WDNet-WACV2021

WatermarkRemoval-WDNet-WACV2021 Thank you for your attention. Citation Please cite the related works in your publications if it helps your research: @

LUYI 63 Dec 05, 2022
FasterAI: A library to make smaller and faster models with FastAI.

Fasterai fasterai is a library created to make neural network smaller and faster. It essentially relies on common compression techniques for networks

Nathan Hubens 193 Jan 01, 2023
Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

ETSformer - Pytorch Implementation of ETSformer, state of the art time-series Transformer, in Pytorch Install $ pip install etsformer-pytorch Usage im

Phil Wang 121 Dec 30, 2022
MCMC samplers for Bayesian estimation in Python, including Metropolis-Hastings, NUTS, and Slice

Sampyl May 29, 2018: version 0.3 Sampyl is a package for sampling from probability distributions using MCMC methods. Similar to PyMC3 using theano to

Mat Leonard 304 Dec 25, 2022