KSAI Lite is a deep learning inference framework of kingsoft, based on tensorflow lite

Overview

KSAI Lite

English | 简体中文

Documentation Status Release License

KSAI Lite是一个轻量级、灵活性强、高性能且易于扩展的深度学习推理框架,底层基于tensorflow lite,定位支持包括移动端、嵌入式以及服务器端在内的多硬件平台。

当前KSAI Lite已经应用在金山office内部业务中,并逐步支持金山企业的生产任务和众多外部用户。

快速入门

使用KSAI Lite,只需几个简单的步骤,就可以把模型部署到多种终端设备中,运行高性能的推理任务,使用流程如下所示:

一. 准备模型

KSAI Lite框架直接支持模型结构为tflite模型。 如果您手中的模型是由诸如Caffe、MXNet、PyTorch等框架产出的,那么您可以使用工具将模型转换为tflite格式。

二. 模型优化

KSAI Lite框架基于底层tensorflow lite的优化方法,拥有优秀的加速、优化策略及实现,包含量化、子图融合、Kernel优选等优化手段。优化后的模型更轻量级,耗费资源更少,并且执行速度也更快。

三. 下载或编译

KSAI Lite提供了多平台的官方Release预测库下载,我们优先推荐您直接下载 KSAI Lite预编译库,包括了Linux-X64, Linux-ARM, Linux-MIPS64以及Windows-X64索引库Windows-X64动态链接库。 您也可以根据目标平台选择对应的源码编译方法。KSAI Lite 提供了源码编译脚本,位于 tools/目录下,只需要按照docs/目录下的准备环境说明文档environment setup.md搭建好环境然后切到tools/目录调用编译脚本两个步骤即可一键编译得到目标平台的KSAI Lite预测库。

四. 预测示例

KSAI Lite提供了C++ API,并且提供了相应API的完整使用示例: 目录为tensorflow/lite/examples/reg_test/reg_test.cc 您可以参考示例快速了解使用方法,并集成到您自己的项目中去,也可以参考KSAI-Toolkits该项目。

主要特性

  • 多硬件支持
    • KSAI Lite架构已经验证和完整支持从 Mobile 到 Server 多种硬件平台,包括 intel X86、ARM、华为 Kunpeng 920、龙芯Loongson-3A R3、兆芯C4600、Phytium FT1500a等,且正在不断增加更多新硬件支持。
  • 轻量级部署
    • KSAI Lite在设计上对图优化模块和执行引擎实现了良好的解耦拆分,移动端可以直接部署执行阶段,无任何第三方依赖。
  • 高性能
    • 极致的 ARM及X86 CPU 性能优化:针对不同微架构特点实现kernel的定制,最大发挥计算性能,在主流模型上展现出领先的速度优势。
  • 多模型多算子
    • KSAI Lite和tensorflow训练框架的OP对齐,提供广泛的模型支持能力。
    • 目前已对视觉类模型做到了较为充分的支持,覆盖分类、检测和识别,包含了特色的OCR模型的支持,并在不断丰富中。
  • 强大的图分析和优化能力
    • 不同于常规的移动端预测引擎基于 Python 脚本工具转化模型, Lite 架构上有完整基于 C++ 开发的 IR 及相应 Pass 集合,以支持操作融合,计算剪枝,存储优化,量化计算等多类计算图优化。

持续集成

System X86 Linux ARM Linux MIPS64 Linux windows
CPU(32bit) Build Status - - Build Status
CPU(64bit) Build Status - - Build Status
高通骁龙845 - Build Status - -
华为kunpeng920 - Build Status - -
龙芯Loongson-3A - - Build Status -
兆芯C4600 - Build Status - -
Phytium FT1500a - Build Status - -

交流与反馈

版权和许可证

KSAI-Lite由Apache-2.0 license提供

Code for binary and multiclass model change active learning, with spectral truncation implementation.

Model Change Active Learning Paper (To Appear) Python code for doing active learning in graph-based semi-supervised learning (GBSSL) paradigm. Impleme

Kevin Miller 1 Jul 24, 2022
Accurate identification of bacteriophages from metagenomic data using Transformer

PhaMer is a python library for identifying bacteriophages from metagenomic data. PhaMer is based on a Transorfer model and rely on protein-based vocab

Kenneth Shang 9 Nov 30, 2022
Official repository for ABC-GAN

ABC-GAN The work represented in this repository is the result of a 14 week semesterthesis on photo-realistic image generation using generative adversa

IgorSusmelj 10 Jun 23, 2022
GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

GNN4Traffic - This is the repository for the collection of Graph Neural Network for Traffic Forecasting

564 Jan 02, 2023
Official code repository for the EMNLP 2021 paper

Integrating Visuospatial, Linguistic and Commonsense Structure into Story Visualization PyTorch code for the EMNLP 2021 paper "Integrating Visuospatia

Adyasha Maharana 23 Dec 19, 2022
Rotation Robust Descriptors

RoRD Rotation-Robust Descriptors and Orthographic Views for Local Feature Matching Project Page | Paper link Evaluation and Datasets MMA : Training on

Udit Singh Parihar 25 Nov 15, 2022
Code repo for "Cross-Scale Internal Graph Neural Network for Image Super-Resolution" (NeurIPS'20)

IGNN Code repo for "Cross-Scale Internal Graph Neural Network for Image Super-Resolution" [paper] [supp] Prepare datasets 1 Download training dataset

Shangchen Zhou 278 Jan 03, 2023
A DeepStack custom model for detecting common objects in dark/night images and videos.

DeepStack_ExDark This repository provides a custom DeepStack model that has been trained and can be used for creating a new object detection API for d

MOSES OLAFENWA 98 Dec 24, 2022
Hyper-parameter optimization for sklearn

hyperopt-sklearn Hyperopt-sklearn is Hyperopt-based model selection among machine learning algorithms in scikit-learn. See how to use hyperopt-sklearn

1.4k Jan 01, 2023
Fast convergence of detr with spatially modulated co-attention

Fast convergence of detr with spatially modulated co-attention Usage There are no extra compiled components in SMCA DETR and package dependencies are

peng gao 135 Dec 07, 2022
Long Expressive Memory (LEM)

Long Expressive Memory for Sequence Modeling This repository contains the implementation to reproduce the numerical experiments of the paper Long Expr

Konstantin Rusch 47 Dec 17, 2022
Label-Free Model Evaluation with Semi-Structured Dataset Representations

Label-Free Model Evaluation with Semi-Structured Dataset Representations Prerequisites This code uses the following libraries Python 3.7 NumPy PyTorch

8 Oct 06, 2022
Deep Learning to Create StepMania SM FIles

StepCOVNet Running Audio to SM File Generator Currently only produces .txt files. Use SMDataTools to convert .txt to .sm python stepmania_note_generat

Chimezie Iwuanyanwu 8 Jan 08, 2023
Sequence Modeling with Structured State Spaces

Structured State Spaces for Sequence Modeling This repository provides implementations and experiments for the following papers. S4 Efficiently Modeli

HazyResearch 896 Jan 01, 2023
Few-Shot Object Detection via Association and DIscrimination

Few-Shot Object Detection via Association and DIscrimination Code release of our NeurIPS 2021 paper: Few-Shot Object Detection via Association and DIs

Cao Yuhang 49 Dec 18, 2022
Pose Transformers: Human Motion Prediction with Non-Autoregressive Transformers

Pose Transformers: Human Motion Prediction with Non-Autoregressive Transformers This is the repo used for human motion prediction with non-autoregress

Idiap Research Institute 26 Dec 14, 2022
It's final year project of Diploma Engineering. This project is based on Computer Vision.

Face-Recognition-Based-Attendance-System It's final year project of Diploma Engineering. This project is based on Computer Vision. Brief idea about ou

Neel 10 Nov 02, 2022
SegNet model implemented using keras framework

keras-segnet Implementation of SegNet-like architecture using keras. Current version doesn't support index transferring proposed in SegNet article, so

185 Aug 30, 2022
Code repository for Self-supervised Structure-sensitive Learning, CVPR'17

Self-supervised Structure-sensitive Learning (SSL) Ke Gong, Xiaodan Liang, Xiaohui Shen, Liang Lin, "Look into Person: Self-supervised Structure-sensi

Clay Gong 219 Dec 29, 2022
NaturalProofs: Mathematical Theorem Proving in Natural Language

NaturalProofs: Mathematical Theorem Proving in Natural Language NaturalProofs: Mathematical Theorem Proving in Natural Language Sean Welleck, Jiacheng

Sean Welleck 83 Jan 05, 2023