CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.

Overview

CV Backbones

including GhostNet, TinyNet, TNT (Transformer in Transformer) developed by Huawei Noah's Ark Lab.

News

2022/01/05 PyramidTNT: An improved TNT baseline is released.

2021/09/28 The paper of TNT (Transformer in Transformer) is accepted by NeurIPS 2021.

2021/09/18 The extended version of Versatile Filters is accepted by T-PAMI.

2021/08/30 GhostNet paper is selected as the Most Influential CVPR 2020 Papers.

2021/08/26 The codes of LegoNet and Versatile Filters has been merged into this repo.

2021/06/15 The code of TNT (Transformer in Transformer) has been released in this repo.

2020/10/31 GhostNet+TinyNet achieves better performance. See details in our NeurIPS 2020 paper: arXiv.

2020/06/10 GhostNet is included in PyTorch Hub.


GhostNet Code

This repo provides GhostNet pretrained models and inference code for TensorFlow and PyTorch:

For training, please refer to tinynet or timm.

TinyNet Code

This repo provides TinyNet pretrained models and inference code for PyTorch:

TNT Code

This repo provides training code and pretrained models of TNT (Transformer in Transformer) for PyTorch:

The code of PyramidTNT is also released:

LegoNet Code

This repo provides the implementation of paper LegoNet: Efficient Convolutional Neural Networks with Lego Filters (ICML 2019)

Versatile Filters Code

This repo provides the implementation of paper Learning Versatile Filters for Efficient Convolutional Neural Networks (NeurIPS 2018)

Citation

@inproceedings{ghostnet,
  title={GhostNet: More Features from Cheap Operations},
  author={Han, Kai and Wang, Yunhe and Tian, Qi and Guo, Jianyuan and Xu, Chunjing and Xu, Chang},
  booktitle={CVPR},
  year={2020}
}
@inproceedings{tinynet,
  title={Model Rubik’s Cube: Twisting Resolution, Depth and Width for TinyNets},
  author={Han, Kai and Wang, Yunhe and Zhang, Qiulin and Zhang, Wei and Xu, Chunjing and Zhang, Tong},
  booktitle={NeurIPS},
  year={2020}
}
@inproceedings{tnt,
  title={Transformer in transformer},
  author={Han, Kai and Xiao, An and Wu, Enhua and Guo, Jianyuan and Xu, Chunjing and Wang, Yunhe},
  booktitle={NeurIPS},
  year={2021}
}
@inproceedings{legonet,
    title={LegoNet: Efficient Convolutional Neural Networks with Lego Filters},
    author={Yang, Zhaohui and Wang, Yunhe and Liu, Chuanjian and Chen, Hanting and Xu, Chunjing and Shi, Boxin and Xu, Chao and Xu, Chang},
    booktitle={ICML},
    year={2019}
  }
@inproceedings{wang2018learning,
  title={Learning versatile filters for efficient convolutional neural networks},
  author={Wang, Yunhe and Xu, Chang and Chunjing, XU and Xu, Chao and Tao, Dacheng},
  booktitle={NeurIPS},
  year={2018}
}

Other versions of GhostNet

This repo provides the TensorFlow/PyTorch code of GhostNet. Other versions and applications can be found in the following:

  1. timm: code with pretrained model
  2. Darknet: cfg file, and description
  3. Gluon/Keras/Chainer: code
  4. Paddle: code
  5. Bolt inference framework: benckmark
  6. Human pose estimation: code
  7. YOLO with GhostNet backbone: code
  8. Face recognition: cavaface, FaceX-Zoo, TFace
Comments
  • TypeError: __init__() got an unexpected keyword argument 'bn_tf'

    TypeError: __init__() got an unexpected keyword argument 'bn_tf'

    Hello, I want to ask what caused the following error when running the train.py file? thank you “ TypeError: init() got an unexpected keyword argument 'bn_tf' ”

    opened by ModeSky 16
  • Counting ReLU vs HardSwish FLOPs

    Counting ReLU vs HardSwish FLOPs

    Thank you very much for sharing the source code. I have a question related to FLOPs counting for ReLU and HardSwish. I saw in the paper the flops are the same in ReLU and HardSwish. Can you explain this situation? image

    opened by jahongir7174 10
  • kernel size in primary convolution of Ghost module

    kernel size in primary convolution of Ghost module

    Hi, It is said in your paper that the primary convolution in Ghost module can have customized kernel size, which is a major difference from existing efficient convolution schemes. However, it seems that in this code all the kernel size of primary convolution in Ghost module are set to [1, 1], and the kernel set in _CONV_DEFS_0 are only used in blocks of stride=2. Is it set intentionally?

    opened by YUHAN666 9
  • 用GhostModule替换Conv2d,loss降的很慢?

    用GhostModule替换Conv2d,loss降的很慢?

    我直接将efficientnet里面的MBConvBlock中的Conv2d替换为GhostModule: Conv2d(in_channels=inp, out_channels=oup, kernel_size=1, bias=False) 替换为 GhostModule(inp, oup), 其他参数不变,为什么损失比以前收敛的更慢了,一直降不下来?请问需要修改其他什么参数吗?

    opened by yc-cui 8
  • Training hyperparams on ImageNet

    Training hyperparams on ImageNet

    Hi, thanks for sharing such a wonderful work, I'd like to reproduce your results on ImageNet, could you please specify training parameters such as initial learning rate, how to decay it, batch size, etc. It would be even better if you can provide tricks to train GhostNet, such as label smoothing and data augmentation. Thx!

    good first issue 
    opened by sean-zhuh 8
  • Why did you exclude EfficientNetB0 from Accuracy-Latency chart?

    Why did you exclude EfficientNetB0 from Accuracy-Latency chart?

    @iamhankai Hi,

    Great work!

    1. Why did you exclude EfficientNetB0 (0.390 BFlops - 76.3% Top1) from Accuracy-Latency chart?

    2. Also what mini_batch_size did you use for training GhostNet?

    flops_latency

    opened by AlexeyAB 8
  • VIG pretrained weights

    VIG pretrained weights

    @huawei-noah-admin cna you please share the VIG pretraiend model on google drive or one drive as baidu is not accessible from our end

    THank in advance

    opened by abhigoku10 7
  • The implementation of Isotropic architecture

    The implementation of Isotropic architecture

    Hi, thanks for sharing this impressive work. The paper mentioned two architectures, Isotropic one and pyramid one. I noticed that in the code, this is a reduce_ratios, and this reduce_ratios are used by a avg_pooling operation to calculate before building the graph. I am wondering whether all I need to do is setting this reduce_ratios to [1,1,1,1] if I want to implement the Isotropic architecture. Thanks.

    self.n_blocks = sum(blocks) channels = opt.channels reduce_ratios = [4, 2, 1, 1] dpr = [x.item() for x in torch.linspace(0, drop_path, self.n_blocks)] num_knn = [int(x.item()) for x in torch.linspace(k, k, self.n_blocks)]

    opened by buptxiaofeng 6
  • Gradient overflow occurs while training tnt-ti model

    Gradient overflow occurs while training tnt-ti model

    ^@^@Train: 41 [ 0/625 ( 0%)] Loss: 4.564162 (4.5642) Time: 96.744s, 21.17/s (96.744s, 21.17/s) LR: 8.284e-04 Data: 94.025 (94.025) ^@^@^@^@Train: 41 [ 50/625 ( 8%)] Loss: 4.395192 (4.4797) Time: 2.742s, 746.96/s (7.383s, 277.38/s) LR: 8.284e-04 Data: 0.057 (4.683) ^@^@^@^@Train: 41 [ 100/625 ( 16%)] Loss: 4.424296 (4.4612) Time: 2.741s, 747.15/s (6.529s, 313.66/s) LR: 8.284e-04 Data: 0.056 (3.831) Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0

    And the top-1 acc is only 0.2 after 40 epochs.

    Any tips available here, dear @iamhankai @yitongh

    opened by jimmyflycv 6
  • Bloated model

    Bloated model

    Hi, I am using Ghostnet backbone for training YoloV3 model in Tensorflow, but I am getting a bloated model. The checkpoint data size is approx. 68MB, but the checkpoint given here is of approx 20MB https://github.com/huawei-noah/ghostnet/blob/master/tensorflow/models/ghostnet_checkpoint.data-00000-of-00001

    I am also training EfficientNet model with YoloV3 and that seems to be working fine, without any bloated size.

    Could anyone or the author please confirm if this is the correct architecture or anything seems weird? I have attached the Ghostnet architecture file out of the code.

    Thanks. ghostnet_model_arch.txt

    opened by ghost 6
  • Replace Conv2d in my network, however it becomes slower, why?

    Replace Conv2d in my network, however it becomes slower, why?

    Above all, thanks for your great work! It really inspires me a lot! But now I have a question.

    I replace all the Conv2d operations in my network except the final ones, the model parameters really becomes much more less. However, when testing, I found that the average forward time decreases a lot by the replacement (from 428FPS down to 354FPS). So, is this a normal phenomenon? Or is this because of the concat operation?

    opened by FunkyKoki 6
  • VIG for segmenation

    VIG for segmenation

    @iamhankai thanks for open-sourcing the code base . Can you please let me knw how to use the pvig for segmentation related activities its really helpful

    THanks in advance

    opened by abhigoku10 0
  • higher performance of ViG

    higher performance of ViG

    I try to train ViG-S on ImageNet and get 80.54% top1 accuracy, which is higher than that in paper, 80.4%. I wonder if 80.4 is the average of multiple trainings? If yes, how many reps do you use?

    opened by tdzdog 9
Releases(GhostNetV2)
Owner
HUAWEI Noah's Ark Lab
Working with and contributing to the open source community in data mining, artificial intelligence, and related fields.
HUAWEI Noah's Ark Lab
The final project for "Applying AI to Wearable Device Data" course from "AI for Healthcare" - Udacity.

Motion Compensated Pulse Rate Estimation Overview This project has 2 main parts. Develop a Pulse Rate Algorithm on the given training data. Then Test

Omar Laham 2 Oct 25, 2022
Source code for our paper "Improving Empathetic Response Generation by Recognizing Emotion Cause in Conversations"

Source code for our paper "Improving Empathetic Response Generation by Recognizing Emotion Cause in Conversations" this repository is maintained by bo

Yuhan Liu 24 Nov 29, 2022
GPU Accelerated Non-rigid ICP for surface registration

GPU Accelerated Non-rigid ICP for surface registration Introduction Preivous Non-rigid ICP algorithm is usually implemented on CPU, and needs to solve

Haozhe Wu 144 Jan 04, 2023
We are More than Our JOints: Predicting How 3D Bodies Move

We are More than Our JOints: Predicting How 3D Bodies Move Citation This repo contains the official implementation of our paper MOJO: @inproceedings{Z

72 Oct 20, 2022
Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs

Project Learning Multiresolution Matrix Factorization and its Wavelet Networks on Graphs, https://arxiv.org/pdf/2111.01940.pdf. Authors Truong Son Hy

5 Jun 28, 2022
This project uses Template Matching technique for object detecting by detection of template image over base image.

Object Detection Project Using OpenCV This project uses Template Matching technique for object detecting by detection the template image over base ima

Pratham Bhatnagar 7 May 29, 2022
Machine Learning Platform for Kubernetes

Reproduce, Automate, Scale your data science. Welcome to Polyaxon, a platform for building, training, and monitoring large scale deep learning applica

polyaxon 3.2k Dec 23, 2022
DeepOBS: A Deep Learning Optimizer Benchmark Suite

DeepOBS - A Deep Learning Optimizer Benchmark Suite DeepOBS is a benchmarking suite that drastically simplifies, automates and improves the evaluation

Aaron Bahde 7 May 12, 2020
QueryDet: Cascaded Sparse Query for Accelerating High-Resolution SmallObject Detection

QueryDet-PyTorch This repository is the official implementation of our paper: QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small O

Chenhongyi Yang 276 Dec 31, 2022
Multi-layer convolutional LSTM with Pytorch

Convolution_LSTM_pytorch Thanks for your attention. I haven't got time to maintain this repo for a long time. I recommend this repo which provides an

Zijie Zhuang 734 Jan 03, 2023
A Framework for Encrypted Machine Learning in TensorFlow

TF Encrypted is a framework for encrypted machine learning in TensorFlow. It looks and feels like TensorFlow, taking advantage of the ease-of-use of t

TF Encrypted 0 Jul 06, 2022
Few-shot NLP benchmark for unified, rigorous eval

FLEX FLEX is a benchmark and framework for unified, rigorous few-shot NLP evaluation. FLEX enables: First-class NLP support Support for meta-training

AI2 85 Dec 03, 2022
Spectral normalization (SN) is a widely-used technique for improving the stability and sample quality of Generative Adversarial Networks (GANs)

Why Spectral Normalization Stabilizes GANs: Analysis and Improvements [paper (NeurIPS 2021)] [paper (arXiv)] [code] Authors: Zinan Lin, Vyas Sekar, Gi

Zinan Lin 32 Dec 16, 2022
A PyTorch implementation of "SelfGNN: Self-supervised Graph Neural Networks without explicit negative sampling"

SelfGNN A PyTorch implementation of "SelfGNN: Self-supervised Graph Neural Networks without explicit negative sampling" paper, which will appear in Th

Zekarias Tilahun 24 Jun 21, 2022
Automatic labeling, conversion of different data set formats, sample size statistics, model cascade

Simple Gadget Collection for Object Detection Tasks Automatic image annotation Conversion between different annotation formats Obtain statistical info

llt 4 Aug 24, 2022
Bottleneck Transformers for Visual Recognition

Bottleneck Transformers for Visual Recognition Experiments Model Params (M) Acc (%) ResNet50 baseline (ref) 23.5M 93.62 BoTNet-50 18.8M 95.11% BoTNet-

Myeongjun Kim 236 Jan 03, 2023
Connecting Java/ImgLib2 + Python/NumPy

imglyb imglyb aims at connecting two worlds that have been seperated for too long: Python with numpy Java with ImgLib2 imglyb uses jpype to access num

ImgLib2 29 Dec 21, 2022
Lenia - Mathematical Life Forms

For full version list, see Timeline in Lenia portal [2020-10-13] Update Python version with multi-kernel and multi-channel extensions (v3.4 LeniaNDK.p

Bert Chan 3.1k Dec 28, 2022
This repository comes with the paper "On the Robustness of Counterfactual Explanations to Adverse Perturbations"

Robust Counterfactual Explanations This repository comes with the paper "On the Robustness of Counterfactual Explanations to Adverse Perturbations". I

Marco 5 Dec 20, 2022
Happywhale - Whale and Dolphin Identification Silver🥈 Solution (26/1588)

Kaggle-Happywhale Happywhale - Whale and Dolphin Identification Silver 🥈 Solution (26/1588) 竞赛方案思路 图像数据预处理-标志性特征图片裁剪:首先根据开源的标注数据训练YOLOv5x6目标检测模型,将训练集

Franxx 20 Nov 14, 2022