NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Last update: Jan 03, 2023

Overview

Real-ESRGAN ncnn Vulkan

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan 😁

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

TODO List
Usages

If Real-ESRGAN is helpful in your photos/projects, please help to ⭐ this repo or recommend it to your friends. Thanks 😊
Other recommended projects:
▶️ Real-ESRGAN: A practical algorithm for general image restoration
▶️ GFPGAN: A practical algorithm for real-world face restoration
▶️ BasicSR: An open-source image and video restoration toolbox
▶️ facexlib: A collection that provides useful face-relation functions.
▶️ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

📖 Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper] [Project Page] [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

⏳ TODO List

Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
Bug: Some PCs will output black images
Add the guidance for ncnn model conversion
Support face restoration - GFPGAN

💻 Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)

input-path and output-path accept either file path or directory path
scale = scale level, 4 = upscale 4x
tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

Intel: https://downloadcenter.intel.com/product/80939/Graphics-Drivers
AMD: https://www.amd.com/en/support
NVIDIA: https://www.nvidia.com/Download/index.aspx

🌏 Other Open-Source Code Used

https://github.com/Tencent/ncnn for fast neural network inference on ALL PLATFORMS
https://github.com/webmproject/libwebp for encoding and decoding Webp images on ALL PLATFORMS
https://github.com/nothings/stb for decoding and encoding image on Linux / MacOS
https://github.com/tronkko/dirent for listing files in directory on Windows

📜 BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

📧 Contact

If you have any question, please email [email protected] or [email protected].

Comments

problem running on aws

I downloaded the ubuntu zip on a g3s.xlarge and the result is a black image. Is the zip missing files?


./realesrgan-ncnn-vulkan-v0.2.0-ubuntu/realesrgan-ncnn-vulkan -i input.jpg -o out/output.jpg -n realesrgan-x4plus -s 4 

[0 Tesla M60]  queueC=0[16]  queueG=0[16]  queueT=1[2]
[0 Tesla M60]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[0 Tesla M60]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
[0 Tesla M60]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
[1 llvmpipe (LLVM 12.0.0, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
[1 llvmpipe (LLVM 12.0.0, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
[1 llvmpipe (LLVM 12.0.0, 256 bits)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/0
[1 llvmpipe (LLVM 12.0.0, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=0
fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.param failed
fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.bin failed
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
0.00%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
4.17%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
8.33%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
12.50%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
16.67%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
20.83%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
25.00%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
29.17%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
33.33%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
37.50%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
41.67%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
45.83%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
50.00%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
54.17%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
58.33%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
62.50%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
66.67%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
70.83%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
75.00%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
79.17%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
83.33%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
87.50%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
91.67%
find_blob_index_by_name data failed
Try
find_blob_index_by_name output failed
Try
95.83%

opened by kmulvey 2

How do I switch from integrated gpu to nvedia gpu

[0 Intel(R) UHD Graphics 630] queueC=0[1] queueG=0[1] queueT=0[1] [0 Intel(R) UHD Graphics 630] bugsbn1=0 bugbilz=3 bugcopc=0 bugihfa=0 [0 Intel(R) UHD Graphics 630] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Intel(R) UHD Graphics 630] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1

When upscaling images it only uses integrated gpu I want to use my gtx 1650 how do I switch this?

opened by csAshish 0
in directory mode, option to skip if destination already exist

if you stop the upscaler while processing a directory and you have to restart it, it will start from the beginning and overwrite existing files

overwriting in file mode is fine but should be an option for direcotry mode ... so I propose a flag to disable overwrite and skip if exist

opened by 6543 0
models: consider adding into README how to get them.
Hey,

Please consider adding to README a section how to get the pre-trained model files. Currently you'd need to extract them from the following links

https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-ubuntu.zip

https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-animevideov3.pth

https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth

It took me a moment to figure out how to get a hold of them and I suspect many others would face similar challenge.
opened by slashbeast 0

Segfault realesrnet-x4plus

%  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n

opened by atomical 1

Releases(v0.2.0)

Owner

Xintao

Researcher at Tencent ARC Lab, (Applied Research Center)

GitHub Repository https://github.com/xinntao/Real-ESRGAN

PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021.

GCResNet PyTorch implementation of Graph Convolutional Networks in Feature Space for Image Deblurring and Super-resolution, IJCNN 2021. The code will

11 May 19, 2022

Project page for our ICCV 2021 paper "The Way to my Heart is through Contrastive Learning"

The Way to my Heart is through Contrastive Learning: Remote Photoplethysmography from Unlabelled Video This is the official project page of our ICCV 2

36 Jan 06, 2023

Music Generation using Neural Networks Streamlit App

Music_Gen_Streamlit "Music Generation using Neural Networks" Streamlit App TO DO: Make a run_app.sh Introduction [~5 min] (Sohaib) Team Member names/i

6 Aug 09, 2022

The repository forked from NVlabs uses our data. (Differentiable rasterization applied to 3D model simplification tasks)

nvdiffmodeling [origin_code] Differentiable rasterization applied to 3D model simplification tasks, as described in the paper: Appearance-Driven Autom

2 Oct 31, 2022

3D Human Pose Machines with Self-supervised Learning

3D Human Pose Machines with Self-supervised Learning Keze Wang, Liang Lin, Chenhan Jiang, Chen Qian, and Pengxu Wei, “3D Human Pose Machines with Self

398 Dec 20, 2022

Pytorch implementation of Deep Recursive Residual Network for Super Resolution (DRRN)

DRRN-pytorch This is an unofficial implementation of "Deep Recursive Residual Network for Super Resolution (DRRN)", CVPR 2017 in Pytorch. [Paper] You

192 Dec 12, 2022

Bayesian Optimization Library for Medical Image Segmentation.

bayesmedaug: Bayesian Optimization Library for Medical Image Segmentation. bayesmedaug optimizes your data augmentation hyperparameters for medical im

7 Feb 10, 2022

Efficient Sparse Attacks on Videos using Reinforcement Learning

EARL This repository provides a simple implementation of the work "Efficient Sparse Attacks on Videos using Reinforcement Learning" Example: Demo: Her

12 Dec 05, 2021

This repository holds code and data for our PETS'22 article 'From "Onion Not Found" to Guard Discovery'.

From "Onion Not Found" to Guard Discovery (PETS'22) This repository holds the code and data for our PETS'22 paper titled 'From "Onion Not Found" to Gu

3 May 04, 2022

Classifies galaxy morphology with Bayesian CNN

Zoobot Zoobot classifies galaxy morphology with deep learning. This code will let you: Reproduce and improve the Galaxy Zoo DECaLS automated classific

39 Dec 20, 2022

Chainer Implementation of Semantic Segmentation using Adversarial Networks

Semantic Segmentation using Adversarial Networks Requirements Chainer (1.23.0) Differences Use of FCN-VGG16 instead of Dilated8 as Segmentor. Caution

99 Jun 28, 2022

a baseline to practice

ccks2021_track3_baseline a baseline to practice 路径可能会有问题，自己改改 torch==1.7.1 pyhton==3.7.1 transformers==4.7.0 cuda==11.0 this is a baseline, you can fi

45 Nov 23, 2022

Rainbow: Combining Improvements in Deep Reinforcement Learning

Rainbow Rainbow: Combining Improvements in Deep Reinforcement Learning [1]. Results and pretrained models can be found in the releases. DQN [2] Double

1.4k Dec 29, 2022

A containerized REST API around OpenAI's CLIP model.

OpenAI's CLIP — REST API This is a container wrapping OpenAI's CLIP model in a RESTful interface. Running the container locally First, build the conta

48 Nov 06, 2022

Auto-Encoding Score Distribution Regression for Action Quality Assessment

DAE-AQA It is an open source program reference to paper Auto-Encoding Score Distribution Regression for Action Quality Assessment. 1.Introduction DAE

13 Nov 16, 2022

Code image classification of MNIST dataset using different architectures: simple linear NN, autoencoder, and highway network

Deep Learning for image classification pip install -r http://webia.lip6.fr/~baskiotisn/requirements-amal.txt Train an autoencoder python3 train_auto

0 Mar 30, 2022

SuperSonic, a new open-source framework to allow compiler developers to integrate RL into compilers easily, regardless of their RL expertise

Automating reinforcement learning architecture design for code optimization. Che

113 Nov 28, 2022