Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Overview

Deep Learning Dataset Maker

Python 3.8 QGIS 3.16.13

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

image

How to use

  1. Download and install QGIS and clone the repo :
git clone [email protected]:deepbands/deep-learning-datasets-maker.git
  1. Copy folder named deep-learning-datasets-maker in QGIS configuration folder and choose the plugin from plugin manager in QGIS (If not appeared restart QGIS).

    • You can know this folder from QGIS Setting Menu at the top-left of QGIS UI Settings > User Profiles > Open Active Profile Folder .
    • Go to python/plugins then paste the deep-learning-datasets-maker folder.
    • Full path should be like : C:\Users\$USER\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\deep-learning-datasets-maker.
  2. Open QGIS, load your raster and vector data then select the output paths for rasterized, images and labels then click ok.

TODO

  • Fix: If vector layer saved in memory not in file, rasterize can't work.
  • Fix: Splitiing Image Size.
Comments
  • Error running Quick Queries with QuickOSM recently started on Mac

    Error running Quick Queries with QuickOSM recently started on Mac

    I have been trying to run a query that I have previously had no problem running at all and have run into this error every time. Here are my system specs and whatnot:

    QGIS version | 3.22.4-Białowieża QGIS code revision 3f4577ce6e Qt version | 5.14.2 Python version | 3.8.7 GDAL/OGR version | 3.2.1 PROJ version | 6.3.2 EPSG Registry database version | v9.8.6 (2020-01-22) GEOS version | 3.9.1-CAPI-1.14.2 SQLite version | 3.31.1 PostgreSQL client version | 12.3 SpatiaLite version | 4.3.0a QWT version | 6.1.4 QScintilla2 version | 2.11.4 OS version | macOS 11.0

    Active Python plugins DigitizingTools | 1.5.1 QuickOSM | 2.0.1 quick_map_services | 0.19.29 usgs_stream_mapper | 0.2 sagaprovider | 2.12.99 grassprovider | 2.12.99 db_manager | 0.1.20 MetaSearch | 0.3.5

    And this is the dialogue from the QuickOSM Panel

    2022-03-19T18:47:06     INFO    All OSM objects with the key 'natural'='water' in the canvas or layer extent are going to be downloaded.
    2022-03-19T18:47:06     INFO    Query: natural_water
    2022-03-19T18:47:06     INFO    Encoded URL: https://lz4.overpass-api.de/api/interpreter?data=[out:xml] [timeout:25];%0A(%0A node[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A way[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A relation[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A);%0A(._;%3E;);%0Aout body;&info=QgisQuickOSMPlugin
    2022-03-19T18:47:09     INFO    Request completed
    2022-03-19T18:47:09     INFO    Checking OSM file content /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     INFO    The OSM file is: /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     CRITICAL   A critical error occurred, this is the traceback:
    2022-03-19T18:47:09     CRITICAL    
    2022-03-19T18:47:09     CRITICAL    base_processing_panel.py
    2022-03-19T18:47:09     CRITICAL    Error: Algorithm qgis:checkvalidity not found
                 
    2022-03-19T18:47:09     CRITICAL     File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/base_processing_panel.py", line 47, in run
                  self._run()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/quick_query_panel.py", line 423, in _run
                  num_layers = process_quick_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 318, in process_quick_query
                  return process_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 257, in process_query
                  return open_file(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 106, in open_file
                  layers = osm_parser.processing_parse()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/parser/osm_parser.py", line 133, in processing_parse
                  validity = processing.run(
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py", line 108, in run
                  return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py", line 171, in runAlgorithm
                  raise QgsProcessingException(msg)
    

    Finally: The Processing panel shows this dialogue each time I try a query.

    Error: Algorithm qgis:checkvalidity not found

    I have made sure my language is american english and that the startup style of the app is Macintosh. If anyone could please please help me I would be so appreciative. This function is so important to the research I am doing for my senior thesis. Thank you so much for taking time to read this.

    opened by cecilia-hampton 1
  • Init create COCO

    Init create COCO

    @Youssef-Harby The function is implemented, please check and use it. If you can, it can be merged into develop. But at present, the cutting image is repeated and can be modified 😃

    opened by geoyee 1
  • Error running in QGIS

    Error running in QGIS

    Using on a tiff image with 4 bands RGB and alpha QGIS 3.22.4

    2022-03-10T15:24:22 WARNING Traceback (most recent call last): File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 315, in run rasterize(ras_path, vec_path, output) File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\rasterize.py", line 11, in rasterize lyr = vec_ds.GetLayer() AttributeError: 'NoneType' object has no attribute 'GetLayer'

    opened by Chris-Has 1
  • Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1. Traceback (most recent call last): File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 10, in from pycocotools import mask File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ModuleNotFoundError: No module named 'pycocotools'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker_init_.py", line 35, in classFactory from .split_rs_data import SplitRSData File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 46, in from .utils.COCO import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO_init.py", line 1, in from .shape_to_coco import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\shape_to_coco.py", line 16, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools_init.py", line 1, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 16, in ["python3", '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']) File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1.

    Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] QGIS version: 3.4.13-Madeira Madeira, 567300ccf1

    Python Path: C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins C:\Users\pprt\matterport\models\research C:\Users\pprt\matterport\models\research\slim C:\Program Files\QGIS 3.4\bin\python37.zip C:\PROGRA~1\QGIS3~1.4\apps\Python37\DLLs C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib C:\Program Files\QGIS 3.4\bin C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages\tf_unet-0.1.2-py3.7.egg C:\PROGRA~1\QGIS3~1.4\apps\Python37 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32\lib C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\Pythonwin C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python

    enhancement 
    opened by aldinorizaldy 3
  • An error occurred during execution of following code

    An error occurred during execution of following code

    An error occurred during execution of following code: pyplugin_installer.instance().installPlugin('deep-learning-datasets-maker', stable=False)

    Traceback (most recent call last): File "", line 1, in File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 333, in installPlugin self.processDependencies(plugin["id"]) File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 682, in processDependencies dlg = QgsPluginDependenciesDialog(plugin_id, to_install, to_upgrade, not_found) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 92, in init _make_row(data, i, name) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 63, in _make_row widget.use_stable_version = data['use_stable_version'] KeyError: 'use_stable_version'

    Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]

    QGIS version: 3.22.3-Białowieża 'Białowieża', 1628765ec7

    Python path: ['/usr/share/qgis/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins/DeepLearningTools']

    bug 
    opened by makamkkumar 1
Releases(0.2.1)
Owner
deepbands
deepbands
Code for the paper "Functional Regularization for Reinforcement Learning via Learned Fourier Features"

Reinforcement Learning with Learned Fourier Features State-space Soft Actor-Critic Experiments Move to the state-SAC-LFF repository. cd state-SAC-LFF

Alex Li 10 Nov 11, 2022
retweet 4 satoshi ⚡️

rt4sat retweet 4 satoshi This bot is the codebase for https://twitter.com/rt4sat please feel free to create an issue if you saw any bugs basically thi

6 Sep 30, 2022
Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition

GCN_LogsigRNN This repository holds the codebase for the paper: Logsig-RNN: a novel network for robust and efficient skeleton-based action recognition

7 Oct 14, 2022
Multi-Task Learning as a Bargaining Game

Nash-MTL Official implementation of "Multi-Task Learning as a Bargaining Game". Setup environment conda create -n nashmtl python=3.9.7 conda activate

Aviv Navon 87 Dec 26, 2022
This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting.

GAN Memory for Lifelong learning This is a pytorch implementation of the NeurIPS paper GAN Memory with No Forgetting. Please consider citing our paper

Miaoyun Zhao 43 Dec 27, 2022
Table-Extractor 表格抽取

(t)able-(ex)tractor 本项目旨在实现pdf表格抽取。 Models 版面分析模块(Yolo) 表格结构抽取(ResNet + Transformer) 文字识别模块(CRNN + CTC Loss) Acknowledgements TableMaster attention-i

2 Jan 15, 2022
Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks

Self-Correcting Quantum Many-Body Control using Reinforcement Learning with Tensor Networks This repository contains the code and data for the corresp

Friederike Metz 7 Apr 23, 2022
Source code for paper "Deep Superpixel-based Network for Blind Image Quality Assessment"

DSN-IQA Source code for paper "Deep Superpixel-based Network for Blind Image Quality Assessment" Requirements Python =3.8.0 Pytorch =1.7.1 Usage wit

7 Oct 13, 2022
Some pvbatch (paraview) scripts for postprocessing OpenFOAM data

pvbatchForFoam Some pvbatch (paraview) scripts for postprocessing OpenFOAM data For every script there is a help message available: pvbatch pv_state_s

Morev Ilya 2 Oct 26, 2022
Visualizing Yolov5's layers using GradCam

YOLO-V5 GRADCAM I constantly desired to know to which part of an object the object-detection models pay more attention. So I searched for it, but I di

Pooya Mohammadi Kazaj 200 Jan 01, 2023
Differentiable Quantum Chemistry (only Differentiable Density Functional Theory and Hartree Fock at the moment)

DQC: Differentiable Quantum Chemistry Differentiable quantum chemistry package. Currently only support differentiable density functional theory (DFT)

75 Dec 02, 2022
git《Tangent Space Backpropogation for 3D Transformation Groups》(CVPR 2021) GitHub:1]

LieTorch: Tangent Space Backpropagation Introduction The LieTorch library generalizes PyTorch to 3D transformation groups. Just as torch.Tensor is a m

Princeton Vision & Learning Lab 482 Jan 06, 2023
Human Detection - Pedestrian Detection using OpenCV Python

Pedestrian Detection using OpenCV Python Follow us on Instagram for Machine Lear

Hrishikesh Dutta 1 Jan 23, 2022
High-Resolution Image Synthesis with Latent Diffusion Models

Latent Diffusion Models arXiv | BibTeX High-Resolution Image Synthesis with Latent Diffusion Models Robin Rombach*, Andreas Blattmann*, Dominik Lorenz

CompVis Heidelberg 5.6k Dec 30, 2022
Which Style Makes Me Attractive? Interpretable Control Discovery and Counterfactual Explanation on StyleGAN

Interpretable Control Exploration and Counterfactual Explanation (ICE) on StyleGAN Which Style Makes Me Attractive? Interpretable Control Discovery an

Bo Li 11 Dec 01, 2022
TRIQ implementation

TRIQ Implementation TF-Keras implementation of TRIQ as described in Transformer for Image Quality Assessment. Installation Clone this repository. Inst

Junyong You 115 Dec 30, 2022
This game was designed to encourage young people not to gamble on lotteries, as the probablity of correctly guessing the number is infinitesimal!

Lottery Simulator 2022 for Web Launch Application Developed by John Seong in Ontario. This game was designed to encourage young people not to gamble o

John Seong 2 Sep 02, 2022
Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation

Implicit Internal Video Inpainting Implementation for our ICCV2021 paper: Internal Video Inpainting by Implicit Long-range Propagation paper | project

202 Dec 30, 2022
Python package to generate image embeddings with CLIP without PyTorch/TensorFlow

imgbeddings A Python package to generate embedding vectors from images, using OpenAI's robust CLIP model via Hugging Face transformers. These image em

Max Woolf 81 Jan 04, 2023
Teaching end to end workflow of deep learning

Deep-Education This repository is now available for public use for teaching end to end workflow of deep learning. This implies that learners/researche

Data Lab at College of William and Mary 2 Sep 26, 2022