Semantic Segmentation in Pytorch. Network include: FCN、FCN_ResNet、SegNet、UNet、BiSeNet、BiSeNetV2、PSPNet、DeepLabv3_plus、 HRNet、DDRNet

Overview

🚀 If it helps you, click a star!

Update log

  • 2020.12.10 Project structure adjustment, the previous code has been deleted, the adjustment will be re-uploaded code
  • 2021.04.09 Re-upload the code, "V1 Commit"
  • 2021.04.22 update torch distributed training
  • Ongoing update .....

1. Display (Cityscapes)

  • Using model DDRNet 1525 test sets, official MIOU =78.4069%
Average results
Class results1
Class results2
Class results3
  • Comparison of the original and predicted images
origin
label
predict

2. Install

pip install -r requirements.txt
Experimental environment:

  • Ubuntu 16.04 Nvidia-Cards >= 1
  • python==3.6.5
  • See Dependency Installation Package for details in requirement.txt

3. Model

All the modeling is done in builders/model_builder.py

  • FCN
  • FCN_ResNet
  • SegNet
  • UNet
  • BiSeNet
  • BiSeNetV2
  • PSPNet
  • DeepLabv3_plus
  • HRNet
  • DDRNet
Model Backbone Val mIoU Test mIoU Imagenet Pretrain Pretrained Model
PSPNet ResNet 50 76.54% - PSPNet
DeeplabV3+ ResNet 50 77.78% - DeeplabV3+
DDRNet23_slim - DDRNet23_slim_imagenet
DDRNet23 - DDRNet23_imagenet
DDRNet39 - 79.63% - DDRNet39_imagenet DDRNet39
Updating more model.......

4. Data preprocessing

This project enables you to expose data sets: CityscapesISPRS
The data set is uploaded later .....
Cityscapes data set preparation is shown here:

4.1 Download the dataset

Download the dataset from the link on the website, You can get *leftImg8bit.png suffix of original image under folder leftImg8bit, a) *color.pngb) *labelIds.pngc) *instanceIds.png suffix of fine labeled image under folder gtFine.

*leftImg8bit.png          : the origin picture
a) *color.png             : the class is encoded by its color
b) *labelIds.png          : the class is encoded by its ID
c) *instanceIds.png       : the class and the instance are encoded by an instance ID

4.2 Onehot encoding of label image

The real label gray scale image Onehot encoding used by the semantic segmentation task is 0-18, so the label needs to be encoded. Using scripts dataset/cityscapes/cityscapes_scripts/process_cityscapes.py to process the image and get the result *labelTrainIds.png. process_cityscapes.py usage: Modify 486 lines `Cityscapes_path'is the path to store your own data.

  • Comparison of original image, color label image and gray label image (0-18)
***_leftImg8bit
***_gtFine_color
***_gtFine_labelTrainIds
  • Local storage path display /data/open_data/cityscapes/:
data
  |--open_data
        |--cityscapes
               |--leftImg8bit
                    |--train
                        |--cologne
                        |--*******
                    |--val
                        |--*******
                    |--test
                        |--*******
               |--gtFine
                    |--train
                        |--cologne
                        |--*******
                    |--val
                        |--*******
                    |--test
                        |--*******

4.3 Generate image path

  • Generate a txt containing the image path
    Use script dataset/generate_txt.py to generate the path txt file containing the original image and labels. A total of 3 txt files will be generated: cityscapes_train_list.txtcityscapes_val_list.txtcityscapes_test_list.txt, and copy the three files to the dataset root directory.
data
  |--open_data
        |--cityscapes
               |--cityscapes_train_list.txt
               |--cityscapes_val_list.txt
               |--cityscapes_test_list.txt
               |--leftImg8bit
                    |--train
                        |--cologne
                        |--*******
                    |--val
                        |--*******
                    |--test
                        |--*******
               |--gtFine
                    |--train
                        |--cologne
                        |--*******
                    |--val
                        |--*******
                    |--test
                        |--*******
  • The contents of the txt are shown as follows:
leftImg8bit/train/cologne/cologne_000000_000019_leftImg8bit.png gtFine/train/cologne/cologne_000000_000019_gtFine_labelTrainIds.png
leftImg8bit/train/cologne/cologne_000001_000019_leftImg8bit.png gtFine/train/cologne/cologne_000001_000019_gtFine_labelTrainIds.png
..............
  • The format of the txt are shown as follows:
origin image path + the separator '\t' + label path +  the separator '\n'

TODO.....

5. How to train

sh train.sh

5.1 Parameters

python -m torch.distributed.launch --nproc_per_node=2 \
                train.py --model PSPNet_res50 --out_stride 8 \
                --max_epochs 200 --val_epochs 20 --batch_size 4 --lr 0.01 --optim sgd --loss ProbOhemCrossEntropy2d \
                --base_size 768 --crop_size 768  --tile_hw_size 768,768 \
                --root '/data/open_data' --dataset cityscapes --gpus_id 1,2

6. How to validate

sh predict.sh
Comments
  • size doesn't match error when run the train.py

    size doesn't match error when run the train.py

    sorry to disturb you, when i run train.py, it goes wrong, and i print the size of the two tensor, there are same, i can't find the resolution, how to solve it? `******* Begining traing *******


    Epoch 0/300: 0%| | 0/909 [00:00<?, ?it/s]/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py:1350: UserWarning: nn.functional.sigmoid is deprecated. Use torch.sigmoid instead. warnings.warn("nn.functional.sigmoid is deprecated. Use torch.sigmoid instead.") Traceback (most recent call last): File "/home/yeluyue/dl/bottle_Segmentation/train.py", line 415, in train_model(args) File "/home/yeluyue/dl/bottle_Segmentation/train.py", line 293, in train_model lossTr, lr = train(args, trainLoader, model, criteria, optimizer, epoch) File "/home/yeluyue/dl/bottle_Segmentation/train.py", line 117, in train loss = criterion(output, labels) File "/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/home/yeluyue/dl/bottle_Segmentation/tools/loss.py", line 27, in forward return self.loss(outputs, targets) File "/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call result = self.forward(*input, **kwargs) File "/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/modules/loss.py", line 498, in forward return F.binary_cross_entropy(input, target, weight=self.weight, reduction=self.reduction) File "/usr/local/anaconda3/lib/python3.6/site-packages/torch/nn/functional.py", line 2047, in binary_cross_entropy new_size = _infer_size(target.size(), weight.size()) RuntimeError: The size of tensor a (512) must match the size of tensor b (2) at non-singleton dimension 2`

    opened by lj107024 4
  • AttributeError in HRNet

    AttributeError in HRNet

    When I run ./model/HRNet.py on Ubuntu 18.04, torch 1.8.0+cu111, the error raise as follows,

    /home/vgc/users/lwz/code/rice_seg/template/Segmentation-Pytorch/model/HRNet.py:329: DeprecationWarning: np.int is a deprecated alias for the builtin int. To silence this warning, use int by itself. Doing this will not modify any behavior and is safe. When replacing np.int, you may wish to use e.g. np.int64 or np.int32 to specify the precision. If you wish to review your current use, check the release note link for additional information. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations last_inp_channels = np.int(np.sum(pre_stage_channels)) Traceback (most recent call last): File "/home/vgc/users/lwz/code/rice_seg/template/Segmentation-Pytorch/model/HRNet.py", line 520, in summary(model, (3, 512, 512), device="cpu") File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torchsummary/torchsummary.py", line 72, in summary model(*x) File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/vgc/users/lwz/code/rice_seg/template/Segmentation-Pytorch/model/HRNet.py", line 447, in forward y_list = self.stage2(x_list) File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 889, in _call_impl result = self.forward(*input, **kwargs) File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torch/nn/modules/container.py", line 119, in forward input = module(input) File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torch/nn/modules/module.py", line 893, in _call_impl hook_result = hook(self, input, result) File "/home/vgc/anaconda3/envs/lwz37/lib/python3.7/site-packages/torchsummary/torchsummary.py", line 19, in hook summary[m_key]["input_shape"] = list(input[0].size()) AttributeError: 'list' object has no attribute 'size'

    opened by rrryan2016 2
  • bug about generate_txt.py

    bug about generate_txt.py

    in line 26 filename_gt = filename.replace('leftImg8bit', 'gtFine') should change to filename_gt = filename.replace('leftImg8bit', 'gtFine').replace('.png','_labelTrainIds.png') to produce the format of leftImg8bit/train/cologne/cologne_000001_000019_leftImg8bit.png gtFine/train/cologne/cologne_000001_000019_gtFine_labelTrainIds.png

    the result will be leftImg8bit/train/cologne/cologne_000001_000019_leftImg8bit.png gtFine/train/cologne/cologne_000001_000019_gtFine.png before. And the program wont find the file.

    if I am wrong, plz tell me. Best wishes.

    opened by songzijiang 2
  • 使用ENet模型,在train时正常,在pridict时会出现超出内存。

    使用ENet模型,在train时正常,在pridict时会出现超出内存。

    在train时正常,在pridict时会出现: RuntimeError.CUDA out of memory. Tried to allocate 188.00 MiB (GPU 0; 6.00 GiB total capacity; 4.21 GiB already allocated; 63.85 MiB free; 81.86 MiB cached) 使用predict_sliding时会出现: 发生异常: TypeError init() got an unexpected keyword argument 'std' 请问怎么解决?

    opened by FelixJiao 2
  • Bump opencv-python from 4.1.0.25 to 4.2.0.32

    Bump opencv-python from 4.1.0.25 to 4.2.0.32

    Bumps opencv-python from 4.1.0.25 to 4.2.0.32.

    Release notes

    Sourced from opencv-python's releases.

    4.2.0.32

    OpenCV version 4.2.0.

    Changes:

    • macOS environment updated from xcode8.3 to xcode 9.4
    • macOS uses now Qt 5 instead of Qt 4
    • Nasm version updated to Docker containers
    • multibuild updated

    Fixes:

    • don't use deprecated brew tap-pin, instead refer to the full package name when installing #267
    • replace get_config_var() with get_config_vars() in setup.py #274
    • add workaround for DLL errors in Windows Server #264

    4.1.2.30

    OpenCV version 4.1.2.

    Changes:

    • Python 3.8 builds added to the build matrix
    • Support for Python 3.4 builds dropped (Python 3.4 is in EOL)
    • multibuild updated
    • minor build logic changes
    • Docker images rebuilt

    Notes:

    Please note that Python 2.7 enters into EOL phase in January 2020. opencv-python Python 2.7 wheels won't be provided after that.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump opencv-python from 4.1.0.25 to 4.1.1.26

    Bump opencv-python from 4.1.0.25 to 4.1.1.26

    Bumps opencv-python from 4.1.0.25 to 4.1.1.26.

    Release notes

    Sourced from opencv-python's releases.

    4.1.1.26

    OpenCV version 4.1.1.

    Changes:

    • FFmpeg has been compiled with https support on Linux builds #229
    • CI build logic related changes #197, #227, #228
    • Custom libjepg-turbo removed because it's provided by OpenCV #231
    • 64-bit Qt builds are now smaller #236
    • Custom builds should be now rather easy to do locally #235:
      1. Clone this repository
      2. Optional: set up ENABLE_CONTRIB and ENABLE_HEADLESS environment variables to 1 if needed
      3. Optional: add additional Cmake arguments to CMAKE_ARGS environment variable
      4. Run python setup.py bdist_wheel
    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 1
  • Bump numpy from 1.15.1 to 1.22.0

    Bump numpy from 1.15.1 to 1.22.0

    Bumps numpy from 1.15.1 to 1.22.0.

    Release notes

    Sourced from numpy's releases.

    v1.22.0

    NumPy 1.22.0 Release Notes

    NumPy 1.22.0 is a big release featuring the work of 153 contributors spread over 609 pull requests. There have been many improvements, highlights are:

    • Annotations of the main namespace are essentially complete. Upstream is a moving target, so there will likely be further improvements, but the major work is done. This is probably the most user visible enhancement in this release.
    • A preliminary version of the proposed Array-API is provided. This is a step in creating a standard collection of functions that can be used across application such as CuPy and JAX.
    • NumPy now has a DLPack backend. DLPack provides a common interchange format for array (tensor) data.
    • New methods for quantile, percentile, and related functions. The new methods provide a complete set of the methods commonly found in the literature.
    • A new configurable allocator for use by downstream projects.

    These are in addition to the ongoing work to provide SIMD support for commonly used functions, improvements to F2PY, and better documentation.

    The Python versions supported in this release are 3.8-3.10, Python 3.7 has been dropped. Note that 32 bit wheels are only provided for Python 3.8 and 3.9 on Windows, all other wheels are 64 bits on account of Ubuntu, Fedora, and other Linux distributions dropping 32 bit support. All 64 bit wheels are also linked with 64 bit integer OpenBLAS, which should fix the occasional problems encountered by folks using truly huge arrays.

    Expired deprecations

    Deprecated numeric style dtype strings have been removed

    Using the strings "Bytes0", "Datetime64", "Str0", "Uint32", and "Uint64" as a dtype will now raise a TypeError.

    (gh-19539)

    Expired deprecations for loads, ndfromtxt, and mafromtxt in npyio

    numpy.loads was deprecated in v1.15, with the recommendation that users use pickle.loads instead. ndfromtxt and mafromtxt were both deprecated in v1.17 - users should use numpy.genfromtxt instead with the appropriate value for the usemask parameter.

    (gh-19615)

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
Releases(v1.0.0)
Owner
Deeachain
Graduate students from outer space
Deeachain
RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation

RL-GAN: Transfer Learning for Related Reinforcement Learning Tasks via Image-to-Image Translation RL-GAN is an official implementation of the paper: T

42 Nov 10, 2022
An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

An official PyTorch Implementation of Boundary-aware Self-supervised Learning for Video Scene Segmentation (BaSSL)

Kakao Brain 72 Dec 28, 2022
AugLiChem - The augmentation library for chemical systems.

AugLiChem Welcome to AugLiChem! The augmentation library for chemical systems. This package supports augmentation for both crystaline and molecular sy

BaratiLab 17 Jan 08, 2023
The code for paper "Learning Implicit Fields for Generative Shape Modeling".

implicit-decoder The tensorflow code for paper "Learning Implicit Fields for Generative Shape Modeling", Zhiqin Chen, Hao (Richard) Zhang. Project pag

Zhiqin Chen 353 Dec 30, 2022
Apply our monocular depth boosting to your own network!

MergeNet - Boost Your Own Depth Boost custom or edited monocular depth maps using MergeNet Input Original result After manual editing of base You can

Computational Photography Lab @ SFU 142 Dec 17, 2022
This code is 3d-CNN model that can predict environmental value

Predict-environmental-value-3dCNN This code is 3d-CNN model that can predict environmental value. Firstly, I built a model that can create a lot of bu

1 Jan 06, 2022
Data-driven reduced order modeling for nonlinear dynamical systems

SSMLearn Data-driven Reduced Order Models for Nonlinear Dynamical Systems This package perform data-driven identification of reduced order model based

Haller Group, Nonlinear Dynamics 27 Dec 13, 2022
Computing Shapley values using VAEAC

Shapley values and the VAEAC method In this GitHub repository, we present the implementation of the VAEAC approach from our paper "Using Shapley Value

3 Nov 23, 2022
Repo for CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning

CReST in Tensorflow 2 Code for the paper: "CReST: A Class-Rebalancing Self-Training Framework for Imbalanced Semi-Supervised Learning" by Chen Wei, Ki

Google Research 75 Nov 01, 2022
PyTorch implementation of Tacotron speech synthesis model.

tacotron_pytorch PyTorch implementation of Tacotron speech synthesis model. Inspired from keithito/tacotron. Currently not as much good speech quality

Ryuichi Yamamoto 279 Dec 09, 2022
Can we do Customers Segmentation using PHP and Unsupervized Machine Learning ? Yes we can ! 🤡

Customers Segmentation using PHP and Rubix ML PHP Library Can we do Customers Segmentation using PHP and Unsupervized Machine Learning ? Yes we can !

Mickaël Andrieu 11 Oct 08, 2022
Pytorch implementation of the Variational Recurrent Neural Network (VRNN).

VariationalRecurrentNeuralNetwork Pytorch implementation of the Variational RNN (VRNN), from A Recurrent Latent Variable Model for Sequential Data. Th

emmanuel 251 Dec 17, 2022
ML-Ensemble – high performance ensemble learning

A Python library for high performance ensemble learning ML-Ensemble combines a Scikit-learn high-level API with a low-level computational graph framew

Sebastian Flennerhag 764 Dec 31, 2022
Code Repository for Liquid Time-Constant Networks (LTCs)

Liquid time-constant Networks (LTCs) [Update] A Pytorch version is added in our sister repository: https://github.com/mlech26l/keras-ncp This is the o

Ramin Hasani 553 Dec 27, 2022
We are More than Our JOints: Predicting How 3D Bodies Move

We are More than Our JOints: Predicting How 3D Bodies Move Citation This repo contains the official implementation of our paper MOJO: @inproceedings{Z

72 Oct 20, 2022
Rethinking Transformer-based Set Prediction for Object Detection

Rethinking Transformer-based Set Prediction for Object Detection Here are the code for the ICCV paper. The code is adapted from Detectron2 and AdelaiD

Zhiqing Sun 62 Dec 03, 2022
Adversarial Graph Augmentation to Improve Graph Contrastive Learning

ADGCL : Adversarial Graph Augmentation to Improve Graph Contrastive Learning Introduction This repo contains the Pytorch [1] implementation of Adversa

susheel suresh 62 Nov 19, 2022
TAug :: Time Series Data Augmentation using Deep Generative Models

TAug :: Time Series Data Augmentation using Deep Generative Models Note!!! The package is under development so be careful for using in production! Fea

35 Dec 06, 2022
code from "Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity"

Code associated with the paper "Tensor decomposition of higher-order correlations by nonlinear Hebbian learning," Ocker & Buice, Neurips 2021. "plot_f

Gabriel Koch Ocker 4 Oct 16, 2022
This is the pytorch code for the paper Curious Representation Learning for Embodied Intelligence.

Curious Representation Learning for Embodied Intelligence This is the pytorch code for the paper Curious Representation Learning for Embodied Intellig

19 Oct 19, 2022