TRACER: Extreme Attention Guided Salient Object Tracing Network implementation in PyTorch

Related tags

Deep LearningTRACER
Overview

TRACER: Extreme Attention Guided Salient Object Tracing Network

This paper was accepted at AAAI 2022 SA poster session.

PWC
PWC
PWC
PWC
PWC

Datasets

All datasets are available in public.

  • Download the DUTS-TR and DUTS-TE from Here
  • Download the DUT-OMRON from Here
  • Download the HKU-IS from Here
  • Download the ECSSD from Here
  • Download the PASCAL-S from Here
  • Download the edge GT from Here.

Data structure


TRACER
├── data
│   ├── DUTS
│   │   ├── Train
│   │   │   ├── images
│   │   │   ├── masks
│   │   │   ├── edges
│   │   ├── Test
│   │   │   ├── images
│   │   │   ├── masks
│   ├── DUT-O
│   │   ├── Test
│   │   │   ├── images
│   │   │   ├── masks
│   ├── HKU-IS
│   │   ├── Test
│   │   │   ├── images
│   │   │   ├── masks
      .
      .
      .

Requirements

  • Python 3.7.x
  • Pytorch >= 1.8.0
  • albumentations >= 0.5.1
  • matplotlib >= 3.3.3
  • tqdm >=4.54.0
  • scikit-learn > 0.23.2

Run

  • Run main.py scripts.

# For training TRACER-TE0 (e.g.)
python main.py train --arch 0 --img_size 320

# For testing TRACER with pre-trained model (e.g.)  
python main.py test --exp_num 0 --arch 0 --img_size 320

Configurations

--img_size: Input image resolution.
--arch: EfficientNet backbone scale: TE0 to TE7.
--frequency_radius: High-pass filter radius in the MEAM.
--gamma: channel confidence ratio \gamma in the UAM.
--denoise: Denoising ratio d in the OAM.
--RFB_aggregated_channel: # of channels in receptive field blocks.
--multi_gpu: Multi-GPU learning options.

Comments
  • Adaptive Pixel Intensity Loss generated NaN values while training

    Adaptive Pixel Intensity Loss generated NaN values while training

    Was training on custom human dataset. Batch Size = 8 No of training images = 3800

    No of steps trained before showing error = 75

    After 75th step It generated an error:

    RuntimeError: Function 'UpsampleBilinear2DBackward1' returned nan values in its 0th output.
    
    

    The model trained successfully when using BCE loss.

    We even checked for NaN values using torch.autograd.set_detect_anamoly(True) But it returned False stating that no NaN values were found

    opened by ThiruRJST 10
  • API_loss for multi_classes semantic segmentation

    API_loss for multi_classes semantic segmentation

    Hello,

    I would like to apply your API_loss to my 3_classes semantic segmentation task, but i am not sure it will be work or not after some modification? Do you have any advice? Look forward to your reply~

    Thanks, best wish!

    opened by DAgalaxy 6
  • Test on my data

    Test on my data

    Thanks for your great work! How can I use the trained model to test when I got the best model after training on my data? Because it seems that the pre training model is still used in the inference stage?

    opened by LitterWindwind 5
  • Reproduce you result on TE1. The MAE is much worse than yours on DUTS-TE

    Reproduce you result on TE1. The MAE is much worse than yours on DUTS-TE

    Hi, I tried your code to reproduce your result. I used python main.py train --arch 1 --img_size 320 to train the model. I did not change any setting in your code. I used four GPU to train this model. But the MAE I got on DUTS-TE is around 0.088 (it is 0.033 in paper). In picture, it shows the loss and the MAE of train and val set when training. It early stopped at 82th epoch. It kind of converged.

    I would like to know did you do any data augmentation off-line (I train on DUTS-TR and the edge gt you provided. I simply change the folder directory to meet your folder structure mentioned in readme)? Should I change any hyper parameter to achieve the same result? image

    opened by YiChuanChang 5
  • Reproducing paper results of TE7

    Reproducing paper results of TE7

    Tried training using the same hyper params of the paper in order to reproduce results on the DUTS-TE split for example.
    Apart from batch size which had to be reduced to 8, all other params were the same.
    However, I reach inferior results:

    Dataset:DUTS
    Test Loss:1.457 | MAX_F:0.903 | AVG_F:0.873 | MAE:0.030 | S_Measure:0.891, time: 68938.211s
    

    This is the training command I used:

    python main.py train --exp_num 1 --arch 7 --img_size 640 --model_path /data/outputs/TRACER --data_path /data/datasets/TRACER/ --multi_gpu true --num_workers 12 --batch_size 8
    

    The best model was achieved at epoch 14 as stated in the logs:

    Best Val Epoch:14 | Val Loss:38.960 | Val MAE:0.019
    

    And the training concluded after epoch 24 due to early stopping mechanism.

    opened by aviadmx 4
  • Reproducible results

    Reproducible results

    I tested the model for some time and witnessed something strange. The same mask doesn't show up if i entered the same input image. It seems there is a random seed generator in the code that affects the output results.

    Do you have any idea where something like this can happen ?

    opened by MohamedAliRashad 4
  • does TRACER pretrained model used for training?

    does TRACER pretrained model used for training?

    Hi, I downloaded pre-trained models of TRACER as you mentioned in README, but I'm confused about the step below Change the model name as 'best_model.pth' and put the weights to the path 'results/DUTS/TEx_0/best_model.pth'

    question is, this path is model save path, not pretrained model load path, and I could not find you load "best_model.pth" at start of training anywhere in your code, did I miss something?

    opened by Alan-Han 3
  • Runtime Speeds

    Runtime Speeds

    Hi, thanks for putting out an interesting paper and the code. I was testing it with the various pretrained weights on DUT-TE, and I'm getting an end-to-end speed of ~11 FPS on an RTX 2070 Super. Is that an expected speed?

    opened by vjsrinivas 3
  • How did you generate gt edge?

    How did you generate gt edge?

    Hi This is really an amazing work. I would like to reproduce your result. But I found you did not mentioned how you generate gt edge data in the paper. And I don't have the permission to access the gt edge data link in README.

    Of course, it would be wonderful if you can provide the full pre-trained model.

    opened by YiChuanChang 3
  • Accessing The Actual Predicited Masks

    Accessing The Actual Predicited Masks

    Hi, I tried to replicate your code and managed to replicate the test on the DUTS test set using TE0_0 pretrained model. However I would like to know how to get the actual masks predicted of the images after running the test. Thanks, Malcolm

    opened by malcolmagius19 2
  • Mask Generation?

    Mask Generation?

    Hello! And thank you for this work. I was curious if you would be releasing or amending (or if I missed it in your paper) the mask generation part of your testing code? To try against more complicated imagery in visual form?

    opened by LBNord 2
  • Bump certifi from 2021.5.30 to 2022.12.7

    Bump certifi from 2021.5.30 to 2022.12.7

    Bumps certifi from 2021.5.30 to 2022.12.7.

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Losses decrease quite slowly with custom dataset

    Losses decrease quite slowly with custom dataset

    I created a custom dataset and tried training with arch 5 with image size 512 but the loss decreased quite slowly. Is this normal, or is my data problem?

    my loss Epoch:[001/200] Train Loss:99.721 | MAE:0.189 Valid Loss:82.747 | MAE:0.125 =============================Epoch:[002/200] Train Loss:91.156 | MAE:0.156 Valid Loss:91.560 | MAE:0.166 =============================Epoch:[003/200] Train Loss:91.532 | MAE:0.155 Valid Loss:78.634 | MAE:0.106 =============================Epoch:[004/200] Train Loss:86.963 | MAE:0.135 Valid Loss:76.247 | MAE:0.095 =============================Epoch:[005/200] Train Loss:85.692 | MAE:0.130 Valid Loss:74.397 | MAE:0.086 =============================Epoch:[006/200] Train Loss:85.188 | MAE:0.127 Valid Loss:74.988 | MAE:0.086 =============================Epoch:[007/200] Train Loss:84.972 | MAE:0.125 Valid Loss:73.414 | MAE:0.082 =============================Epoch:[008/200] Train Loss:83.838 | MAE:0.121 Valid Loss:75.131 | MAE:0.087 =============================Epoch:[009/200] Train Loss:83.403 | MAE:0.119

    opened by bbsvip 0
  • Custom Data

    Custom Data

    Hi, I'm getting this error. @Karel911

    <---- Training Params ----> Namespace(RFB_aggregated_channel=[32, 64, 128], action='train', arch='7', aug_ver=1, batch_size=8, channels=[24, 40, 112, 320], clipping=2, criterion='API', data_path='/home/nagas/TRACER/data', dataset='DUTS', denoise=0.93, epochs=100, exp_num=0, frequency_radius=16, gamma=0.1, img_size=640, lr=5e-05, lr_factor=0.1, model_path='/home/nagas/TRACER/results', multi_gpu=True, num_workers=4, optimizer='Adam', patience=5, save_map=None, scheduler='Reduce', seed=42, weight_decay=0.0001) train length : 42 val length : 3 Loaded pretrained weights for efficientnet-b7 0%| | 0/5 [00:00<?, ?it/s]ERROR: Unexpected segmentation fault encountered in worker. ERROR: Unexpected segmentation fault encountered in worker. ERROR: Unexpected segmentation fault encountered in worker. ERROR: Unexpected segmentation fault encountered in worker. 0%| | 0/5 [00:00<?, ?it/s] Traceback (most recent call last): File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 990, in _try_get_data data = self._data_queue.get(timeout=timeout) File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/multiprocessing/queues.py", line 104, in get if not self._poll(timeout): File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/multiprocessing/connection.py", line 257, in poll return self._poll(timeout) File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/multiprocessing/connection.py", line 414, in _poll r = wait([self], timeout) File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/multiprocessing/connection.py", line 921, in wait ready = selector.select(timeout) File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/selectors.py", line 415, in select fd_event_list = self._selector.poll(timeout) File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/_utils/signal_handling.py", line 66, in handler _error_if_any_worker_fails() RuntimeError: DataLoader worker (pid 20604) is killed by signal: Segmentation fault.

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last): File "main.py", line 56, in main(args) File "main.py", line 35, in main Trainer(args, save_path) File "/home/nagas/TRACER/trainer.py", line 56, in init train_loss, train_mae = self.training(args) File "/home/nagas/TRACER/trainer.py", line 101, in training for images, masks, edges in tqdm(self.train_loader): File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/tqdm/std.py", line 1185, in iter for obj in iterable: File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1186, in _next_data idx, data = self._get_data() File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1152, in _get_data success, data = self._try_get_data() File "/home/nagas/miniconda3/envs/tracer/lib/python3.7/site-packages/torch/utils/data/dataloader.py", line 1003, in _try_get_data raise RuntimeError('DataLoader worker (pid(s) {}) exited unexpectedly'.format(pids_str)) from e RuntimeError: DataLoader worker (pid(s) 20604) exited unexpectedly

    opened by arjunsankar 1
  • multi class question

    multi class question

    Hello. Thank you for your wonderful research.

    I want to extend your methodology to multi-class classification. Is this possible? If possible, could you give me some implementation tips?

    opened by DeepHM 4
  • Adaptive pixel intensity loss

    Adaptive pixel intensity loss

    hi,thank you for your excellent work,I don't understand the following formula. Can you explain it for me? image image 1、Why does equation 8 need to be multiplied by yij? This will cause all background weights to be 0。 2、What is the meaning of adding 1.5 to the denominator in formula 9? Thank you very much and hope to get your reply!!

    opened by liuzhihui2046 1
Owner
Karel
Karel
Official code for NeurIPS 2021 paper "Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN"

Towards Scalable Unpaired Virtual Try-On via Patch-Routed Spatially-Adaptive GAN Official code for NeurIPS 2021 paper "Towards Scalable Unpaired Virtu

68 Dec 21, 2022
SpineAI Bilsky Grading With Python

SpineAI-Bilsky-Grading SpineAI Paper with Code 📫 Contact Address correspondence to J.T.P.D.H. (e-mail: james_hallinan AT nuhs.edu.sg) Disclaimer This

<a href=[email protected]"> 2 Dec 16, 2021
[CVPR 2022 Oral] TubeDETR: Spatio-Temporal Video Grounding with Transformers

TubeDETR: Spatio-Temporal Video Grounding with Transformers Website • STVG Demo • Paper This repository provides the code for our paper. This includes

Antoine Yang 108 Dec 27, 2022
Official PyTorch Implementation of paper "Deep 3D Mask Volume for View Synthesis of Dynamic Scenes", ICCV 2021.

Deep 3D Mask Volume for View Synthesis of Dynamic Scenes Official PyTorch Implementation of paper "Deep 3D Mask Volume for View Synthesis of Dynamic S

Ken Lin 17 Oct 12, 2022
[ICCV'2021] "SSH: A Self-Supervised Framework for Image Harmonization", Yifan Jiang, He Zhang, Jianming Zhang, Yilin Wang, Zhe Lin, Kalyan Sunkavalli, Simon Chen, Sohrab Amirghodsi, Sarah Kong, Zhangyang Wang

SSH: A Self-Supervised Framework for Image Harmonization (ICCV 2021) code for SSH Representative Examples Main Pipeline RealHM DataSet Google Drive Pr

VITA 86 Dec 02, 2022
Python Library for Signal/Image Data Analysis with Transport Methods

PyTransKit Python Transport Based Signal Processing Toolkit Website and documentation: https://pytranskit.readthedocs.io/ Installation The library cou

24 Dec 23, 2022
DeepLabv3+:Encoder-Decoder with Atrous Separable Convolution语义分割模型在tensorflow2当中的实现

DeepLabv3+:Encoder-Decoder with Atrous Separable Convolution语义分割模型在tensorflow2当中的实现 目录 性能情况 Performance 所需环境 Environment 注意事项 Attention 文件下载 Download

Bubbliiiing 31 Nov 25, 2022
DETReg: Unsupervised Pretraining with Region Priors for Object Detection

DETReg: Unsupervised Pretraining with Region Priors for Object Detection Amir Bar, Xin Wang, Vadim Kantorov, Colorado J Reed, Roei Herzig, Gal Chechik

Amir Bar 283 Dec 27, 2022
Deep Multi-Magnification Network for multi-class tissue segmentation of whole slide images

Deep Multi-Magnification Network This repository provides training and inference codes for Deep Multi-Magnification Network published here. Deep Multi

Computational Pathology 12 Aug 06, 2022
Code for the Paper: Conditional Variational Capsule Network for Open Set Recognition

Conditional Variational Capsule Network for Open Set Recognition This repository hosts the official code related to "Conditional Variational Capsule N

Guglielmo Camporese 35 Nov 21, 2022
This repository contains the source code of our work on designing efficient CNNs for computer vision

Efficient networks for Computer Vision This repo contains source code of our work on designing efficient networks for different computer vision tasks:

Sachin Mehta 386 Nov 26, 2022
Bringing Computer Vision and Flutter together , to build an awesome app !!

Bringing Computer Vision and Flutter together , to build an awesome app !! Explore the Directories Flutter · Machine Learning Table of Contents About

Padmanabha Banerjee 14 Apr 07, 2022
Simulation of self-focusing of laser beams in condensed media

What is it? Program for scientific research, which allows to simulate the phenomenon of self-focusing of different laser beams (including Gaussian, ri

Evgeny Vasilyev 13 Dec 24, 2022
SW components and demos for visual kinship recognition. An emphasis is put on the FIW dataset-- data loaders, benchmarks, results in summary.

FIW Data Development Kit Table of Contents Introduction Families In the Wild Database Publications Organization To Do License Getting Involved Introdu

Joseph P. Robinson 12 Jun 04, 2022
Official Implementation for the "An Empirical Investigation of 3D Anomaly Detection and Segmentation" paper.

An Empirical Investigation of 3D Anomaly Detection and Segmentation Project | Paper Official PyTorch Implementation for the "An Empirical Investigatio

Eliahu Horwitz 55 Dec 14, 2022
Semi-supervised Transfer Learning for Image Rain Removal. In CVPR 2019.

Semi-supervised Transfer Learning for Image Rain Removal This package contains the Python implementation of "Semi-supervised Transfer Learning for Ima

Wei Wei 59 Dec 26, 2022
SAPIEN Manipulation Skill Benchmark

ManiSkill Benchmark SAPIEN Manipulation Skill Benchmark (abbreviated as ManiSkill, pronounced as "Many Skill") is a large-scale learning-from-demonstr

Hao Su's Lab, UCSD 107 Jan 08, 2023
LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation

LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation by Junjue Wang, Zhuo Zheng, Ailong Ma, Xiaoyan Lu, and Yanfei Zh

Payphone 8 Nov 21, 2022
Differentiable Surface Triangulation

Differentiable Surface Triangulation This is our implementation of the paper Differentiable Surface Triangulation that enables optimization for any pe

61 Dec 07, 2022
Laplace Redux -- Effortless Bayesian Deep Learning

Laplace Redux - Effortless Bayesian Deep Learning This repository contains the code to run the experiments for the paper Laplace Redux - Effortless Ba

Runa Eschenhagen 28 Dec 07, 2022