Code of the paper "Part Detector Discovery in Deep Convolutional Neural Networks" by Marcel Simon, Erik Rodner and Joachim Denzler

Overview

Part Detector Discovery

This is the code used in our paper "Part Detector Discovery in Deep Convolutional Neural Networks" by Marcel Simon, Erik Rodner and Joachim Denzler published at ACCV 2014. If you would like to refer to this work, please cite the corresponding paper

@inproceedings{Simon14:PDD,
  author = {Marcel Simon and Erik Rodner and Joachim Denzler},
  booktitle = {Asian Conference on Computer Vision (ACCV)},
  title = {Part Detector Discovery in Deep Convolutional Neural Networks},
  year = {2014},
}

The following steps will guide you through the usage of the code.

1. Python Environment

Setup a python environment, preferably a virtual environment using e. g. virtual_env. The requirements file might install more than you need.

virtualenv pyhton-env && pip install -r requirements.txt

2. DeCAF Installation

Build and install decaf into this environment

source python-env/bin/activate
cd decaf-tools/decaf/
python setup.py build
python setup.py install

3. Pre-Trained ImageNet Model

Get the decaf ImageNet model:

cd decaf-tools/models/
bash get_model.sh

You now might need to adjust the path to the decaf model in decaf-tools/extract_grad_map.py, line 75!

4. Gradient Map Calculation

Now you can calculate the gradient maps using the following command. For a single image, use decaf-tools/extract_grad_map.py :

usage: extract_grad_map.py [-h] [--layers LAYERS [LAYERS ...]] [--limit LIMIT]
                           [--channel_limit CHANNEL_LIMIT]
                           [--images pattern [pattern ...]] [--outdir OUTDIR]

Calculate the gradient maps for an image.

optional arguments:
  -h, --help            show this help message and exit
  --layers LAYERS [LAYERS ...]
  --limit LIMIT         When calculating the gradient of the class scores,
                        calculate the gradient for the output elements with the
                        [limit] highest probabilities.
  --channel_limit CHANNEL_LIMIT
                        Sets the number of channels per layer you want to
                        calculate the gradient of.
  --images pattern [pattern ...]
			Absolute image path to the image. You can use wildcards.
  --outdir OUTDIR

For a list of absolute image paths call this script this way:

python extract_grad_map.py --images $(cat /path/to/imagelist.txt) --limit 1 --channel_limit 256 --layers probs pool5 --outdir /path/to/output/

The gradient maps are stored as Matlab .mat file and as png. In addition to these, the script also generates A html file to view the gradient maps and the input image. The gradient map is placed in the directory outdir/images'_parent_dir/image_filename/*. Be aware that approx. 45 MiB of storage is required per input image. For the whole CUB200-2011 dataset this means a total storage size of approx 800 GiB!

5. Part Localization

Apply the part localization using GMM fitting or maximum finding. Have a look in the part_localization folder for that. Open calcCUBPartLocs.m and adjust the paths. Now simply run calcCUBPartLocs(). This will create a file which has the same format as the part_locs.txt file of the CUB200-2011 dataset. You can use it for part-based classification.

6. Classification

We also provide the classification framework to use these part localizations and feature extraction with DeCAF. Go to the folder classification and open partEstimationDeepLearing.m. Have a look at line 40 and adjust the path such that it points to the correct file. Open settings.m and adjust the paths. Next, open settings.m and adjust the paths to liblinear and the virtual python environment. Now you can execute for example:

init
recRate = experimentParts('cub200_2011',200, struct('descriptor','plain','preprocessing_useMask','none','preprocessing_cropToBoundingbox',0), struct('partSelection',[1 2 3 9 14],'bothSymmetricParts',0,'descriptor','plain','trainPartLocation','est','preprocessing_relativePartSize',1.0/8,'preprocessing_cropToBoundingbox',0))

This will evaluate the classification performance on the standard train-test-split using the estimated part locations. Experiment parts has four parameters. The first one tell the function which dataset to use. You want to keep 'cub200_2011' here.

The second one is the number of classes to use, 3, 14 and 200 is supported here. Next is the setup for the global feature extraction. The only important setting is preprocessing_cropToBoundingbox. A value of 0 will tell the function not to use the ground truth bounding box during testing. You should leave the other two options as shown here.

The last one is the setup for the part features. You can select here, which parts you want to use and if you want to extract features from both symmetric parts, if both are visible. Since the part detector discovery associates some parts with the same channel, the location prediction will be the same for these. In this case, only select the parts which have unique channels here. In the example, the part 1, 2, 3, 9 and 14 are associated with different channels.

'trainPartLocation' tells the function, if grount-truth ('gt') or estimated ('est') part locations should be used for training. Since the discovered part detectors do not necessarily relate to semantic parts, 'est' usually is the better option here.

'preprocessing_relativePartSize' adjusts the size of patches, that are extracted at the estimated part locations. Please have a look at the paper for more information.

For the remaining options, you should keep everything as it is.

Acknowledgements

The classification framework is an extension of the excellent fine-grained recognition framework by Christoph Göring, Erik Rodner, Alexander Freytag and Joachim Denzler. You can find their project at https://github.com/cvjena/finegrained-cvpr2014.

Our work is based on DeCAF, a framework for convolutional neural networks. You can find the repository of the corresponding project at https://github.com/UCB-ICSI-Vision-Group/decaf-release/ .

License

Part Detector Discovery Framework by Marcel Simon, Erik Rodner and Joachim Denzler is licensed under the non-commercial license Creative Commons Attribution 4.0 International License. For usage beyond the scope of this license, please contact Marcel Simon.

You might also like...
Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency
Code for reproducing our analysis in the paper titled: Image Cropping on Twitter: Fairness Metrics, their Limitations, and the Importance of Representation, Design, and Agency

Image Crop Analysis This is a repo for the code used for reproducing our Image Crop Analysis paper as shared on our blog post. If you plan to use this

Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning".

ERICA Source code and dataset for ACL2021 paper: "ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive L

Code for the ICML 2021 paper
Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Training and Effective Adaptation", Haoxiang Wang, Han Zhao, Bo Li.

Bridging Multi-Task Learning and Meta-Learning Code for the ICML 2021 paper "Bridging Multi-Task Learning and Meta-Learning: Towards Efficient Trainin

Data and Code for ACL 2021 Paper
Data and Code for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning"

Introduction Code and data for ACL 2021 Paper "Inter-GPS: Interpretable Geometry Problem Solving with Formal Language and Symbolic Reasoning". We cons

Code for the paper "VisualBERT: A Simple and Performant Baseline for Vision and Language"

This repository contains code for the following two papers: VisualBERT: A Simple and Performant Baseline for Vision and Language (arxiv) with a short

Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.
Open source repository for the code accompanying the paper 'Non-Rigid Neural Radiance Fields Reconstruction and Novel View Synthesis of a Deforming Scene from Monocular Video'.

Non-Rigid Neural Radiance Fields This is the official repository for the project "Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synt

Code for the Shortformer model, from the paper by Ofir Press, Noah A. Smith and Mike Lewis.

Shortformer This repository contains the code and the final checkpoint of the Shortformer model. This file explains how to run our experiments on the

Open source code for Paper
Open source code for Paper "A Co-Interactive Transformer for Joint Slot Filling and Intent Detection"

A Co-Interactive Transformer for Joint Slot Filling and Intent Detection This repository contains the PyTorch implementation of the paper: A Co-Intera

A code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Vanderhaeghe, and Yotam Gingold from SIGGRAPH Asia 2020.

A Benchmark for Rough Sketch Cleanup This is the code repository associated with the paper A Benchmark for Rough Sketch Cleanup by Chuan Yan, David Va

Releases(v1.0)
Owner
Computer Vision Group Jena
Computer Vision Group Jena
Code for the paper "M2m: Imbalanced Classification via Major-to-minor Translation" (CVPR 2020)

M2m: Imbalanced Classification via Major-to-minor Translation This repository contains code for the paper "M2m: Imbalanced Classification via Major-to

79 Oct 13, 2022
TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured Scenarios

TPH-YOLOv5 This repo is the implementation of "TPH-YOLOv5: Improved YOLOv5 Based on Transformer Prediction Head for Object Detection on Drone-Captured

cv516Buaa 439 Dec 22, 2022
Graph Robustness Benchmark: A scalable, unified, modular, and reproducible benchmark for evaluating the adversarial robustness of Graph Machine Learning.

Homepage | Paper | Datasets | Leaderboard | Documentation Graph Robustness Benchmark (GRB) provides scalable, unified, modular, and reproducible evalu

THUDM 66 Dec 22, 2022
Hierarchical Uniform Manifold Approximation and Projection

HUMAP Hierarchical Manifold Approximation and Projection (HUMAP) is a technique based on UMAP for hierarchical non-linear dimensionality reduction. HU

Wilson Estécio Marcílio Júnior 160 Jan 06, 2023
Combinatorial model of ligand-receptor binding

Combinatorial model of ligand-receptor binding The binding of ligands to receptors is the starting point for many import signal pathways within a cell

Mobolaji Williams 0 Jan 09, 2022
Neural network chess engine trained on Gary Kasparov's games.

Neural Chess It's not the best chess engine, but it is a chess engine. Proof of concept neural network chess engine (feed-forward multi-layer perceptr

3 Jun 22, 2022
Jremesh-tools - Blender addon for quad remeshing

JRemesh Tools Blender 2.8 - 3.x addon for quad remeshing. Currently it is a wrap

Jayanam 89 Dec 30, 2022
Tooling for GANs in TensorFlow

TensorFlow-GAN (TF-GAN) TF-GAN is a lightweight library for training and evaluating Generative Adversarial Networks (GANs). Can be installed with pip

803 Dec 24, 2022
A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

MMF is a modular framework for vision and language multimodal research from Facebook AI Research. MMF contains reference implementations of state-of-t

Facebook Research 5.1k Jan 04, 2023
Given a 2D triangle mesh, we could randomly generate cloud points that fill in the triangle mesh

generate_cloud_points Given a 2D triangle mesh, we could randomly generate cloud points that fill in the triangle mesh. Run python disp_mesh.py Or you

Peng Yu 2 Dec 24, 2021
Code for our paper "SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization", ACL 2021

SimCLS Code for our paper: "SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization", ACL 2021 1. How to Install Requirements

Yixin Liu 150 Dec 12, 2022
Lightweight Face Image Quality Assessment

LightQNet This is a demo code of training and testing [LightQNet] using Tensorflow. Uncertainty Losses: IDQ loss PCNet loss Uncertainty Networks: Mobi

Kaen 5 Nov 18, 2022
The official implementation of the research paper "DAG Amendment for Inverse Control of Parametric Shapes"

DAG Amendment for Inverse Control of Parametric Shapes This repository is the official Blender implementation of the paper "DAG Amendment for Inverse

Elie Michel 157 Dec 26, 2022
Face and other object detection using OpenCV and ML Yolo

Object-and-Face-Detection-Using-Yolo- Opencv and YOLO object and face detection is implemented. You only look once (YOLO) is a state-of-the-art, real-

Happy N. Monday 3 Feb 15, 2022
Pytorch implementation of CVPR2021 paper "MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generation"

MUST-GAN Code | paper The Pytorch implementation of our CVPR2021 paper "MUST-GAN: Multi-level Statistics Transfer for Self-driven Person Image Generat

TianxiangMa 46 Dec 26, 2022
EEGEyeNet is benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty

Introduction EEGEyeNet EEGEyeNet is a benchmark to evaluate ET prediction based on EEG measurements with an increasing level of difficulty. Overview T

Ard Kastrati 23 Dec 22, 2022
Welcome to The Eigensolver Quantum School, a quantum computing crash course designed by students for students.

TEQS Welcome to The Eigensolver Quantum School, a crash course designed by students for students. The aim of this program is to take someone who has n

The Eigensolvers 53 May 18, 2022
Second-order Attention Network for Single Image Super-resolution (CVPR-2019)

Second-order Attention Network for Single Image Super-resolution (CVPR-2019) "Second-order Attention Network for Single Image Super-resolution" is pub

516 Dec 28, 2022
The implementation for "Comprehensive Knowledge Distillation with Causal Intervention".

Comprehensive Knowledge Distillation with Causal Intervention This repository is a PyTorch implementation of "Comprehensive Knowledge Distillation wit

Xiang Deng 10 Nov 03, 2022
a reimplementation of UnFlow in PyTorch that matches the official TensorFlow version

pytorch-unflow This is a personal reimplementation of UnFlow [1] using PyTorch. Should you be making use of this work, please cite the paper according

Simon Niklaus 134 Nov 20, 2022