VCM EE1.2 P-layer feature map anchor generation 137th MPEG-VCM

Overview

VCM_EE1.2_P-layer_feature_map_anchor_generation_137th_MPEG-VCM

#######################################################

Author: Minhun Lee, Hansol Choi, Seungjin Park, Minsub Kim, and Donggyu Sim

E-mail: {minhun, hschoi95, promo, minsub, dgsim}@kw.ac.kr

####################################################### [Introduction]

This package contains scripts to generate anchor results of object detection on P-layer features (p2, p3, p4, p5) extracted from OpenImages dataset for MPEG Video Coding for Machines(VCM).

Please note that this test procedure is organized based on Nokia's latest contribution(m57343) for generating VCM anchor on the OpenImages dataset V6.

####################################################### [Software environment]

Ubuntu 20.04.1 LTS

Python 3.8.11

Torch 1.9.0

Detectron2 0.5

Object-detection 0.1

Pandas 1.3.3

Numpy 1.21.2

Opencv-python 4.5.3.56

Pillow 8.3.1

ffmpeg 4.4

VTM 12.0

####################################################### [Faster-RCNN model parameter]

Download the Faster-RCNN model parameters from the following link: https://dl.fbaipublicfiles.com/detectron2/COCO-Detection/faster_rcnn_X_101_32x8d_FPN_3x/139173657/model_final_68b088.pkl

Place the downloaded model_final_68b088.pkl file in the models/x101fpn/ directory.

####################################################### [OpenImages V6 Dataset]

Download OpenImages V6 validation set according to the instruction from the following webpage: https://storage.googleapis.com/openimages/web/challenge2019_downloads.html

The downloaded validation.tar.gz file has size of 12G bytes and contains 41620 jpg images. Untar this file to directory dataset/validation

For annotations, we have already set files to './dataset/annotations/' and './oi_eval/' directories. For dataset, have to move only 5k images of the OpenImages dataset V6 to './dataset/val_openimage_v6/*.jpg' directory as below.

####################################################### [Dataset directory structure]

./dataset/val_openimage_v6/ 0a1bd356f90aaab6.jpg ... ffddf3805faf3cbf.jpg # only 5k images

####################################################### [Instructions]

Please run 'demo.sh' script to generate P-layer anchor results. The outputs will be stored in './feature/' and './output/' directories which are generated automatically, and the results from our experiments are also included in 'P-layer_anchor_report.xlsm' file. This top procedure consits of three phases as below.

In the first phase, the P-layer features are extracted from the faster_rcnn_X_101_32x8d_FPN_3 network, and the extracted P-layer features are stored as YUV 4:0:0 format using FFmpeg (png to yuv) after tiling and uniform quantization (10-bits). For feature tiling into YUV 4:0:0, we arranged 256 channels of the p2, p3, p4, and p5 feature maps in a raster scanning order, respectively, so that each YUV 4:0:0 data includes 2D feature for each input image. For the uniform quantisation process, we measured the global maximum and minimum values in the P-layer features over the whole dataset, and the the global maximum and minimum values were 20.3891 and -22.3948, respectively.

In the second phase, the YUV format data are encoded and then decoded via VTM 12.0 software with six different QP values, 35, 37, 39, 41, 43 and 45. Here we store the encoded bitstreams ('./feature/{QP}_bit/') and the reconstructed YUV format data ('./feature/{QP}_rec/') and the original feature map data ('./feature/{QP}_ori/') in the designated directory for each QP value. In addition, please note that we actually performed the encoding jobs in a parallel manner using threading, the thread is setting the default value '4', you can change the value at each './settings/{QP}.json' file.

In the thrid phase, we calculate the bit-per-pixel(bpp) and measure the mAP performance for each QP, based on the bitstreams and the reconstructions generated at the phase two. And the result files are stored './output/{QP}_AP.txt' for each qp value.

Owner
IPSL
Welcome to the Image Processing Systems Laboratory. The IPSL was established in the department of Computer Engineering at Kwangwoon University in 2005.
IPSL
Amazon SageMaker Delta Sharing Examples

This repository contains examples and related resources showing you how to preprocess, train, and serve your models using Amazon SageMaker with data fetched from Delta Lake.

Eitan Sela 5 May 02, 2022
RxPY - The Reactive Extensions for Python (RxPY)

The Reactive Extensions for Python (RxPY) A library for composing asynchronous and event-based programs using observable collections and query operato

ReactiveX 4.4k Dec 29, 2022
External Network Pentest Automation using Shodan API and other tools.

Chopin External Network Pentest Automation using Shodan API and other tools. Workflow Input a file containing CIDR ranges. Converts CIDR ranges to ind

Aditya Dixit 9 Aug 04, 2022
Release for Improved Denoising Diffusion Probabilistic Models

improved-diffusion This is the codebase for Improved Denoising Diffusion Probabilistic Models. Usage This section of the README walks through how to t

OpenAI 1.2k Dec 30, 2022
Structural basis for solubility in protein expression systems

Structural basis for solubility in protein expression systems Large-scale protein production for biotechnology and biopharmaceutical applications rely

ProteinQure 16 Aug 18, 2022
In this project, we are going to display the battery notification and the time left for the battery to drain out using the battery capacity value.

In this project, we are going to display the battery notification and the time left for the battery to drain out using the battery capacity value.

Ritoban Biswas 1 Dec 20, 2021
jmespath.rs Python binding

rjmespath-py jmespath.rs Python binding.

messense 3 Dec 14, 2022
Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels that can be solved in any programming language you like.

Advent Of Code 2021 - Python English Advent of Code is an Advent calendar of small programming puzzles for a variety of skill sets and skill levels th

Coral Izquierdo Muñiz 2 Jan 09, 2022
You will need to install a few python packages for this one.

Features Bait support Auto repair will repair every 10 catches Anti detection (still a work in progress) but using random times and click positions Pr

12 Sep 21, 2022
Customisable coding font with alternates, ligatures and contextual positioning

Guide Ligature Support Links Log License Guide Live Preview + Download larsenwork.com/monoid Install Quit your editor/program. Unzip and open the fold

Andreas Larsen 7.6k Dec 30, 2022
A python package template that can be adapted for RAP projects

Warning - this repository is a snapshot of a repository internal to NHS Digital. This means that links to videos and some URLs may not work. Repositor

NHS Digital 3 Nov 08, 2022
Data and analysis relating to the 5.8M Melbourne quake of 2021

quake2021 Data and analysis relating to the 5.8M Melbourne quake of 2021 Monash University Woodside Living Lab Building The building is located here T

Colin Caprani 6 May 16, 2022
Xbox-Flood is for flood anything

Intruduction Installation Usage Installing Python 3 Wiki Getting Started Creating a Key Intruduction Xbox-Flood is for flooding messages (invitations

kayake 4 Feb 18, 2022
In this repo, I will put all the code related to data science using python libraries like Numpy, Pandas, Matplotlib, Seaborn and many more.

Python-for-DS In this repo, I will put all the code related to data science using python libraries like Numpy, Pandas, Matplotlib, Seaborn and many mo

1 Jan 10, 2022
Submission to the HEAR2021 Challenge

Submission to the HEAR 2021 Challenge For model evaluation, python=3.8 and cuda10.2 with cudnn7.6.5 have been tested. The work uses a mixed supervised

Heinrich Dinkel 10 Dec 08, 2022
Template for pre-commit hooks

Pre-commit hook template This repo is a template for a pre-commit hook. Try it out by running: pre-commit try-repo https://github.com/stefsmeets/pre-c

Stef Smeets 1 Dec 09, 2021
Taking the fight to the establishment.

Throwdown Taking the fight to the establishment. Wat? I wanted a simple markdown interpreter in python and/or javascript to output html for my website

Trevor van Hoof 1 Feb 01, 2022
A tool that bootstraps your dotfiles ⚡️

Dotbot Dotbot makes installing your dotfiles as easy as git clone $url && cd dotfiles && ./install, even on a freshly installed system! Rationale Gett

Anish Athalye 5.9k Jan 07, 2023
Jarvis Python BOT acts like Google-assistance

Jarvis-Python-BOT Jarvis Python BOT acts like Google-assistance Setup Add Mail ID (Gmail) in the file at line no 82.

Ishan Jogalekar 1 Jan 08, 2022
A smart personal companion and health assistant.

Steps to Install : Clone the repository Go to ResQ-Sources Execute ResQ-Lite.py --: Manual Controls : DanceRobot.py --: You can call functions like fo

Tuhinadri Banerjee 1 May 25, 2022