Source codes of CenterTrack++ in 2021 ICME Workshop on Big Surveillance Data Processing and Analysis

Overview

MOT Tracked object bounding box association (CenterTrack++)

New association method based on CenterTrack. Two new branches (Tracked Size and IOU) are added onto the original CenterTrack tracker. The proposed method enables the computation of IOU distance matrix for more accurate object association compared to single displacement offset in the original CenterTrack.

Modification to CenterTrack method, image modified from CenterTrack

Abstract

The recent development of multi-object tracking (MOT) on point-based joint detection and tracking methods has attracted much research attention. CenterTrack tracking algorithm is one of such promising methods. It achieves state-of-the-art tracking performance using a simple detection model and single-frame spatial offsets to localize objects and predict their associations in a single network. However, this method still suffers from high identity switches due to the inferior association method. Only point displacement distance matrix is used to associate objects, which is not robust to deal with occlusion scenarios. To reduce the high number of identity switches and improve the tracking accuracy, more effective spatial information should be used in association. In this paper, we propose to incorporate a simple tracked object bounding box and overlapping prediction based on the current frame onto the CenterTrack algorithm. Specifically, we propose a Intersection over Union (IOU) distance cost matrix in the association step instead of point displacement distance. We evaluate our proposed tracker on the MOT17 test dataset, showing that our proposed method can reduce identity switches significantly by 22.6% and obtain a notable improvement of 1.5% in IDF1 compared to the original CenterTrack’s under the same tracklet lifetime.

Main Contributions

  • Proposed two branches (tracked box size and IOU)on top of the existing CenterTrack method for IOU distance metric computation in object association
  • Evaluation the proposed method on MOT17 dataset and obtain significant reduction in IDs and notable improvements in tracking accuracy score

Two new branches

The idea of the proposed method is to enhance the original displacement only association. Inspired by the IOU distance in SORT and IOU-Tracker, IOU distance can be used for more accurate object association across frames. IOU distance is calculated as 1 - IOU(bounding box of detected object in the previous frame and the predicted tracked object bounding box in the previous frame based on the current frame)

Tracked Object Size prediction

In order to obtain the IOU distance, the bounding box of the tracked object in the previous frame should be learnt. In this project, two methods were used to learn the tracked bounding box.

Tracking_wh: Directly learn the width and height of the tracked object bounding box in the previous frame.

Tracking_ltrb: Learn the offsets of the left, top, right and bottom of bounding box from the tracked object center in the previous frame.

The tracking_wh(left) and tracking_ltrb(right) approach illustration.

IOU prediction

To further suppress inaccurate association, the IOU value of the tracked object bounding box in adjacent frames is learnt to provide a threshold to filter unlikely associations. We would set the IOU distance to infinity if IOU distance > IOU.

Association Method

Main results

Comparison with other SOTA tracker on MOT17 test set

Note: S= Spatial features, A=appearance features

Tracker Association Features MOTA IDF1 IDs
TubeTK S 63 58.6 4137
CenterTrack S 67.8 64.7 3039
Ours S 68.1 66.2 2352
SST A 52.4 49.5 8431
CTrackerV1 S+A 66.6 57.4 5529
DEFT S+A 66.6 65.4 2823
FairMOT S+A 73.7 72.3 3303

Ablative studies on tracked size prediction method

Tracking_wh

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 71.1 66.7 204 3.6 29.3
Combined 70.9 66.2 233 3.9 29.6
DIS→IOU 70 66.2 218 3.9 29.5
IOU→DIS 69.8 66.8 185 3.6 29.2

Tracking_ltrb

Association Method IDF1 MOTA IDs FP(%) FN(%)
DIS 69.2 66.2 219 3.9 29.5
IOU 72.4 66.7 191 3.8 29.2
Combined 70.8 66.5 236 3.8 29.3
DIS→IOU 70.5 66.6 202 3.8 29.2
IOU→DIS 71.4 66.7 166 3.8 29.2

Installation

Please refer to INSTALL.md for installation instructions.

Training and Evaluation

  • Download the crowdhuman pretrained model from xinyizhou/CenterTrack MODEL ZOO.md to models
  • prepare the data and convert it into COCO format refer to the original CenterTrack repo.
  • change the dataset root directory data_dir in opt.py
  • ablative studies for tracking_wh and tracking_ltrb approach respectively with five association method (IOU,DIS,Combined, IOU→DIS, DIS→IOU)
sh experiments/mot17val_tracking_wh.sh

sh experiments/mot17val_tracking_ltrb.sh

The trained model on MOT17val dataset using two approach are available in google drive, tracking_ltrb_70val.pth, tracking_wh_70val.pth.

  • Train on full mot17 training set and run model on the test set for evaluation
sh experiments/mot17full.sh

The trained models on full MOT17 dataset using ltrb approach is available in the google drive.

Demo comparison

Occlusion case

Original CenterTrack (left) vs CenterTrack++ (right)

Object exiting the frame

Original CenterTrack (left) vs CenterTrack++ (right)

Acknowledgement

A large part of the code is adapted from xingyizhou/CenterTrack, thanks for their wonderful inspiration.

Citation

If you find this paper and code useful in your research, please cite our papers.

@misc{yang2021multiobject,
      title={Multi-object Tracking with Tracked Object Bounding Box Association}, 
      author={Nanyang Yang and Yi Wang and Lap-Pui Chau},
      year={2021},
      eprint={2105.07901},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}
Owner
Nanyang Technological University Information Engineering and Media Student
NAS-Bench-x11 and the Power of Learning Curves

NAS-Bench-x11 NAS-Bench-x11 and the Power of Learning Curves Shen Yan, Colin White, Yash Savani, Frank Hutter. NeurIPS 2021. Surrogate NAS benchmarks

AutoML-Freiburg-Hannover 13 Nov 18, 2022
text_recognition_toolbox: The reimplementation of a series of classical scene text recognition papers with Pytorch in a uniform way.

text recognition toolbox 1. 项目介绍 该项目是基于pytorch深度学习框架,以统一的改写方式实现了以下6篇经典的文字识别论文,论文的详情如下。该项目会持续进行更新,欢迎大家提出问题以及对代码进行贡献。 模型 论文标题 发表年份 模型方法划分 CRNN 《An End-t

168 Dec 24, 2022
Adaptive Attention Span for Reinforcement Learning

Adaptive Transformers in RL Official implementation of Adaptive Transformers in RL In this work we replicate several results from Stabilizing Transfor

100 Nov 15, 2022
CTRL-C: Camera calibration TRansformer with Line-Classification

CTRL-C: Camera calibration TRansformer with Line-Classification This repository contains the official code and pretrained models for CTRL-C (Camera ca

57 Nov 14, 2022
Instance-wise Feature Importance in Time (FIT)

Instance-wise Feature Importance in Time (FIT) FIT is a framework for explaining time series perdiction models, by assigning feature importance to eve

Sana 46 Dec 25, 2022
Godot RL Agents is a fully Open Source packages that allows video game creators

Godot RL Agents The Godot RL Agents is a fully Open Source packages that allows video game creators, AI researchers and hobbiest the opportunity to le

Edward Beeching 326 Dec 30, 2022
Official implementation of "A Unified Objective for Novel Class Discovery", ICCV2021 (Oral)

A Unified Objective for Novel Class Discovery This is the official repository for the paper: A Unified Objective for Novel Class Discovery Enrico Fini

Enrico Fini 118 Dec 26, 2022
Rethinking the Importance of Implementation Tricks in Multi-Agent Reinforcement Learning

RIIT Our open-source code for RIIT: Rethinking the Importance of Implementation Tricks in Multi-AgentReinforcement Learning. We implement and standard

405 Jan 06, 2023
Project for tracking occupancy in Tel-Aviv parking lots.

Ahuzat Dibuk - Tracking occupancy in Tel-Aviv parking lots main.py This module was set-up to be executed on Google Cloud Platform. I run it every 15 m

Geva Kipper 35 Nov 22, 2022
Data, notebooks, and articles associated with the RSNA AI Deep Learning Lab at RSNA 2021

RSNA AI Deep Learning Lab 2021 Intro Welcome Deep Learners! This document provides all the information you need to participate in the RSNA AI Deep Lea

RSNA 65 Dec 16, 2022
PyTorch Autoencoders - Implementing a Variational Autoencoder (VAE) Series in Pytorch.

PyTorch Autoencoders Implementing a Variational Autoencoder (VAE) Series in Pytorch. Inspired by this repository Model List check model paper conferen

Subin An 8 Nov 21, 2022
Official implementation for (Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching, AAAI-2021)

Show, Attend and Distill: Knowledge Distillation via Attention-based Feature Matching Official pytorch implementation of "Show, Attend and Distill: Kn

Clova AI Research 80 Dec 16, 2022
Object DGCNN and DETR3D, Our implementations are built on top of MMdetection3D.

This repo contains the implementations of Object DGCNN (https://arxiv.org/abs/2110.06923) and DETR3D (https://arxiv.org/abs/2110.06922). Our implementations are built on top of MMdetection3D.

Wang, Yue 539 Jan 07, 2023
For IBM Quantum Challenge Africa 2021, 9 September (07:00 UTC) - 20 September (23:00 UTC).

IBM Quantum Challenge Africa 2021 To ensure Africa is able to apply quantum computing to solve problems relevant to the continent, the IBM Research La

Qiskit Community 48 Dec 25, 2022
Python package facilitating the use of Bayesian Deep Learning methods with Variational Inference for PyTorch

PyVarInf PyVarInf provides facilities to easily train your PyTorch neural network models using variational inference. Bayesian Deep Learning with Vari

342 Dec 02, 2022
RMTD: Robust Moving Target Defence Against False Data Injection Attacks in Power Grids

RMTD: Robust Moving Target Defence Against False Data Injection Attacks in Power Grids Real-time detection performance. This repo contains the code an

0 Nov 10, 2021
Whisper is a file-based time-series database format for Graphite.

Whisper Overview Whisper is one of three components within the Graphite project: Graphite-Web, a Django-based web application that renders graphs and

Graphite Project 1.2k Dec 25, 2022
OneShot Learning-based hotword detection.

EfficientWord-Net Hotword detection based on one-shot learning Home assistants require special phrases called hotwords to get activated (eg:"ok google

ANT-BRaiN 102 Dec 25, 2022
MlTr: Multi-label Classification with Transformer

MlTr: Multi-label Classification with Transformer This is official implement of "MlTr: Multi-label Classification with Transformer". Abstract The task

程星 38 Nov 08, 2022
M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images

M2MRF: Many-to-Many Reassembly of Features for Tiny Lesion Segmentation in Fundus Images This repo is the official implementation of paper "M2MRF: Man

12 Dec 14, 2022