Official implementation of the AAAI 2022 paper "Learning Token-based Representation for Image Retrieval"

Related tags

AuthenticationToken
Overview

Token: Token-based Representation for Image Retrieval

PyTorch training code for Token-based Representation for Image Retrieval. We propose a joint local feature learning and aggregation framework, obtaining 82.3 mAP on ROxf with Medium evaluation protocols. Inference in 50 lines of PyTorch.

Token

What it is. Given an image, Token first uses a CNN and a Local Feature Self-Attention (LFSA) module to extract local features $F_c$. Then, they are tokenized into $L$ visual tokens with spatial attention. Further, a refinement block is introduced to enhance the obtained visual tokens with self-attention and cross-attention. Finally, Token concatenates all the visual tokens to form a compact global representation $f_g$ and reduce its dimension. The aggreegated global feature is discriminative and efficient.

About the code. Token is very simple to implement and experiment with. Training code follows this idea - it is not a library, but simply a train.py importing model and criterion definitions with standard training loops.

mAP performance of the proposed model

We provide results of Token. mAP is computed with Medium and Hard evaluation protocols. model will come soon.

Token

Requirements

  • Python 3
  • cuda 11.0
  • PyTorch tested on 1.8.0, torchvision 0.9.0
  • numpy
  • matplotlib

Usage - Representation learning

There are no extra compiled components in Token and package dependencies are minimal, so the code is very simple to use. We provide instructions how to install dependencies via conda. Install PyTorch 1.8.0 and torchvision 0.9.0:

conda install -c pytorch pytorch torchvision

Data preparation

Before going further, please check out Google landmarkv2 github. We use their training images. If you use this code in your research, please also cite their work!

Download and extract Google landmarkv2 train and val images with annotations from https://github.com/cvdfoundation/google-landmark.

Download ROxf and RPar datastes with annotations. We expect the directory structure to be the following:

/data/
  ├─ Google-landmark-v2 # train images
  │   ├─ train.csv
  │   ├─ train_clean.csv
  │   ├─ GLDv2-clean-train-split.pkl
  │   ├─ GLDv2-clean-val-split.pkl
  |   └─ train
  └─test # test images
      ├─ roxford5k
      |   ├─ jpg
      |   └─ gnd_roxford5k.pkl
      └─ rparis6k
          ├─ jpg
          └─ gnd_rparis6k.pkl

Training

To train Token on a single node with 4 gpus for 30 epochs run:

sh experiment.sh

A single epoch takes 2.5 hours, so 30 epoch training takes around 3 days on a single machine with 4 3090Ti cards.

We train Token with SGD setting learning rate to 0.01. The refinement block is trained with dropout of 0.1, and linearly decaying scheduler is adopted to gradually decay the learning rate to 0 when the desired number of steps is reached.

Evaluation

To evaluate on Roxf and Rparis with a single GPU run:

python test.py

and get results as below

>> Test Dataset: roxford5k *** local aggregation >>
>> mAP Medium: 82.28, Hard: 66.57

>> Test Dataset: rparis6k *** local aggregation >>
>> mAP Medium: 89.34, Hard: 78.56

We found that there is a change in performance when the test environment is different, for example, when the environment is GeForce RTX 2080Ti with cuda 10.2, pytorch 1.7.1 and torchvision 0.8.2, the test performance is

>> Test Dataset: roxford5k *** local aggregation >>
>> mAP Medium: 81.36, Hard: 62.09

>> Test Dataset: rparis6k *** local aggregation >>
>> mAP Medium: 90.19, Hard: 80.16

Qualitative examples

Selected qualitative examples of different methods. Top-11 results are shown in the figure. The image with green denotes the true positives and the red bounding boxes are false positives.

Token

Owner
Hui Wu
Department of Electronic Engineering and Information Science University of Science and Technology of China
Hui Wu
Per object permissions for Django

django-guardian django-guardian is an implementation of per object permissions [1] on top of Django's authorization backend Documentation Online docum

3.3k Jan 01, 2023
Django Admin Two-Factor Authentication, allows you to login django admin with google authenticator.

Django Admin Two-Factor Authentication Django Admin Two-Factor Authentication, allows you to login django admin with google authenticator. Why Django

Iman Karimi 9 Dec 07, 2022
Quick and simple security for Flask applications

Note This project is non maintained anymore. Consider the Flask-Security-Too project as an alternative. Flask-Security It quickly adds security featur

Matt Wright 1.6k Dec 19, 2022
OAuthlib support for Python-Requests!

Requests-OAuthlib This project provides first-class OAuth library support for Requests. The OAuth 1 workflow OAuth 1 can seem overly complicated and i

1.6k Dec 28, 2022
This script helps you log in to your LMS account and enter the currently running session

This script helps you log in to your LMS account and enter the currently running session, all in a second

Ali Ebrahimi 5 Sep 01, 2022
API-key based security utilities for FastAPI, focused on simplicity of use

FastAPI simple security API key based security package for FastAPI, focused on simplicity of use: Full functionality out of the box, no configuration

Tolki 154 Jan 03, 2023
OAuth2 goodies for the Djangonauts!

Django OAuth Toolkit OAuth2 goodies for the Djangonauts! If you are facing one or more of the following: Your Django app exposes a web API you want to

Jazzband 2.7k Dec 31, 2022
Includes Automation and Personal Projects

Python Models, and Connect Forclient & OpenCv projects Completed Automation** Alarm (S

tushar malhan 1 Jan 15, 2022
Kube OpenID Connect is an application that can be used to easily enable authentication flows via OIDC for a kubernetes cluster

Kube OpenID Connect is an application that can be used to easily enable authentication flows via OIDC for a kubernetes cluster. Kubernetes supports OpenID Connect Tokens as a way to identify users wh

7 Nov 20, 2022
OpenStack Keystone auth plugin for HTTPie

httpie-keystone-auth OpenStack Keystone auth plugin for HTTPie. Installation $ pip install --upgrade httpie-keystone-auth You should now see keystone

Pavlo Shchelokovskyy 1 Oct 20, 2021
Brute force a JWT token. Script uses multithreading.

JWT BF Brute force a JWT token. Script uses multithreading. Tested on Kali Linux v2021.4 (64-bit). Made for educational purposes. I hope it will help!

Ivan Šincek 5 Dec 02, 2022
Python One-Time Password Library

PyOTP - The Python One-Time Password Library PyOTP is a Python library for generating and verifying one-time passwords. It can be used to implement tw

PyAuth 2.2k Dec 26, 2022
Mock authentication API that acceccpts email and password and returns authentication result.

Mock authentication API that acceccpts email and password and returns authentication result.

Herman Shpryhau 1 Feb 11, 2022
Simple implementation of authentication in projects using FastAPI

Fast Auth Facilita implementação de um sistema de autenticação básico e uso de uma sessão de banco de dados em projetos com tFastAPi. Instalação e con

3 Jan 08, 2022
OAuth2 goodies for the Djangonauts!

Django OAuth Toolkit OAuth2 goodies for the Djangonauts! If you are facing one or more of the following: Your Django app exposes a web API you want to

Jazzband 2.7k Jan 01, 2023
Easy and secure implementation of Azure AD for your FastAPI APIs 🔒 Single- and multi-tenant support.

Easy and secure implementation of Azure AD for your FastAPI APIs 🔒 Single- and multi-tenant support.

Intility 220 Jan 05, 2023
User Authentication in Flask using Flask-Login

User-Authentication-in-Flask Set up & Installation. 1 .Clone/Fork the git repo and create an environment Windows git clone https://github.com/Dev-Elie

ONDIEK ELIJAH OCHIENG 31 Dec 11, 2022
Spotify User Token Generator Template

Spotify User Token Generator Template Quick Start $ pip3 install -r requirements

Arda Soyer 1 Feb 01, 2022
FastAPI-Login tries to provide similar functionality as Flask-Login does.

FastAPI-Login FastAPI-Login tries to provide similar functionality as Flask-Login does. Installation $ pip install fastapi-login Usage To begin we hav

417 Jan 07, 2023
MikroTik Authentication POCs

Proofs of concept which successfully authenticate with MikroTik Winbox and MAC Telnet servers running on RouterOS version 6.45.1+

Margin Research 56 Dec 08, 2022