Final project in KAIST AI class

Overview

mmodal_mixer

MLP-Mixer based Multi-modal image-text retrieval

Image:

Original image is cropped with 16 x 16 patch size without overlap. Then, it is reshaped to (batch, (hxw), (patch x patch x channel)).

Text:

Also, original text is tokenized and embedded with BERT-based approach (BERT-base-uncased).

Data processing:

When we train our model, we randomly samples(50 %) reports to make the matched- and un-matched image-text set. Basically, matched and un-matched set is classified with label information using chexpert labeler, we consider unmatched set when randomly sampled report is not exactly same with original one.

Mixer based approach is trained efficiently with xxxx throuput with xxx accuracy.

Exp settings.

batch: 256 batch epoch: 50 epoch

Chest X-ray Image-reports retrieval

Model spec: patch size:16, embedding dim: 768

Input spec: img size: 224x224x3 -> pathch size: (224/16) x (224/16) text max len: 128 legth

input embedding: cls, txt, sep, img output: matched or unmatched

Results

Max memory: 15.903G

Model forward time: Model backward time:

Owner
SuperSuperMoon
PhD student at Graduate School of AI, KAIST. Previously AI research engineer at Samsung Medical Center. Medical AI. Computer Vision & NLP.
SuperSuperMoon
LibreMind is a free meditation app made in under 24 hours. It has various meditation, breathwork, and visualization exercises.

libreMind Meditation exercises What is it? LibreMind is a free meditation app made in under 24 hours. It has various meditation, breathwork, and visua

1 May 24, 2022
A set of decks and notebooks with exercises for use in a hands-on causal inference tutorial session

intro-to-causal-inference A introduction to causal inference using common tools from the python data stack Table of Contents Getting Started Install g

Roni Kobrosly 15 Dec 07, 2022
This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets.

SaleaePacketParser This is a Saleae Logic custom high level analyzer that allows you to search and mark specific packets. Field "Search For" is used f

1 Dec 16, 2021
This Curve Editor, written by Jehee Lee in 2015

Splines Abstract This Curve Editor, written by Jehee Lee in 2015, is a freeware. You can use, modify, redistribute the code without restriction. This

Movement Research Lab 8 Mar 11, 2022
A tool for removing PUPs using signatures

Unwanted program removal tool A tool for removing PUPs using signatures What is the unwanted program removal tool? The unwanted program removal tool i

4 Sep 20, 2022
🤖🧭Creates google-like navigation menu using python-telegram-bot wrapper

python telegram bot menu pagination Makes a google style pagination line for a list of items. In other words it builds a menu for navigation if you ha

Sergey Smirnov 9 Nov 27, 2022
👀 nothing to see here

Woofy Woofy is blue dog companion token of YFI (Wifey) It utilizes a special Woof bonding curve which allows two-way conversion between the tokens. Th

Yearn Finance 36 Mar 14, 2022
A simple but fully functional calculator that will take multiple operations.

Functional-Calculator A simple but fully functional calculator that will take multiple operations. Usage Run the following command through terminal: p

Uzziel Ariel 1 Dec 22, 2022
Check a discord message and give it a percentage of scamminess

scamChecker Check a discord message and give it a percentage of scamminess Run the bot, and run the command !scamCheck and it will return a percentage

3 Sep 22, 2022
K2HASH Python library - NoSQL Key Value Store(KVS) library

k2hash_python Overview k2hash_python is an official python driver for k2hash. Install Firstly you must install the k2hash shared library: curl -o- htt

Yahoo! JAPAN 3 Oct 19, 2022
Weakly-Divisable - Takes an interger and seee if it is weakly divisible by seven

Weakly Divisble Project by Diana Arce-Hernandez, Ryan McAlpine, and Rommel Ravan

Diana Arce-Hernandez 1 Jan 12, 2022
A multi-platform fuzzer for poking at userland binaries and servers

litefuzz A multi-platform fuzzer for poking at userland binaries and servers litefuzz intro why how it works what it does what it doesn't do support p

52 Nov 18, 2022
The official Repository wherein newbies into Open Source can Contribute during the Hacktoberfest 2021

Hacktoberfest 2021 Get Started With your first Contrinution/Pull Request : Fork/Copy the repo by clicking the right most button on top of the page. Go

HacOkars 25 Aug 20, 2022
Animations made using manim-ce

ManimCE Animations Animations made using manim-ce The code turned out to be a bit complicated than expected.. It can be greatly simplified, will work

sparshg 17 Jan 06, 2023
TallerStereoVision Convencion Python Chile 2021

TallerStereoVision Convencion Python Chile 2021 Taller Stereo Vision & Python PyCon.cl 2021 Instalación Se recomienta utilizar Virtual Environment pyt

2 Oct 20, 2022
Python script to automate the change of desktop background

wallomator Python script to automate the change of desktop background A python script that automates the process of changing the desktop background. I

Mohammed Haaris Javed 10 Jun 16, 2022
A simple wrapper for joy library

Joy CodeGround A simple wrapper for joy library to render joy sketches in browser using vs code, (or in other words, for those who are allergic to Jup

rijfas 9 Sep 08, 2022
My tools box script for sigma

sigma_python_toolbox My tools box script for sigma purpose My goal is not to replace sigma but to put at disposal the scripts that I think to help me

4 Jun 20, 2022
A Python version of Canvacord

A copy of canvacord made in python! Installation Run any of these commands in terminal: Mac / Linux pip install canvacord Windows python -m pip insta

10 Mar 28, 2022
Python MapReduce library written in Cython.

Python MapReduce library written in Cython. Visit us in #hadoopy on freenode. See the link below for documentation and tutorials.

Brandyn White 243 Sep 16, 2022