✂️ EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video.

Overview

EyeLipCropper

EyeLipCropper is a Python tool to crop eyes and mouth ROIs of the given video. The whole process consists of three parts: frame extraction, face alignment, and eye/mouth cropping. The cropped eye/mouth image size can be customized.

vis

Usage

Prerequisites

>>> pip install -r requirements.txt

1. Extract frames of a given video

>>> python frame_extract.py -h
usage: frame_extract.py [-h] [--video-path VIDEO_PATH] [--images-path IMAGES_PATH]

extract frames with opencv

optional arguments:
  -h, --help            show this help message and exit
  --video-path VIDEO_PATH
                        the input video path
  --images-path IMAGES_PATH
                        the output frames path
 
# default for test: this will generate frames of the video in `./test/images`
>>> python frame_extract.py

2. Align faces of the frames, with library face-alignment

>>> python face_align.py -h
usage: face_align.py [-h] [--images-path IMAGES_PATH] [--landmarks-path LANDMARKS_PATH] [--boxes-path BOXES_PATH] [--device DEVICE] [--log-path LOG_PATH]

align faces with `https://github.com/1adrianb/face-alignment`

optional arguments:
  -h, --help            show this help message and exit
  --images-path IMAGES_PATH
                        the input frames path
  --landmarks-path LANDMARKS_PATH
                        the output 68 landmarks path
  --boxes-path BOXES_PATH
                        the output bounding boxes path
  --device DEVICE       cpu or gpu cuda device
  --log-path LOG_PATH   logging when there are no faces detected
  
# default for test: this will generate landmarks and bounding boxes in
# `./test/landmarks` and `./test/boxes`
>>> python face_align.py

3. Crop left eye, right eye, mouth ROIs, with code modified from processing tools of [Eye] RT-GENE and [Mouth] LipForensics

>>> python eye_mouth_crop.py -h
usage: eye_mouth_crop.py [-h] [--images-path IMAGES_PATH] [--landmarks-path LANDMARKS_PATH] [--boxes-path BOXES_PATH] [--eye-width EYE_WIDTH] [--eye-height EYE_HEIGHT]
                         [--face-roi-width FACE_ROI_WIDTH] [--face-roi-height FACE_ROI_HEIGHT] [--left-eye-path LEFT_EYE_PATH] [--right-eye-path RIGHT_EYE_PATH]
                         [--mean-face MEAN_FACE] [--mouth-width MOUTH_WIDTH] [--mouth-height MOUTH_HEIGHT] [--start-idx START_IDX] [--stop-idx STOP_IDX]
                         [--window-margin WINDOW_MARGIN] [--mouth-path MOUTH_PATH]

crop eye and mouth regions

optional arguments:
  -h, --help            show this help message and exit
  --images-path IMAGES_PATH
                        [COMMON] the input frames path
  --landmarks-path LANDMARKS_PATH
                        [COMMON] the input 68 landmarks path
  --boxes-path BOXES_PATH
                        [EYE] the input bounding boxes path
  --eye-width EYE_WIDTH
                        [EYE] width of cropped eye ROIs
  --eye-height EYE_HEIGHT
                        [EYE] height of cropped eye ROIs
  --face-roi-width FACE_ROI_WIDTH
                        [EYE] maximize this argument until there is a warning message
  --face-roi-height FACE_ROI_HEIGHT
                        [EYE] maximize this argument until there is a warning message
  --left-eye-path LEFT_EYE_PATH
                        [EYE] the output left eye images path
  --right-eye-path RIGHT_EYE_PATH
                        [EYE] the output right eye images path
  --mean-face MEAN_FACE
                        [MOUTH] mean face pathname
  --mouth-width MOUTH_WIDTH
                        [MOUTH] width of cropped mouth ROIs
  --mouth-height MOUTH_HEIGHT
                        [MOUTH] height of cropped mouth ROIs
  --start-idx START_IDX
                        [MOUTH] start of landmark index for mouth
  --stop-idx STOP_IDX   [MOUTH] end of landmark index for mouth
  --window-margin WINDOW_MARGIN
                        [MOUTH] window margin for smoothed_landmarks
  --mouth-path MOUTH_PATH
                        [MOUTH] the output mouth images path

# default for test: this will generate the final cropped left eye,
# right eye, and mouth images in `./test/left_eye`, `./test/right_eye`
# , and `./test/mouth`
>>> python eye_mouth_crop.py
  • Note that the argument --face-roi-width and --face-roi-height should be maximized until there is a printed warning.

License

GPL-3.0 License

Reference

[1] Bulat, Adrian, and Georgios Tzimiropoulos. "How far are we from solving the 2d & 3d face alignment problem?(and a dataset of 230,000 3d facial landmarks)." Proceedings of the IEEE International Conference on Computer Vision (ICCV). 2017. GitHub: https://github.com/1adrianb/face-alignment

[2] Fischer, Tobias, Hyung Jin Chang, and Yiannis Demiris. "Rt-gene: Real-time eye gaze estimation in natural environments." Proceedings of the European Conference on Computer Vision (ECCV). 2018. GitHub: https://github.com/Tobias-Fischer/rt_gene

[3] Haliassos, Alexandros, et al. "Lips Don't Lie: A Generalisable and Robust Approach To Face Forgery Detection." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2021. GitHub: https://github.com/ahaliassos/LipForensics/

Owner
Zi-Han Liu
Senior @ SJTU
Zi-Han Liu
some academic posters as references. May we have in-person poster session soon!

some academic posters as references. May we have in-person poster session soon!

Bolei Zhou 472 Jan 06, 2023
Real-CUGAN - Real Cascade U-Nets for Anime Image Super Resolution

Real Cascade U-Nets for Anime Image Super Resolution 中文 | English 🔥 Real-CUGAN

tarsin 111 Dec 28, 2022
This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021.

Off-Belief Learning Introduction This repo contains the implementation of the algorithm proposed in Off-Belief Learning, ICML 2021. Environment Setup

Facebook Research 32 Jan 05, 2023
Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)

Junction Tree Variational Autoencoder for Molecular Graph Generation Official implementation of our Junction Tree Variational Autoencoder https://arxi

Wengong Jin 418 Jan 07, 2023
Tooling for converting STAC metadata to ODC data model

手语识别 0、使用到的模型 (1). openpose,作者:CMU-Perceptual-Computing-Lab https://github.com/CMU-Perceptual-Computing-Lab/openpose (2). 图像分类classification,作者:Bubbl

Open Data Cube 65 Dec 20, 2022
The audio-video synchronization of MKV Container Format is exploited to achieve data hiding

The audio-video synchronization of MKV Container Format is exploited to achieve data hiding, where the hidden data can be utilized for various management purposes, including hyper-linking, annotation

Maxim Zaika 1 Nov 17, 2021
Code Repository for The Kaggle Book, Published by Packt Publishing

The Kaggle Book Data analysis and machine learning for competitive data science Code Repository for The Kaggle Book, Published by Packt Publishing "Lu

Packt 1.6k Jan 07, 2023
PyTorch implementation of our ICCV 2019 paper: Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

Impersonator PyTorch implementation of our ICCV 2019 paper: Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer an

SVIP Lab 1.7k Jan 06, 2023
Tool for live presentations using manim

manim-presentation Tool for live presentations using manim Install pip install manim-presentation opencv-python Usage Use the class Slide as your sce

Federico Galatolo 146 Jan 06, 2023
Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Explainable Fact Checking: A Survey This repository and the accompanying webpage contain resources for the paper "Explainable Fact Checking: A Survey"

Neema Kotonya 42 Nov 17, 2022
Robot Reinforcement Learning on the Constraint Manifold

Implementation of "Robot Reinforcement Learning on the Constraint Manifold"

31 Dec 05, 2022
YOLOX-Paddle - A reproduction of YOLOX by PaddlePaddle

YOLOX-Paddle A reproduction of YOLOX by PaddlePaddle 数据集准备 下载COCO数据集,准备为如下路径 /ho

QuanHao Guo 6 Dec 18, 2022
Simple ray intersection library similar to coldet - succedeed by libacc

Ray Intersection This project offers a header only acceleration structure library including implementations for a BVH- and KD-Tree. Applications may i

Nils Moehrle 29 Jun 23, 2022
A PyTorch implementation of "From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network" (ICCV2021)

From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network The official code of VisionLAN (ICCV2021). VisionLAN successfully a

81 Dec 12, 2022
Implements pytorch code for the Accelerated SGD algorithm.

AccSGD This is the code associated with Accelerated SGD algorithm used in the paper On the insufficiency of existing momentum schemes for Stochastic O

205 Jan 02, 2023
Pytorch implementation of the paper SPICE: Semantic Pseudo-labeling for Image Clustering

SPICE: Semantic Pseudo-labeling for Image Clustering By Chuang Niu and Ge Wang This is a Pytorch implementation of the paper. (In updating) SOTA on 5

Chuang Niu 154 Dec 15, 2022
Official code for "EagerMOT: 3D Multi-Object Tracking via Sensor Fusion" [ICRA 2021]

EagerMOT: 3D Multi-Object Tracking via Sensor Fusion Read our ICRA 2021 paper here. Check out the 3 minute video for the quick intro or the full prese

Aleksandr Kim 276 Dec 30, 2022
A template repository for submitting a job to the Slurm Cluster installed at the DISI - University of Bologna

Cluster di HPC con GPU per esperimenti di calcolo (draft version 1.0) Per poter utilizzare il cluster il primo passo è abilitare l'account istituziona

20 Dec 16, 2022
Code for our paper Aspect Sentiment Quad Prediction as Paraphrase Generation in EMNLP 2021.

Aspect Sentiment Quad Prediction (ASQP) This repo contains the annotated data and code for our paper Aspect Sentiment Quad Prediction as Paraphrase Ge

Isaac 39 Dec 11, 2022
Code for our paper 'Generalized Category Discovery'

Generalized Category Discovery This repo is a placeholder for code for our paper: Generalized Category Discovery Abstract: In this paper, we consider

107 Dec 28, 2022