In this project, we develop a face recognize platform based on MTCNN object-detection netcwork and FaceNet self-supervised network.

Overview

模式识别大作业——人脸检测与识别平台

本项目是一个简易的人脸检测识别平台,提供了人脸信息录入人脸识别的功能。前端采用 html+css+js,后端采用 pytorch,前后端交互采用 flask。

0 环境依赖

flask==2.0.1
werkzeug==2.0.1
torch==1.10.1
torchvision==0.11.1
pillow==8.2.0

1 文件结构

image-20211121230548707

MTCNN_FaceNet:人脸检测算法接口

simplified:人脸识别算法接口

static:静态资源文件夹(包含数据库)

templates:前端Html框架

app.py:前后端交互flask框架

2 人脸识别算法——facenet

  • 一次性导入数据库:使用 face_in.py,请将数据库中每个人组织成单个文件夹的形式,如图

    image-20211121230548707

    • 格式为 python face_in.py -i -d
    • 样例输入:python face_in.py -i In_data -d dataset.json
    • 样例输出:在当前工作目录下生成(default)名为"dataset.json"的文件,即为数据库
    • 若为直接调用函数的话,传入包含上面两种信息的字典即可,输出不变
      • 即类似 {'image_path':<>, 'dataset_path':<>} 的参数
  • 添加单个人像:使用 face_append.py,格式为 python face_append.py -i -n -d

    • 样例输入:python face_append.py -i In_data/acatsa/acatsa.1.jpg -n acatsa -d dataset.json
    • 样例输出:修改指定的 dataset.json,向其中添加新的人脸数据
    • 若为直接调用函数的话,传入包含上面三种信息的字典即可,输出不变
      • 即类似 {'image_path':<>, 'dataset_path':<>, 'name':<>} 的参数
  • 从数据库中判别人脸:使用 classify_func.py,格式为 python classify_func.py -i -d

    • 样例输入: python classify_func.py -i In_data/acatsa/acatsa.1.jpg -d dataset.json
    • 样例输出:'acatsa'
    • 若为直接调用函数的话,传入包含上面三种信息的字典即可,输出不变
      • 即类似 {'image_path':<>, 'dataset_path':<>} 的参数
  • 剪切人脸 和 输出特征向量的 接口,见 interface.py 中的 mtcnn_single() 和 embedding_single() 函数

    • mtcnn_single()
      • 输入:字典,{'image_path':<>, 'save_path':< default:None >}
      • 输出:返回剪切后的图片,同时在 save_path 保存剪切后的图片
    • embedding_single()
      • 输入:字典, {'image_path':<>}
      • 输出:返回编码向量
  • 一键将图片库中人脸进行 mtcnn 剪裁,见 mtcnn_trans() 函数

    • 输入:字典,{'image_path':<>}

    • 输出:无返回值,剪裁后替换原有图片位置

    • 注意:需要图片库的组织形式如本文开头 face_in.py 的要求那样见 mtcnn_trans() 函数

  • classify_test() 函数

    • 输入:字典,{'img_path':<>, 'dataset_path':<>, 'origin_data':<>}
      • img_path,输入图片的路径位置
      • dataset_path,之前保存的数据 json
      • origin_data,图片的保存位置,即各个人脸的总保存位置
      • image-20211225172640828
      • 就像上面这样的话,origin_data = 'In_data'
    • 输出:
      • 若找到匹配的人脸。返回路径,示例:'In_data/acatsa/acatsa_1.jpg'
      • 若未找到,返回字符串 'no matched people'

3 人脸检测算法——mtcnn

4 平台使用

本平台采用flask框架搭建,运行时,在flask_FC文件夹下打开终端,运行如下指令:

python -m flask run

在浏览器中输入网址 http://127.0.0.1:5000/

前端设置了两个接口,分别进行信息录入人脸截图识别。将新录入的人脸图片传入后端,可利用mtcnn算法进行人脸检测,在数据库中加入该用户的人脸信息;将视频流截图后的图片传入后端,可利用facenet算法进行人脸识别,在后台数据库中信息匹配,返回识别成功或错误信息。

image-20211225172640828

4.1 人脸信息录入

form表单将文件流传入后端 —— mtcnn接口检测人脸 —— DataBase中更新图片信息 —— dataset.json中更新编码信息 —— 检测人脸图片返回前端

aaa.html

">
<form action="/" id="uploadForm" method="post" enctype="multipart/form-data" >
	<button class="btn btn-danger" type="submit" >
      <h3>Enter Photo to experienceh3> 
    button>
	<input type="file" name="photo">
form>

app.py

@app.route('/', methods=['GET', 'POST'])
def upinfo():
    if request.method == 'POST':
        if request.files.get('photo'):
            # 创建文件夹,保存录入图片
            photo = request.files.get('photo')
            basepath = os.path.dirname(__file__)
            filename = secure_filename(photo.filename)
            uploadpath = os.path.join(basepath, 'static/DataBase', filename[:-4], filename)
            path = os.path.join(basepath, 'static/DataBase', filename[:-4])
            if not path:
                os.makedirs(path)

            Reshape = transforms.Resize((160, 160))
            trans = transforms.Compose([Reshape])
            img = trans(tojpg(Image.open(photo)))
            save_path = uploadpath
            newphoto = mtcnn_single(img, save_path=save_path)

            # 更新dataset.json
            args = {'image_path': uploadpath, "dataset_path": 'static/face_dataset.json', 'name': filename[:-4]}
            face_append(args)
            return render_template('aaa.html', output='DataBase/' + filename[:-4] + '/' + filename)

    return render_template('aaa.html')

4.2 视频流截图检测

前端视频流截图传入后端 —— facenet接口识别人脸 —— 后端数据库匹配 —— 返回数据库已录入图片(匹配成功)/返回失败信息

aaa.html

">
<video id="myVideo" autoplay>video>
			<script>

				let v = document.getElementById("myVideo");

				//create a canvas to grab an image for upload
				let imageCanvas = document.createElement('canvas');
				let imageCtx = imageCanvas.getContext("2d");

				//Add file blob to a form and post
				function postFile(file) {
					let formdata = new FormData();
					formdata.append("image", file);
					let xhr = new XMLHttpRequest();
					xhr.open('POST', 'http://localhost:5000/', true);
					xhr.onload = function () {
						if (this.status === 200){
							var path = JSON.parse(this.response)['path']
							console.log(this.response['path']);
							$('#img').attr('src',path);
						}
						else
							console.error(xhr);
					};
					xhr.send(formdata);
				}

				//Get the image from the canvas
				function sendImagefromCanvas() {

					//Make sure the canvas is set to the current video size
					imageCanvas.width = v.videoWidth;
					imageCanvas.height = v.videoHeight;

					imageCtx.drawImage(v, 0, 0, v.videoWidth, v.videoHeight);

					//Convert the canvas to blob and post the file
					imageCanvas.toBlob(postFile, 'image/jpeg');
				}

				//Take a picture on click
				v.onclick = function() {
					console.log('click');
					sendImagefromCanvas();
				};

				window.onload = function () {

					//Get camera video
					navigator.mediaDevices.getUserMedia({video: {width: 640, height: 360}, audio: false})
						.then(stream => {
							v.srcObject = stream;
						})
						.catch(err => {
							console.log('navigator.getUserMedia error: ', err)
						});

				};

			script>

app.py

@app.route('/', methods=['GET', 'POST'])
def upinfo():
    if request.method == 'POST':
        if request.files['image']:
            photo = request.files['image']
            basepath = os.path.dirname(__file__)
            filename = secure_filename(photo.filename)
            uploadpath = os.path.join(basepath, 'static/screenshot', filename)
            photo.save(uploadpath + '.jpg')

            Reshape = transforms.Resize((160, 160))
            trans = transforms.Compose([Reshape])
            img = trans(tojpg(Image.open(photo)))
            save_path = 'static/recognized_screenshot/' + "recognized_" + filename + '.jpg'
            newphoto = mtcnn_single(img, save_path=save_path)

            uploadpath = os.path.join(basepath, 'static/recognized_screenshot', 'recognized_'+filename)
            args = {'img_path': uploadpath + '.jpg', 'dataset_path': 'static/face_dataset.json',
                    'origin_data': 'static/DataBase'}
            out = classify_test(args)
            if out != "no matched people":
                print("数据库存储路径:" + out)
                print("识别成功!")
            else:
                print(out)
                print("数据库中不存在该人脸信息!")

            return {'path': out}

    return render_template('aaa.html')
Owner
Xuhua Huang
Xuhua Huang
Deep Learning applied to Integral data analysis

DeepIntegralCompton Deep Learning applied to Integral data analysis Module installation Move to the root directory of the project and execute : pip in

Thomas Vuillaume 1 Dec 10, 2021
TLoL (Python Module) - League of Legends Deep Learning AI (Research and Development)

TLoL-py - League of Legends Deep Learning Library TLoL-py is the Python component of the TLoL League of Legends deep learning library. It provides a s

7 Nov 29, 2022
Code for the paper Progressive Pose Attention for Person Image Generation in CVPR19 (Oral).

Pose-Transfer Code for the paper Progressive Pose Attention for Person Image Generation in CVPR19(Oral). The paper is available here. Video generation

Tengteng Huang 679 Jan 04, 2023
Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

Nils Thuerey 1.3k Jan 08, 2023
Pose estimation for iOS and android using TensorFlow 2.0

💃 Mobile 2D Single Person (Or Your Own Object) Pose Estimation for TensorFlow 2.0 This repository is forked from edvardHua/PoseEstimationForMobile wh

tucan9389 165 Nov 16, 2022
This repository contains the code used to quantitatively evaluate counterfactual examples in the associated paper.

On Quantitative Evaluations of Counterfactuals Install To install required packages with conda, run the following command: conda env create -f requi

Frederik Hvilshøj 1 Jan 16, 2022
RIM: Reliable Influence-based Active Learning on Graphs.

RIM: Reliable Influence-based Active Learning on Graphs. This repository is the official implementation of RIM. Requirements To install requirements:

Wentao Zhang 4 Aug 29, 2022
Transfer Learning library for Deep Neural Networks.

Transfer and meta-learning in Python Each folder in this repository corresponds to a method or tool for transfer/meta-learning. xfer-ml is a standalon

Amazon 245 Dec 08, 2022
Official PyTorch code for "BAM: Bottleneck Attention Module (BMVC2018)" and "CBAM: Convolutional Block Attention Module (ECCV2018)"

BAM and CBAM Official PyTorch code for "BAM: Bottleneck Attention Module (BMVC2018)" and "CBAM: Convolutional Block Attention Module (ECCV2018)" Updat

Jongchan Park 1.7k Jan 01, 2023
The Official Repository for "Generalized OOD Detection: A Survey"

Generalized Out-of-Distribution Detection: A Survey 1. Overview This repository is with our survey paper: Title: Generalized Out-of-Distribution Detec

Jingkang Yang 338 Jan 03, 2023
Implementation of ConvMixer-Patches Are All You Need? in TensorFlow and Keras

Patches Are All You Need? - ConvMixer ConvMixer, an extremely simple model that is similar in spirit to the ViT and the even-more-basic MLP-Mixer in t

Sayan Nath 8 Oct 03, 2022
Learned model to estimate number of distinct values (NDV) of a population using a small sample.

Learned NDV estimator Learned model to estimate number of distinct values (NDV) of a population using a small sample. The model approximates the maxim

2 Nov 21, 2022
Pytorch implementation of FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks

flownet2-pytorch Pytorch implementation of FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. Multiple GPU training is supported, a

NVIDIA Corporation 2.8k Dec 27, 2022
Python codes for Lite Audio-Visual Speech Enhancement.

Lite Audio-Visual Speech Enhancement (Interspeech 2020) Introduction This is the PyTorch implementation of Lite Audio-Visual Speech Enhancement (LAVSE

Shang-Yi Chuang 85 Dec 01, 2022
Blender Add-on that sets a Material's Base Color to one of Pantone's Colors of the Year

Blender PCOY (Pantone Color of the Year) MCMC (Mid-Century Modern Colors) HG71 (House & Garden Colors 1971) Blender Add-ons That Assign a Custom Color

Don Schnitzius 15 Nov 20, 2022
:hot_pepper: R²SQL: "Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing." (AAAI 2021)

R²SQL The PyTorch implementation of paper Dynamic Hybrid Relation Network for Cross-Domain Context-Dependent Semantic Parsing. (AAAI 2021) Requirement

huybery 60 Dec 31, 2022
FSL-Mate: A collection of resources for few-shot learning (FSL).

FSL-Mate is a collection of resources for few-shot learning (FSL). In particular, FSL-Mate currently contains FewShotPapers: a paper list which tracks

Yaqing Wang 1.5k Jan 08, 2023
Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals

LapDepth-release This repository is a Pytorch implementation of the paper "Monocular Depth Estimation Using Laplacian Pyramid-Based Depth Residuals" M

Minsoo Song 205 Dec 30, 2022
An end-to-end machine learning library to directly optimize AUC loss

LibAUC An end-to-end machine learning library for AUC optimization. Why LibAUC? Deep AUC Maximization (DAM) is a paradigm for learning a deep neural n

Andrew 75 Dec 12, 2022
这是一个yolox-keras的源码,可以用于训练自己的模型。

YOLOX:You Only Look Once目标检测模型在Keras当中的实现 目录 性能情况 Performance 实现的内容 Achievement 所需环境 Environment 小技巧的设置 TricksSet 文件下载 Download 训练步骤 How2train 预测步骤 Ho

Bubbliiiing 64 Nov 10, 2022