[CVPR'22] COAP: Learning Compositional Occupancy of People

Related tags

Deep LearningCOAP
Overview

COAP: Compositional Articulated Occupancy of People

Paper | Video | Project Page

teaser figure

This is the official implementation of the CVPR 2022 paper COAP: Learning Compositional Occupancy of People.

Description

This repository provides the official implementation of an implicit human body model (COAP) which implements efficient loss terms for resolving self-intersection and collisions with 3D geometries.

Installation

The necessary requirements are specified in the requrements.txt file. To install COAP, execute:

pip install git+https://github.com/markomih/COAP.git

Note that Pytorch3D may require manuall installation (see instructions here). Alternatively, we provide a conda environment file to install the dependences:

conda env create -f environment.yml
conda activate coap
pip install git+https://github.com/markomih/COAP.git

Optional Dependencies

Install the pyrender package to use the visualization/tutorial scripts and follow the additional instructions specified here if you wish to retrain COAP.

Tutorials

COAP extends the interface of the SMPL-X package (follow its instructions for the usage) via two volumetric loss terms: 1) a loss for resolving self-intersections and 2) a loss for resolving collisions with 3D geometries flexibly represented as point clouds. In the following, we provide a minimal interface to access the COAP's functionalities:

import smplx
from coap import attach_coap

# create a SMPL body and extend the SMPL body via COAP (we support: smpl, smplh, and smplx model types)
model = smplx.create(**smpl_parameters)
attach_coap(model)

smpl_output = model(**smpl_data)  # smpl forward pass
# NOTE: make sure that smpl_output contains the valid SMPL variables (pose parameters, joints, and vertices). 
assert model.joint_mapper is None, 'COAP requires valid SMPL joints as input'

# access two loss functions
model.coap.selfpen_loss(smpl_output)  # self-intersections
model.coap.collision_loss(smpl_output, scan_point_cloud)  # collisions with other geometris

Additionally, we provide two tutorials on how to use these terms to resolve self-intersections and collisions with the environment.

Pretrained Models

A respective pretrained model will be automatically fetched and loaded. All the pretrained models are available on the dev branch inside the ./models directory.

Citation

@inproceedings{Mihajlovic:CVPR:2022,
   title = {{COAP}: Compositional Articulated Occupancy of People},
   author = {Mihajlovic, Marko and Saito, Shunsuke and Bansal, Aayush and Zollhoefer, Michael and Tang, Siyu},
   booktitle = {Proceedings IEEE Conf. on Computer Vision and Pattern Recognition (CVPR)},
   month = jun,
   year = {2022}
}

Contact

For questions, please contact Marko Mihajlovic ([email protected]) or raise an issue on GitHub.

Owner
Marko Mihajlovic
PhD Student in Computer Vision and Machine Learning at ETH Zurich
Marko Mihajlovic
Rest API Written In Python To Classify NSFW Images.

Rest API Written In Python To Classify NSFW Images.

Wahyusaputra 2 Dec 23, 2021
The Easy-to-use Dialogue Response Selection Toolkit for Researchers

Easy-to-use toolkit for retrieval-based Chatbot Recent Activity Our released RRS corpus can be found here. Our released BERT-FP post-training checkpoi

GMFTBY 32 Nov 13, 2022
BarcodeRattler - A Raspberry Pi Powered Barcode Reader to load a game on the Mister FPGA using MBC

Barcode Rattler A Raspberry Pi Powered Barcode Reader to load a game on the Mist

Chrissy 29 Oct 31, 2022
Spatial-Temporal Transformer for Dynamic Scene Graph Generation, ICCV2021

Spatial-Temporal Transformer for Dynamic Scene Graph Generation Pytorch Implementation of our paper Spatial-Temporal Transformer for Dynamic Scene Gra

Yuren Cong 119 Jan 01, 2023
Cross-view Transformers for real-time Map-view Semantic Segmentation (CVPR 2022 Oral)

Cross View Transformers This repository contains the source code and data for our paper: Cross-view Transformers for real-time Map-view Semantic Segme

Brady Zhou 363 Dec 25, 2022
SMIS - Semantically Multi-modal Image Synthesis(CVPR 2020)

Semantically Multi-modal Image Synthesis Project page / Paper / Demo Semantically Multi-modal Image Synthesis(CVPR2020). Zhen Zhu, Zhiliang Xu, Anshen

316 Dec 01, 2022
Rethinking Transformer-based Set Prediction for Object Detection

Rethinking Transformer-based Set Prediction for Object Detection Here are the code for the ICCV paper. The code is adapted from Detectron2 and AdelaiD

Zhiqing Sun 62 Dec 03, 2022
Pytorch implementation for DFN: Distributed Feedback Network for Single-Image Deraining.

DFN:Distributed Feedback Network for Single-Image Deraining Abstract Recently, deep convolutional neural networks have achieved great success for sing

6 Nov 05, 2022
[NeurIPS 2021] Low-Rank Subspaces in GANs

Low-Rank Subspaces in GANs Figure: Image editing results using LowRankGAN on StyleGAN2 (first three columns) and BigGAN (last column). Low-Rank Subspa

112 Dec 28, 2022
Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

Daniele Grattarola 2.2k Jan 08, 2023
This repo is the code release of EMNLP 2021 conference paper "Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories".

Connect-the-Dots: Bridging Semantics between Words and Definitions via Aligning Word Sense Inventories This repo is the code release of EMNLP 2021 con

12 Nov 22, 2022
A collection of semantic image segmentation models implemented in TensorFlow

A collection of semantic image segmentation models implemented in TensorFlow. Contains data-loaders for the generic and medical benchmark datasets.

bobby 16 Dec 06, 2019
Keras implementation of Real-Time Semantic Segmentation on High-Resolution Images

Keras-ICNet [paper] Keras implementation of Real-Time Semantic Segmentation on High-Resolution Images. Training in progress! Requisites Python 3.6.3 K

Aitor Ruano 87 Dec 16, 2022
A PyTorch implementation of "SimGNN: A Neural Network Approach to Fast Graph Similarity Computation" (WSDM 2019).

SimGNN ⠀⠀⠀ A PyTorch implementation of SimGNN: A Neural Network Approach to Fast Graph Similarity Computation (WSDM 2019). Abstract Graph similarity s

Benedek Rozemberczki 534 Dec 25, 2022
Simple image captioning model - CLIP prefix captioning.

Simple image captioning model - CLIP prefix captioning.

688 Jan 04, 2023
Efficient 3D human pose estimation in video using 2D keypoint trajectories

3D human pose estimation in video with temporal convolutions and semi-supervised training This is the implementation of the approach described in the

Meta Research 3.1k Dec 29, 2022
A framework for the elicitation, specification, formalization and understanding of requirements.

A framework for the elicitation, specification, formalization and understanding of requirements.

NASA - Software V&V 161 Jan 03, 2023
Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

eXtreme Gradient Boosting Community | Documentation | Resources | Contributors | Release Notes XGBoost is an optimized distributed gradient boosting l

Distributed (Deep) Machine Learning Community 23.6k Dec 31, 2022
Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations

Seeing All the Angles: Learning Multiview Manipulation Policies for Contact-Rich Tasks from Demonstrations Trevor Ablett, Daniel (Yifan) Zhai, Jonatha

STARS Laboratory 3 Feb 01, 2022
Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

108 Dec 01, 2022