An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Last update: Dec 14, 2022

Related tags

Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
PointConv: Deep Convolutional Networks on 3D Point Clouds

Comments

which version should the `finetuner` module be?
I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

from finetuner.tuner.callback import BestModelCheckpoint from finetuner.tuner.pytorch.losses import TripletLoss from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.
opened by elenacliu 4
fix: fix previous structure

I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

opened by elenacliu 2
Add dataset, preprocess and pointnet2

Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

@numb3r3 @jemmyshin

opened by elenacliu 1
In `farthest_point_sample` function, the first centroid sampled is not random.

I'm reading your codes.

https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

opened by elenacliu 1
Provide diffusion encoder

https://github.com/nmwsharp/diffusion-net

I'm unfamiliar with the jina ecosystem. So this is my first issue.

Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

opened by fire 2

Releases(v0.3.0)

v0.3.0(May 6, 2022)

Source code(tar.gz)
Source code(zip)
v0.2.0(Feb 21, 2022)

Source code(tar.gz)
Source code(zip)

Owner

Jina AI

A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.

GitHub Repository

Instagram-like image filters.

PyGram Instagram-like image filters. Usage First, import the client: from filters import * Instanciate a filter and apply it: f = Nashville("image.jp

0 Oct 18, 2022

An application that maps an image of a LaTeX math equation to LaTeX code.

Convert images of LaTex math equations into LaTex code.

1.3k Jan 06, 2023

Conversion of Image, video, text into ASCII format

asciju Python package that converts image to ascii Free software: MIT license

11 Aug 22, 2022

NFT collection generator. Generates layered images

NFT collection generator Generates layered images, whole collections. Provides additional functionality. Repository includes three scripts generate.py

10 Nov 15, 2022

Extracts dominating colors from an image and presents them as a palette.

ColorPalette A simple web app to extract dominant colors from an image. Demo Live View it live at : https://colorpalettedemo.herokuapp.com/ You can de

214 Dec 29, 2022

Pythonocc nodes for Ryven

Pythonocc-nodes-for-Ryven Pythonocc nodes for Ryven Here a way to work on Pythonocc with a node editor, Ryven in that case. To get it functional you w

30 Dec 18, 2022

Extract the ISO 11146 beam size from an image file

laserbeamsize Simple and fast calculation of beam sizes from a single monochrome image based on the ISO 11146 method of variances. Some effort has bee

21 Jan 06, 2023

Simple Python / ImageMagick script to package images into WAD3s for use as GoldSrc textures.

WADs Out For [The] Ladies Simple Python / ImageMagick script to package images into WAD3s for use as GoldSrc textures. Development mostly focused on L

5 Apr 09, 2022

Blue noise image stippling in Processing (requires GPU)

141 Oct 09, 2022

Hello, this project is an example of how to generate a QR Code using python 😁

2 Oct 12, 2021

Photini - A free, easy to use, digital photograph metadata (Exif, IPTC, XMP) editing application for Linux, Windows and MacOS.

A free, easy to use, digital photograph metadata (Exif, IPTC, XMP) editing application for Linux, Windows and MacOS. "Metadata" is said to mea

120 Dec 20, 2022

Alternate Python bindings for the Open Asset Import Library (ASSIMP)

Impasse A simple Python wrapper for assimp using cffi to access the library. Requires Python = 3.7. It's a fork of PyAssimp, Assimp's official Python

3 Sep 26, 2022

PyLibTiff - a wrapper to the libtiff library to Python using ctypes

PyLibTiff is a package that provides: a wrapper to the libtiff library to Python using ctypes. a pure Python module for reading and writing TIFF and L

105 Dec 21, 2022

Generate different types of random avatars.

avatar-generator Generate different types of random avatars. Requirements Python3 pytorch=1.6 cv2=3.4 tqdm 1. Github-like avatars python generate_gi

11 Apr 02, 2022

This is an app that allows users to upload photos and display and store the photos in a file until the user deletes them.

Qt Photo App This is an app that allows users to upload photos and display and store the photos in a file until the user deletes them. Setup python3 -

5 Jan 22, 2022

🎨 Generate and change color-schemes on the fly.

Generate and change color-schemes on the fly. Pywal is a tool that generates a color palette from the dominant colors in an image. It then applies the

6.9k Jan 03, 2023

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

img2dataset Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine. Also supports

1.4k Jan 01, 2023