An executor that wraps 3D mesh models and encodes 3D content documents to d-dimension vector.

Related tags

Image Processingexecutor-3d-encoder
Overview

3D Mesh Encoder

An Executor that receives Documents containing point sets data in its blob attribute, with shape (N, 3) and encodes it to embeddings of shape (D,). Now, the following pretrained models are ready to be used to create embeddings:

  • PointConv-Shapenet-d512: A PointConv model resulted in 512 dimension of embeddings, which is finetuned based on ShapeNet dataset.
  • PointConv-Shapenet-d1024: A PointConv model resulted in 1024 dimension of embeddings, which is finetuned based on ShapeNet dataset.

Usage

via Docker image (recommended)

from jina import Flow

f = Flow().add(uses='jinahub+docker://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

via source code

from jina import Flow

f = Flow().add(uses='jinahub://3DMeshEncoder', \
               uses_with={'pretrained_model': 'PointConv-Shapenet-d512'})

This Executor offers a GPU tag to speed up encoding. For more information on how to run the executor on GPU, check out the documentation.

How to finetune pretrained-model?

install finetuner

$ pip install finetuner

prepare dataset

TBD...

finetuning model with labeled dataset

$ python finetune.py --help

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/train.bin \
    --eval_dataset /path/to/eval.bin \
    --batch_size 128 \
    --epochs 50

finetuning model with unlabeled dataset

$ python finetune.py --model_name pointconv \
    --train_dataset /path/to/unlabeled_data.bin \
    --interactive

References

  • PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
  • PointConv: Deep Convolutional Networks on 3D Point Clouds
Comments
  • which version should the `finetuner` module be?

    which version should the `finetuner` module be?

    I was preparing the environment of this repo. However, I found it confused that the python import code in finetune.py gives error message:

    from finetuner.tuner.callback import BestModelCheckpoint
from finetuner.tuner.pytorch.losses import TripletLoss
from finetuner.tuner.pytorch.miner import TripletEasyHardMiner

    It seems that there's no finetuner.tuner in finetuner module but finetuner.finetuner. Besides, callback/pytorch is not in finetuner.finetuner now.

    Now my finetuner module installed by running pip install -U finetuner is 0.5.0. Is there any solution? Maybe you can freeze the version in requirements.txt.

    opened by elenacliu 4
  • fix: fix previous structure

    fix: fix previous structure

    I got pretrain and finetune process mixed up, and previously I pretrained the models using finetune model structure, which might lead to worse results. The fix code is in process and may be finished in a few days.

    opened by elenacliu 2
  • Add dataset, preprocess and pointnet2

    Add dataset, preprocess and pointnet2

    Here is the modification of the structure. There are still some bugs in my pointnet2 codes. I will solve the possible problems after you comment.

    @numb3r3 @jemmyshin

    opened by elenacliu 1
  • In `farthest_point_sample` function, the first centroid sampled is not random.

    In `farthest_point_sample` function, the first centroid sampled is not random.

    I'm reading your codes.

    https://github.com/jina-ai/executor-3d-encoder/blob/main/executor/models/modules.py#L75

    The randint line is commented, but in the original implementation and the algorithm itself, it should be random? Maybe we can provide a fixed seed if we want to get the same result?

    opened by elenacliu 1
  • Provide diffusion encoder

    Provide diffusion encoder

    https://github.com/nmwsharp/diffusion-net

    I'm unfamiliar with the jina ecosystem. So this is my first issue.

    Final goal is to have dalle generate 3d digital human meshes like https://github.com/jina-ai/dalle-flow

    opened by fire 2
Releases(v0.3.0)
Owner
Jina AI
A Neural Search Company. We provide the cloud-native neural search solution powered by state-of-the-art AI technology.
Jina AI
GitHub Repository
Water marker for images.

watermarker linux users: To fix this error,please add truetype font path File "watermark.py", line 58, in module font = ImageFont.truetype("Dro

13 Oct 27, 2022
Python scripts for semi-automated morphometric analysis of atolls from Landsat satellite Imagery.

AtollGeoMorph Python scripts for semi-automated morphometric analysis of atolls from Landsat satellite Imagery. The python scripts included allow user

1 Dec 16, 2022
Python avatar generator for absolute nerds

pagan Welcome to the Python Avatar Generator for Absolute Nerds. Current version: 0.4.3 View the change history here. Remember those good old days whe

David Bothe 280 Dec 16, 2022
Vignette is a Python library to create and manage thumbnails following the FreeDesktop standard.

Vignette Vignette is a Python library to create and manage thumbnails following the FreeDesktop standard. Thumbnails are stored in a shared directory

3 Feb 06, 2022
Polyfoto - Create image mosaics.

Polyfoto Create image mosaics. Showcase "Before and After Science" by Brian Eno "Scott 3" by Scott Walker Installation Clone this repository to your l

Shane Drabing 149 Dec 25, 2022
PSD (Photoshop, Krita, Gimp...) -> Godot.

limage v0.2.2 Features Getting Started Tags Settings Todo Customizer Changes Solutions WARNING: Requires Python to be installed PSD (Photoshop, Krita,

21 Nov 10, 2022
3D Reconstruction Software

Meshroom is a free, open-source 3D Reconstruction Software based on the AliceVision Photogrammetric Computer Vision framework. Learn more details abou

AliceVision 8.7k Jan 02, 2023
A script to generate a profile picture and a banner that show the same image on Discord.

Discord profile picture & banner generator A script to generate a profile picture and a banner that show the same image on Discord. Installation / Upd

Victor B. 9 Nov 27, 2022
Hide sensitive information in images

Data-Preserved Script allowing to blur the most sensitive information on images. Prerequisites Before you begin, ensure you have met the following req

2 Dec 01, 2021
The coolest python qrcode maker for small businesses.

QR.ify The coolest python qrcode maker for small businesses. Author Zach Yusuf Project description Python final project. Built to test python skills P

zachystuff 2 Jan 14, 2022
Nutrify - take a photo of food and learn about it

Nutrify - take a photo of food and learn about it Work in progress. To make this a thing, we're going to need lots of food images... Start uploading y

Daniel Bourke 93 Dec 30, 2022
MikuMikuRig是一款集生成控制器，自动导入动画，自动布料为一体的blender插件

Miku_Miku_Rig MikuMikuRig是一款集生成控制器，自动导入动画，自动布料为一体的blender插件。 MikumiKurig is a Blender plugin that can generates rig, automatically imports animations

小威廉伯爵 342 Dec 29, 2022
Docbarcodes extracts 1D and 2D barcodes from scanned PDF documents or images. It can be used to automate extraction and processing of all kind of documents.

Intro Barcodes are being used in many documents or forms to enable machine reading capabilities and reduce manual processing effort. Simple 1D barcode

Arlind Nocaj 3 Jun 18, 2022
Craft PNG files that appear completely different in Apple software

Ambiguous PNG Packer Craft PNG files that appear completely different in Apple software

David Buchanan 1k Dec 29, 2022
A little Python tool to convert a TrueType (ttf/otf) font into a PNG for use in demos.

font2png A little Python tool to convert a TrueType (ttf/otf) font into a PNG for use in demos. To use from command line it expects python3 to be at /

Rich Elmes 3 Dec 22, 2021
Fill holes in binary 2D & 3D images fast.

Fill holes in binary 2D & 3D images fast.

11 Dec 09, 2022
Python script to generate vector graphics of an oriented lattice unit cell

unitcell Python script to generate vector graphics of an oriented lattice unit cell Examples unitcell --type hexagonal --eulers 12 23 34 --axes --crys

Philip Eisenlohr 2 Dec 10, 2021
This script is for photographers to do timeslice with one click.

One Click TimeSlice Tool What is this for This is for photographers who want to create TimeSlice pictures without installing PS plugins. Before using

Xi Zhao 13 Sep 23, 2022
Convert a DOS Punk image to text

DOS Punk Text Inspired by MAX CAPACITY's DOS Punks & the amazing DOS Punk community. DOS Punk Text is a Python 3 script that renders a DOS Punk image

4 Jan 13, 2022
An executor that performs standard pre-processing and normalization on images.

An executor that performs standard pre-processing and normalization on images.

Jina AI 6 Jun 30, 2022