TransMVSNet: Global Context-aware Multi-view Stereo Network with Transformers.

Overview
Comments
  • abs_depth_error

    abs_depth_error

    I find ABS_DEPTH_ERROR is close to 6 or even 7 during training, is this normal? Here are the training results for Epoch 5. Is it because of the slow convergence?

    avg_test_scalars: {'loss': 4.360309665948113, 'depth_loss': 6.535046514014081, 'entropy_loss': 4.360309665948113, 'abs_depth_error': 6.899323051878795, 'thres2mm_error': 0.16829867261163733, 'thres4mm_error': 0.10954744909229193, 'thres8mm_error': 0.07844322964626443, 'thres14mm_error': 0.06323695212957076, 'thres20mm_error': 0.055751020700780536, 'thres2mm_abserror': 0.597563438798779, 'thres4mm_abserror': 2.7356186663791666, 'thres8mm_abserror': 5.608324628466483, 'thres14mm_abserror': 10.510002394554125, 'thres20mm_abserror': 16.67409769420184, 'thres>20mm_abserror': 78.15814284054947}

    opened by zhang-snowy 7
  • About the fusion setting in DTU

    About the fusion setting in DTU

    Thank you for your great contribution. The script use the gipuma as the fusion method with num_consistent=5prob_threshold=0.05disp_threshold=0.25. However, it produces point cloud results with only 1/2 points compared with the point cloud results you provide in DTU, leading to a much poorer result in DTU. Is there any setting wrong in the script? Or because it does not use the dynamic fusion method described in the paper. Could you provide the dynamic fusion process in DTU?

    opened by DIVE128 5
  • Testing on TnT advanced dataset

    Testing on TnT advanced dataset

    Hi, thank you for sharing this great work!

    I'm try to test transmvsnet on tnt advanced dataset, but meet some problem. My test environment is ubuntu16.04 with cuda11.3 and pytorch 1.10.

    The first thing is that there is no cams_1 folder under tnt dataset, is it a revised version of original cams folder or you just changed the folder name?

    I just changed the folder name, then run scripts/test_tnt.sh, but I find the speed is rather slow, about 10 seconds on 1080ti for a image (1056 x 1920), is it normal?

    Finally I get the fused point cloud, but the cloud is meaningless, I checked the depth map and confidence map, all of the data are very strange, apperantly not right.

    Can you help me with these problems?

    opened by CanCanZeng 4
  • Some implement details about the paper

    Some implement details about the paper

    Firstly thanks for your paper and I'm looking forward to your open-sourced code.

    And I have some questions about your paper: (Hopefully you can reply, thanks in advance!) (1) In section 4.2, "The model is trained with Adam for 10 epochs with an initial learning rate of 0.001, which decays by a factor of 0.5 respectively after 6, 8, and 12 epochs." I'm confused about the epochs. And I also noticed that this training strategy is different from CasMVSNet. Did you try the training strategy in CasMVSNet? What's the difference? (2) In Table4(b), focal loss(what is the value of \gamma?) suppresses CE loss by 0.06. However, In Table4(e) and Table 6, we infer that the best model use CE loss(FL with \gamma=0). My question is: did you keep Focal loss \gamma unchanged in the Ablation study in Table4? If not, how \gamma changes? Could you elaborate?

    Really appreciate it!

    opened by JeffWang987 4
  • source code

    source code

    Hi, @Lxiangyue Thank you for the nice paper.

    It's been over a month since authors announced that the code will be available. May I know when the code will be released? (or whether it will not be released)

    opened by Ys-Jung77 3
  • Testing on my own dataset

    Testing on my own dataset

    Hi thanks for your interesting work. I tested your code on one of the DTU dataset (Moda). as you can see from the following image, the results are quite well. image

    but I got a very bad result, when i tried to tested on one of my dataset (see the following pic) using your pretrained model (model_dtu). Now, my question is that do you thing that the object is too complicated and different compared to DTU dataset and it is all we can get from the pretrain model without retraining it? is it possible to improve by changing the input parameters? In general, would you please share your opinion about this result? image

    opened by AliKaramiFBK 1
  • generate dense 3D point cloud

    generate dense 3D point cloud

    thanks for your greate work I just tried to do a test on DTU testing dataset I got the depth map for each view but I got a bit confised on how to generate 3D point cloud using your code would you please let me know Best

    opened by AliKaramiFBK 1
  • GPU memory consumption

    GPU memory consumption

    Hi! Thanks for your excellent work! When I tested on the DTU dataset with pretrained model, the gpu memory consumption is 4439MB, but the paper gives 3778MB.

    I do not know where the problem is.

    opened by JianfeiJ 0
  • Using my own data

    Using my own data

    If I have the intrinsic matrics and extrinsic matrics of cameras, which means I don't need to run SFM in COLMAP, how should I struct my data to train the model?

    opened by PaperDollssss 2
  • TnT dataset results

    TnT dataset results

    Thanks for the great job. I follow the instruction and upload the reconstruction result of tnt but find the F-score=60.29, and I find the point cloud sizes are a larger than the upload ones. Whether the reconstructed point cloud use the param settting of test_tnt.sh or it should be tuned manually? :smile:

    opened by CC9310 1
  • TankAndTemple Test

    TankAndTemple Test

    Hi, 我测试了TAT数据集中的Family,使用的是默认脚本test_tnt.sh,采用normal融合,最近仅得到13MB点云文件。经检查发现生成的mask文件夹中的_geo.png都是大部分区域黑色图片,从而最后得到的 final.png的大部分区域都是无效的。geometric consistency阈值分别是默认的0.01和1。不知道您这边是否有一样的问题?

    opened by lt-xiang 13
  • Why is there a big gap between the reproducing results and the paper results?

    Why is there a big gap between the reproducing results and the paper results?

    I have tried the pre-trained model you offered on DTU dataset. But the results I got are mean_acc=0.299, mean_comp=0.385, overall=0.342, and the results you presented in the paper are mean_acc=0.321, mean_comp=0.289, overall=0.305.

    I do not know where the problem is.

    opened by cainsmile 14
Releases(T&T_ply)
Owner
旷视研究院 3D 组
旷视科技(Face++)研究院 3D 组(原 SLAM 组)
旷视研究院 3D 组
A Python Package for Portfolio Optimization using the Critical Line Algorithm

PyCLA A Python Package for Portfolio Optimization using the Critical Line Algorithm Getting started To use PyCLA, clone the repo and install the requi

19 Oct 11, 2022
Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image classification, in Pytorch

Transformer in Transformer Implementation of Transformer in Transformer, pixel level attention paired with patch level attention for image c

Phil Wang 272 Dec 23, 2022
Code and models for "Rethinking Deep Image Prior for Denoising" (ICCV 2021)

DIP-denosing This is a code repo for Rethinking Deep Image Prior for Denoising (ICCV 2021). Addressing the relationship between Deep image prior and e

Computer Vision Lab. @ GIST 36 Dec 29, 2022
GAN-based Matrix Factorization for Recommender Systems

GAN-based Matrix Factorization for Recommender Systems This repository contains the datasets' splits, the source code of the experiments and their res

Ervin Dervishaj 9 Nov 06, 2022
[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 04, 2023
PyTorch and Tensorflow functional model definitions

functional-zoo Model definitions and pretrained weights for PyTorch and Tensorflow PyTorch, unlike lua torch, has autograd in it's core, so using modu

Sergey Zagoruyko 590 Dec 22, 2022
Bringing sanity to world of messed-up data

Sanitize sanitize is a Python module for making sure various things (e.g. HTML) are safe to use. It was originally written by Mark Pilgrim and is dist

Alireza Savand 63 Oct 26, 2021
Based on Stockfish neural network(similar to LcZero)

MarcoEngine Marco Engine - interesnaya neyronnaya shakhmatnaya set', kotoraya ispol'zuyet metod samoobucheniya(dostizheniye khoroshoy igy putem proboy

Marcus Kemaul 4 Mar 12, 2022
RNN Predict Street Commercial Vitality

RNN-for-Predicting-Street-Vitality Code and dataset for Predicting the Vitality of Stores along the Street based on Business Type Sequence via Recurre

Zidong LIU 1 Dec 15, 2021
In this project we predict the forest cover type using the cartographic variables in the training/test datasets.

Kaggle Competition: Forest Cover Type Prediction In this project we predict the forest cover type (the predominant kind of tree cover) using the carto

Marianne Joy Leano 1 Mar 15, 2022
ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

ONNX-GLPDepth - Python scripts for performing monocular depth estimation using the GLPDepth model in ONNX

Ibai Gorordo 18 Nov 06, 2022
Code for: https://berkeleyautomation.github.io/bags/

DeformableRavens Code for the paper Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks. Here is the

Daniel Seita 121 Dec 30, 2022
Conceptual 12M is a dataset containing (image-URL, caption) pairs collected for vision-and-language pre-training.

Conceptual 12M We introduce the Conceptual 12M (CC12M), a dataset with ~12 million image-text pairs meant to be used for vision-and-language pre-train

Google Research Datasets 226 Dec 07, 2022
Code for AutoNL on ImageNet (CVPR2020)

Neural Architecture Search for Lightweight Non-Local Networks This repository contains the code for CVPR 2020 paper Neural Architecture Search for Lig

Yingwei Li 104 Aug 31, 2022
This is the official PyTorch implementation of our paper: "Artistic Style Transfer with Internal-external Learning and Contrastive Learning".

Artistic Style Transfer with Internal-external Learning and Contrastive Learning This is the official PyTorch implementation of our paper: "Artistic S

51 Dec 20, 2022
👨‍💻 run nanosaur in simulation with Gazebo/Ingnition

🦕 👨‍💻 nanosaur_gazebo nanosaur The smallest NVIDIA Jetson dinosaur robot, open-source, fully 3D printable, based on ROS2 & Isaac ROS. Designed & ma

nanosaur 9 Jul 19, 2022
Deep learning with dynamic computation graphs in TensorFlow

TensorFlow Fold TensorFlow Fold is a library for creating TensorFlow models that consume structured data, where the structure of the computation graph

1.8k Dec 28, 2022
Dilated RNNs in pytorch

PyTorch Dilated Recurrent Neural Networks PyTorch implementation of Dilated Recurrent Neural Networks (DilatedRNN). Getting Started Installation: $ pi

Zalando Research 200 Nov 17, 2022
[CVPR'21] FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space

FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space by Quande Liu, Cheng Chen, Ji

Quande Liu 178 Jan 06, 2023