When I run the command of "bash render_batch.sh debug", it gives an error as following

OpenGL.raw.EGL._errors.EGLError: EGLError( err = EGL_NOT_INITIALIZED, baseOperation = eglInitialize, cArguments = ( <OpenGL._opaque.EGLDisplay_pointer object at 0x7f7b3d0ee2c0>, <importlib._bootstrap.LP_c_int object at 0x7f7b3d0ee440>, <importlib._bootstrap.LP_c_int object at 0x7f7b3d106bc0>, ), result = 0 )

How can I fix this?

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='drive.google.com', port=443): Max retries exceeded with url: /uc?id=1tCU5MM1LhRgGou5OpmpjBQbSrYIUoYab (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7fbc4ad7deb0>: Failed to establish a new connection: [Errno 110] Connection timed out'))

After installing all packages, I got the results successfully for PIFu and PaMIR. I faced the runtime error when trying to get the ICON demo result. Could you guide what setting was wrong?

$ python infer.py -cfg ../configs/icon-filter.yaml -gpu 0 -in_dir ../examples -out_dir ../results

Traceback (most recent call last):
  File "infer.py", line 304, in <module>
    verts_pr, faces_pr, _ = model.test_single(in_tensor)
  File "./ICON/apps/ICON.py", line 738, in test_single
    sdf = self.reconEngine(opt=self.cfg,
  File "./.virtualenvs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "../lib/common/seg3d_lossless.py", line 148, in forward
    return self._forward_faster(**kwargs)
  File "../lib/common/seg3d_lossless.py", line 170, in _forward_faster
    occupancys = self.batch_eval(coords, **kwargs)
  File "../lib/common/seg3d_lossless.py", line 139, in batch_eval
    occupancys = self.query_func(**kwargs, points=coords2D)
  File "../lib/common/train_util.py", line 338, in query_func
    preds = netG.query(features=features,
  File "../lib/net/HGPIFuNet.py", line 285, in query
    smpl_sdf, smpl_norm, smpl_cmap, smpl_ind = cal_sdf_batch(
  File "../lib/dataset/mesh_util.py", line 231, in cal_sdf_batch
    residues, normals, pts_cmap, pts_ind = func(
  File "./.virtualenvs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./.virtualenvs/icon/lib/python3.8/site-packages/bvh_distance_queries/mesh_distance.py", line 79, in forward
    output = self.search_tree(triangles, points)
  File "./.virtualenvs/icon/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "./.virtualenvs/icon/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "./.virtualenvs/icon/lib/python3.8/site-packages/bvh_distance_queries/bvh_search_tree.py", line 109, in forward
    output = BVHFunction.apply(
  File "./.virtualenvs/icon/lib/python3.8/site-packages/bvh_distance_queries/bvh_search_tree.py", line 42, in forward
    outputs = bvh_distance_queries_cuda.distance_queries(
RuntimeError: after reduction step 1: cudaErrorInvalidDevice: invalid device ordinal

Hi, I found the program was running so slowly when I ran bash render_batch.sh debug all, I figured out it stopped at hits = mesh.ray.intersects_any (origins + delta * normals, vectors), and the number of rays is the millions, is that the reason why it was too slow?

Hi, first thanks for your code In colab version, when running the cell that starts with

run the test on examples

the execution is very fast, no errors, but do nothing. Next cell show errors: FileNotFoundError: [Errno 2] No such file or directory: '/content/ICON/results/icon-filter/vid/22097467bffc92d4a5c4246f7d4edb75_display.mp4'

many thanks

Getting this error when installing locally on my workstation via colab bash script.

.../ICON/pytorch3d/pytorch3d/_C.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNSt15__exception_ptr13exception_ptr10_M_releaseEv

This after installing pytorch3d locally as recommended. Conda has too many conflicts and never resolves.

Installing torch through pip works (1.8.2+cu111) up until the next steps of infer.py because bvh_distance_queries only supports cuda 11.0. This would most likely require compiling against 11.0, but it will probably lead to more errors as I don't know what this repository's dependencies require as far as torch goes.

I followed the instruction to process the THuman2.0 dataset and train the ICON. After 10 epochs, the result on tensorboard seems not good especially the reconstruction of the side face. I can't find where the problem is.

The change of loss during training is

Hi, I reproduced ICON and trained with THuman2.0.

As a result of the my training, only this model produces a lot of noise and isolated mesh.

Would you give me an advice?

I used [-1, 1] SMPL vertices, query points, and 15.0 SDF clipping.

I believe that ICON has great potential and generalization performance.

But I don't know what is the problem.

The results below are about unseen data, and I used pre-trained GCMR just like PaMIR for SMPL prediction.

Thank you.

Many thanks to the author for his work. I found a question in the process of reading papers and code. The paper introduces in the Refining SMPL section that the results of SMPL modeling can be iteratively optimized during the inference process. The loss function includes two parts, the L1 difference between the unclothed normal map and the normal map of the model prediction results, and The L1 difference between the mask of the smpl normal map and the mask of the original image, but you did not implement the corresponding implementation in the code. What is the reason for this? Is the existing code implementation more efficient than the original implementation?

            # silhouette loss
            smpl_arr = torch.cat([T_mask_F, T_mask_B], dim=-1)[0]       # smpl mask maps
            gt_arr = torch.cat(                                         # clothed normal maps
                [in_tensor['normal_F'][0], in_tensor['normal_B'][0]],
                dim=2).permute(1, 2, 0)
            gt_arr = ((gt_arr + 1.0) * 0.5).to(device)
            bg_color = torch.Tensor([0.5, 0.5,
                                     0.5]).unsqueeze(0).unsqueeze(0).to(device)
            gt_arr = ((gt_arr - bg_color).sum(dim=-1) != 0.0).float()
            diff_S = torch.abs(smpl_arr - gt_arr)
            losses['silhouette']['value'] = diff_S.mean()

I would like to know some details about training. Is the Ground-Truth SMPL or the predicted SMPL used in training ICON? Also, what about normal images? According to my understanding of the paper and practice, ICON should train the normal network first and then train the implicit reconstruction network. When I reproduce ICON, I don't know whether to choose the Ground-Truth or the predicted data for SMPL model and normal images, respectively.

Hi, thank you so much for the wonderful work and corresponding codes. I am facing the following issue: https://github.com/YuliangXiu/ICON/blob/0045bd10f076bf367d25b7dac41d0d5887b8694f/lib/bvh-distance-queries/bvh_distance_queries/bvh_search_tree.py#L27

Is there any .py file called bvh_distance_queries_cuda ? Please let me know a possible solution. Thank you for your effort and help :) :) :)

After training of the implicit MLP, I got quite wired results. The reconstructed meshes are poor. The evaluation results shows that NC is very low, but chamfer and p2s are very high. Do you know where the problem is? I would appreciate it a lot if you could give me some suggestions!

Hi, I'm confused about local_affine_model in the cloth refinement. I find that you use the LocalAffine() converting the verts of the refinemesh to the affined verts. The step seems to train a Affinemodel. I know that we need get the loss between P_normal from the mesh and normal of the image to optimizer the verts, but I don't understand why we need introduct the Affinemodel. The procedure of localAffine is meaning getting the Affine Matrix under the Camera coordinate system? I confused about these.

Unfortunately, I can't manage to install ICON locally. The Colab Notebook and the model on Huggingface also seem to be broken. Would it be possible to update the documentation / environment or share a Docker file?

Hi, recently I've been trying to use depth map as prior information, but how can I generate the point cloud at camera coordinate space or model space from the given depth map(generated by render_single.py) ? For example, how can i generate point cloud in camera/view space by depth_F/xxx.png below, i.e. how can i get the real depth value in camera space. I tried to see the coordinate transformation in the code, it seems that I can get perspective matrix and model_view matrix in lib/render/camer.py , which may transform the model from local space into clip space. But I am still confused about how to convert the depth map to another space.

It would be very appricated if anyone could give me some advice!

ubuntu 22.04 + NVIDIA 2080Ti.

I encountered this error when execute this script "bash scripts/render_batch.sh debug all".

thuman2 START----------
Debug renderer
Rendering thuman2 0001
/home/hcp/anaconda3/envs/metaverse/lib/python3.8/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.16.5 and <1.23.0 is required for this version of SciPy (detected version 1.23.3
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"

**scripts/render_single.sh: line 33: 137550 Killed                  python $PYTHON_SCRIPT -s $SUBJECT -o $SAVE_DIR -r $NUM_VIEWS -w $SIZE**
thuman2 END----------

https://github.com/YuliangXiu/ICON/blob/ece5a09aa2d56aec28017430e65a0352622a0f30/lib/dataset/mesh_util.py#L283

` print(triangles.device) # cuda:1

print(points.device) # cuda:1

residues, pts_ind, _ = point_to_mesh_distance(points, triangles)

print(triangles.device) # cuda:1

print(pts_ind.device) # cuda:0

print(residues.device) # cuda:0`

command: python -m apps.infer -cfg ./configs/icon-filter.yaml -gpu 1 -in_dir {*} -out_dir {*} 'CUDA_VISIBLE_DEVICES=1' doesn't work either.