Hello：
There is a obvious difference between the actual evaluation results of the algorithm and those described in the paper, the cause of the problem has not been found yet, please give me some suggestions.
1.Evaluation results for the model provided in github of SamsungLabs/fcaf3d：(12 epoches)
1.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>>] 5050/5050, 15.9 task/s, elapsed: 318s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.0496 | 0.7612 | 0.0094 | 0.5883 |
| table | 0.1570 | 0.8514 | 0.1222 | 0.7909 |
| sofa | 0.2261 | 0.8469 | 0.1440 | 0.7735 |
| chair | 0.1985 | 0.7662 | 0.1671 | 0.7272 |
| toilet | 0.3586 | 0.9379 | 0.2778 | 0.8552 |
| desk | 0.0970 | 0.7641 | 0.0665 | 0.6982 |
| dresser | 0.2063 | 0.9128 | 0.1956 | 0.8670 |
| night_stand | 0.4174 | 0.9608 | 0.4127 | 0.9451 |
| bookshelf | 0.1585 | 0.8865 | 0.1489 | 0.8511 |
| bathtub | 0.3658 | 0.9592 | 0.3561 | 0.8571 |
+-------------+---------+---------+---------+---------+
| Overall | 0.2235 | 0.8647 | 0.1900 | 0.7954 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.049614179879426956, 'table_AP_0.25': 0.15698249638080597, 'sofa_AP_0.25': 0.2261233627796173, 'chair_AP_0.25': 0.19846875965595245, 'toilet_AP_0.25': 0.35864582657814026, 'desk_AP_0.25': 0.09701932221651077, 'dresser_AP_0.25': 0.20626720786094666, 'night_stand_AP_0.25': 0.41742968559265137, 'bookshelf_AP_0.25': 0.15852327644824982, 'bathtub_AP_0.25': 0.3657997250556946, 'mAP_0.25': 0.22348737716674805, 'bed_rec_0.25': 0.7611650485436893, 'table_rec_0.25': 0.8513628620102215, 'sofa_rec_0.25': 0.84688995215311, 'chair_rec_0.25': 0.7661741214057508, 'toilet_rec_0.25': 0.9379310344827586, 'desk_rec_0.25': 0.7640807651434643, 'dresser_rec_0.25': 0.9128440366972477, 'night_stand_rec_0.25': 0.9607843137254902, 'bookshelf_rec_0.25': 0.8865248226950354, 'bathtub_rec_0.25': 0.9591836734693877, 'mAR_0.25': 0.8646940630326154, 'bed_AP_0.50': 0.009411284700036049, 'table_AP_0.50': 0.12223674356937408, 'sofa_AP_0.50': 0.14403118193149567, 'chair_AP_0.50': 0.1670922487974167, 'toilet_AP_0.50': 0.27775609493255615, 'desk_AP_0.50': 0.06645272672176361, 'dresser_AP_0.50': 0.19564399123191833, 'night_stand_AP_0.50': 0.41268840432167053, 'bookshelf_AP_0.50': 0.14886672794818878, 'bathtub_AP_0.50': 0.35609957575798035, 'mAP_0.50': 0.19002790749073029, 'bed_rec_0.50': 0.5883495145631068, 'table_rec_0.50': 0.790885860306644, 'sofa_rec_0.50': 0.773524720893142, 'chair_rec_0.50': 0.7272364217252396, 'toilet_rec_0.50': 0.8551724137931035, 'desk_rec_0.50': 0.6981934112646121, 'dresser_rec_0.50': 0.8669724770642202, 'night_stand_rec_0.50': 0.9450980392156862, 'bookshelf_rec_0.50': 0.851063829787234, 'bathtub_rec_0.50': 0.8571428571428571, 'mAR_0.50': 0.7953639545755846}
1.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>>>>>>>>>>>>>>] 312/312, 10.4 task/s, elapsed: 30s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0077 | 0.2742 | 0.0052 | 0.2419 |
| bed | 0.0003 | 0.0370 | 0.0000 | 0.0247 |
| chair | 0.0193 | 0.2661 | 0.0138 | 0.2354 |
| sofa | 0.0015 | 0.0825 | 0.0005 | 0.0825 |
| table | 0.0016 | 0.1429 | 0.0011 | 0.1229 |
| door | 0.0104 | 0.2441 | 0.0079 | 0.2313 |
| window | 0.0028 | 0.1773 | 0.0021 | 0.1631 |
| bookshelf | 0.0081 | 0.1558 | 0.0042 | 0.1429 |
| picture | 0.0028 | 0.2658 | 0.0025 | 0.2568 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0012 | 0.0866 | 0.0010 | 0.0709 |
| curtain | 0.0000 | 0.0448 | 0.0000 | 0.0448 |
| refrigerator | 0.0000 | 0.0351 | 0.0000 | 0.0000 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| sink | 0.0000 | 0.0102 | 0.0000 | 0.0000 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0048 | 0.2113 | 0.0039 | 0.1811 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0034 | 0.1130 | 0.0024 | 0.0999 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.007672073319554329, 'bed_AP_0.25': 0.0003270927118137479, 'chair_AP_0.25': 0.019300922751426697, 'sofa_AP_0.25': 0.0015425414312630892, 'table_AP_0.25': 0.0016294954111799598, 'door_AP_0.25': 0.010358083993196487, 'window_AP_0.25': 0.0027569225057959557, 'bookshelf_AP_0.25': 0.008131678216159344, 'picture_AP_0.25': 0.002778079127892852, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.0011750475969165564, 'curtain_AP_0.25': 2.2466720110969618e-05, 'refrigerator_AP_0.25': 1.3386352293309756e-05, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': 0.0, 'sink_AP_0.25': 1.440034111510613e-06, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.00475729675963521, 'mAP_0.25': 0.003359251655638218, 'cabinet_rec_0.25': 0.27419354838709675, 'bed_rec_0.25': 0.037037037037037035, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25': 0.08247422680412371, 'table_rec_0.25': 0.14285714285714285, 'door_rec_0.25': 0.24411134903640258, 'window_rec_0.25': 0.1773049645390071, 'bookshelf_rec_0.25': 0.15584415584415584, 'picture_rec_0.25': 0.26576576576576577, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.08661417322834646, 'curtain_rec_0.25': 0.04477611940298507, 'refrigerator_rec_0.25': 0.03508771929824561, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.0, 'sink_rec_0.25': 0.01020408163265306, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.21132075471698114, 'mAR_0.25': 0.11298182832749847, 'cabinet_AP_0.50': 0.005249418318271637, 'bed_AP_0.50': 7.024568276392529e-06, 'chair_AP_0.50': 0.013795703649520874, 'sofa_AP_0.50': 0.0005232820985838771, 'table_AP_0.50': 0.001064108102582395, 'door_AP_0.50': 0.007914243265986443, 'window_AP_0.50': 0.0021169877145439386, 'bookshelf_AP_0.50': 0.0042017120867967606, 'picture_AP_0.50': 0.0025250331964343786, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.0010463166981935501, 'curtain_AP_0.50': 2.2466720110969618e-05, 'refrigerator_AP_0.50': 0.0, 'showercurtrain_AP_0.50': 0.0, 'toilet_AP_0.50': 0.0, 'sink_AP_0.50': 0.0, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.003941362723708153, 'mAP_0.50': 0.0023559811525046825, 'cabinet_rec_0.
50': 0.24193548387096775, 'bed_rec_0.50': 0.024691358024691357, 'chair_rec_0.50': 0.23538011695906433, 'sofa_rec_0.50': 0.08247422680412371,
'table_rec_0.50': 0.12285714285714286, 'door_rec_0.50': 0.23126338329764454, 'window_rec_0.50': 0.16312056737588654, 'bookshelf_rec_0.50':
0.14285714285714285, 'picture_rec_0.50': 0.25675675675675674, 'counter_rec_0.50': 0.0, 'desk_rec_0.50': 0.07086614173228346, 'curtain_rec_0.
50': 0.04477611940298507, 'refrigerator_rec_0.50': 0.0, 'showercurtrain_rec_0.50': 0.0, 'toilet_rec_0.50': 0.0, 'sink_rec_0.50': 0.0, 'batht
ub_rec_0.50': 0.0, 'garbagebin_rec_0.50': 0.1811320754716981, 'mAR_0.50': 0.0998950286339104}
2.Evaluation results for the model trained by myself：(12 epoches)
2.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>] 5050/5050, 0.3 task/s, elapsed: 16300s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.5560 | 0.9903 | 0.4873 | 0.8738 |
| table | 0.3110 | 0.9817 | 0.2184 | 0.7785 |
| sofa | 0.4583 | 0.9841 | 0.4016 | 0.8740 |
| chair | 0.5615 | 0.9633 | 0.4967 | 0.8643 |
| toilet | 0.6838 | 0.9931 | 0.6078 | 0.8966 |
| desk | 0.1938 | 0.9644 | 0.0907 | 0.6791 |
| dresser | 0.2366 | 0.9541 | 0.1876 | 0.7752 |
| night_stand | 0.4379 | 0.9765 | 0.3945 | 0.8824 |
| bookshelf | 0.2321 | 0.9184 | 0.0987 | 0.5426 |
| bathtub | 0.3912 | 0.9796 | 0.3296 | 0.8367 |
+-------------+---------+---------+---------+---------+
| Overall | 0.4062 | 0.9705 | 0.3313 | 0.8003 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.5559849143028259, 'table_AP_0.25': 0.3109968602657318, 'sofa_AP_0.25': 0.4582793712615967, 'chair_AP_0.25': 0.5614634156227112, 'toilet_AP_0.25': 0.6837515830993652, 'desk_AP_0.25': 0.1938285529613495, 'dresser_AP_0.25': 0.23655451834201813, 'night_stand_AP_0.25': 0.43787485361099243, 'bookshelf_AP_0.25': 0.2321251481771469, 'bathtub_AP_0.25': 0.3912363648414612, 'mAP_0.25': 0.4062095582485199, 'bed_rec_0.25': 0.9902912621359223, 'table_rec_0.25': 0.981686541737649, 'sofa_rec_0.25': 0.9840510366826156, 'chair_rec_0.25': 0.963258785942492, 'toilet_rec_0.25': 0.993103448275862, 'desk_rec_0.25': 0.9643995749202976, 'dresser_rec_0.25': 0.9541284403669725, 'night_stand_rec_0.25': 0.9764705882352941, 'bookshelf_rec_0.25': 0.9184397163120568, 'bathtub_rec_0.25': 0.9795918367346939, 'mAR_0.25': 0.9705421231343856, 'bed_AP_0.50': 0.4872896671295166, 'table_AP_0.50': 0.21839171648025513, 'sofa_AP_0.50': 0.40156134963035583, 'chair_AP_0.50': 0.4966593384742737, 'toilet_AP_0.50': 0.607752799987793, 'desk_AP_0.50': 0.09067006409168243, 'dresser_AP_0.50': 0.18762624263763428, 'night_stand_AP_0.50': 0.39452967047691345, 'bookshelf_AP_0.50': 0.09874459356069565, 'bathtub_AP_0.50': 0.32958272099494934, 'mAP_0.50': 0.33128082752227783, 'bed_rec_0.50': 0.8737864077669902, 'table_rec_0.50': 0.778534923339012, 'sofa_rec_0.50': 0.8740031897926634, 'chair_rec_0.50': 0.8643170926517572, 'toilet_rec_0.50': 0.896551724137931, 'desk_rec_0.50': 0.6790648246546227, 'dresser_rec_0.50': 0.7752293577981652, 'night_stand_rec_0.50': 0.8823529411764706, 'bookshelf_rec_0.50': 0.5425531914893617, 'bathtub_rec_0.50': 0.8367346938775511, 'mAR_0.50': 0.8003128346684527}
2.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tun$
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>> ] 116/312, 0.2 task/s, elapsed[ ] 117/312, 0.2 t$
sk/s, elapsed: 547s,[ ] 118/312, 0.2 task/s, elapsed: 547s, ETA: 89[>>>>>>>>>>>>> ] 240/312, [> ] 241/312, 0.2 task/s, elapsed: 1262$
[> ] 242/312, 0.2 task/s, elapsed: 1262s[> ] 243/312, 0.2 task/s, elap[>>>>>>>>>>>>>>>>>>] 312/312, 0.2 task/s, elapsed: 1531s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0089 | 0.2500 | 0.0071 | 0.2419 |
| bed | 0.0022 | 0.0494 | 0.0005 | 0.0123 |
| chair | 0.0195 | 0.2661 | 0.0131 | 0.2383 |
| sofa | 0.0009 | 0.0825 | 0.0000 | 0.0309 |
| table | 0.0030 | 0.1400 | 0.0021 | 0.1257 |
| door | 0.0072 | 0.2698 | 0.0066 | 0.2548 |
| window | 0.0030 | 0.1844 | 0.0020 | 0.1525 |
| bookshelf | 0.0041 | 0.1429 | 0.0031 | 0.1429 |
| picture | 0.0049 | 0.2523 | 0.0045 | 0.2342 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0041 | 0.0709 | 0.0041 | 0.0551 |
| curtain | 0.0001 | 0.0597 | 0.0001 | 0.0597 |
| refrigerator | 0.0001 | 0.0877 | 0.0001 | 0.0877 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0005 | 0.0345 | 0.0001 | 0.0345 |
| sink | 0.0000 | 0.0408 | 0.0000 | 0.0408 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0076 | 0.2038 | 0.0063 | 0.1887 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0037 | 0.1186 | 0.0028 | 0.1056 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.0088879419490695, 'bed_AP_0.25': 0.002196365036070347, 'chair_AP_0.25': 0.019513951614499092, 'sofa_AP_0.25': 0.00092$
0919025614858, 'table_AP_0.25': 0.0029890856239944696, 'door_AP_0.25': 0.007219596300274134, 'window_AP_0.25': 0.002968191634863615, 'books$
elf_AP_0.25': 0.0041231452487409115, 'picture_AP_0.25': 0.004887890536338091, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.004088526125997305,
'curtain_AP_0.25': 0.0001449216069886461, 'refrigerator_AP_0.25': 0.00010905037925112993, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': $
.0005089639453217387, 'sink_AP_0.25': 3.358132016728632e-05, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.007584888022392988, 'mAP_0.25'$
0.003676677355542779, 'cabinet_rec_0.25': 0.25, 'bed_rec_0.25': 0.04938271604938271, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25$
: 0.08247422680412371, 'table_rec_0.25': 0.14, 'door_rec_0.25': 0.2698072805139186, 'window_rec_0.25': 0.18439716312056736, 'bookshelf_rec_0
.25': 0.14285714285714285, 'picture_rec_0.25': 0.25225225225225223, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.07086614173228346, 'curtain_
rec_0.25': 0.05970149253731343, 'refrigerator_rec_0.25': 0.08771929824561403, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.034482758
620689655, 'sink_rec_0.25': 0.04081632653061224, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.2037735849056604, 'mAR_0.25': 0.118589569
75081056, 'cabinet_AP_0.50': 0.007079910486936569, 'bed_AP_0.50': 0.0005367686389945447, 'chair_AP_0.50': 0.013134335167706013, 'sofa_AP_0.5
0': 4.525698022916913e-05, 'table_AP_0.50': 0.0021134275011718273, 'door_AP_0.50': 0.00658177025616169, 'window_AP_0.50': 0.0019997931085526
943, 'bookshelf_AP_0.50': 0.0031221157405525446, 'picture_AP_0.50': 0.004526467993855476, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.00407800
916582346, 'curtain_AP_0.50': 0.0001449216069886461, 'refrigerator_AP_0.50': 0.0001018800976453349, 'showercurtrain_AP_0.50': 0.0, 'toilet_A
P_0.50': 5.409148434409872e-05, 'sink_AP_0.50': 3.358132016728632e-05, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.006303954869508743, '
mAP_0.50': 0.0027697933837771416, 'cabinet_rec_0.50': 0.24193548387096775, 'bed_rec_0.50': 0.012345679012345678, 'chair_rec_0.50': 0.2383040
9356725146, 'sofa_rec_0.50': 0.030927835051546393, 'table_rec_0.50': 0.12571428571428572, 'door_rec_0.50': 0.25481798715203424, 'window_rec_
0.50': 0.1524822695035461, 'bookshelf_rec_0.50': 0.14285714285714285, 'picture_rec_0.50': 0.23423423423423423, 'counter_rec_0.50': 0.0, 'des
k_rec_0.50': 0.05511811023622047, 'curtain_rec_0.50': 0.05970149253731343, 'refrigerator_rec_0.50': 0.08771929824561403, 'showercurtrain_rec
1 Jan 14, 2022
712 Dec 19, 2022
41 Nov 17, 2022
29 Sep 20, 2022
47 Oct 04, 2022
67 Dec 04, 2022
1k Dec 28, 2022
107 Nov 02, 2022
456 Jan 08, 2023
37 Dec 29, 2022
4 May 27, 2022
79 Dec 02, 2022
6 Jun 15, 2021
24 May 21, 2022
434 Jan 04, 2023
722 Dec 28, 2022
253 Dec 21, 2022
115 Nov 21, 2022
938 Dec 26, 2022
3 Dec 17, 2022