Hello：
There is a obvious difference between the actual evaluation results of the algorithm and those described in the paper, the cause of the problem has not been found yet, please give me some suggestions.
1.Evaluation results for the model provided in github of SamsungLabs/fcaf3d：(12 epoches)
1.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>>] 5050/5050, 15.9 task/s, elapsed: 318s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.0496 | 0.7612 | 0.0094 | 0.5883 |
| table | 0.1570 | 0.8514 | 0.1222 | 0.7909 |
| sofa | 0.2261 | 0.8469 | 0.1440 | 0.7735 |
| chair | 0.1985 | 0.7662 | 0.1671 | 0.7272 |
| toilet | 0.3586 | 0.9379 | 0.2778 | 0.8552 |
| desk | 0.0970 | 0.7641 | 0.0665 | 0.6982 |
| dresser | 0.2063 | 0.9128 | 0.1956 | 0.8670 |
| night_stand | 0.4174 | 0.9608 | 0.4127 | 0.9451 |
| bookshelf | 0.1585 | 0.8865 | 0.1489 | 0.8511 |
| bathtub | 0.3658 | 0.9592 | 0.3561 | 0.8571 |
+-------------+---------+---------+---------+---------+
| Overall | 0.2235 | 0.8647 | 0.1900 | 0.7954 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.049614179879426956, 'table_AP_0.25': 0.15698249638080597, 'sofa_AP_0.25': 0.2261233627796173, 'chair_AP_0.25': 0.19846875965595245, 'toilet_AP_0.25': 0.35864582657814026, 'desk_AP_0.25': 0.09701932221651077, 'dresser_AP_0.25': 0.20626720786094666, 'night_stand_AP_0.25': 0.41742968559265137, 'bookshelf_AP_0.25': 0.15852327644824982, 'bathtub_AP_0.25': 0.3657997250556946, 'mAP_0.25': 0.22348737716674805, 'bed_rec_0.25': 0.7611650485436893, 'table_rec_0.25': 0.8513628620102215, 'sofa_rec_0.25': 0.84688995215311, 'chair_rec_0.25': 0.7661741214057508, 'toilet_rec_0.25': 0.9379310344827586, 'desk_rec_0.25': 0.7640807651434643, 'dresser_rec_0.25': 0.9128440366972477, 'night_stand_rec_0.25': 0.9607843137254902, 'bookshelf_rec_0.25': 0.8865248226950354, 'bathtub_rec_0.25': 0.9591836734693877, 'mAR_0.25': 0.8646940630326154, 'bed_AP_0.50': 0.009411284700036049, 'table_AP_0.50': 0.12223674356937408, 'sofa_AP_0.50': 0.14403118193149567, 'chair_AP_0.50': 0.1670922487974167, 'toilet_AP_0.50': 0.27775609493255615, 'desk_AP_0.50': 0.06645272672176361, 'dresser_AP_0.50': 0.19564399123191833, 'night_stand_AP_0.50': 0.41268840432167053, 'bookshelf_AP_0.50': 0.14886672794818878, 'bathtub_AP_0.50': 0.35609957575798035, 'mAP_0.50': 0.19002790749073029, 'bed_rec_0.50': 0.5883495145631068, 'table_rec_0.50': 0.790885860306644, 'sofa_rec_0.50': 0.773524720893142, 'chair_rec_0.50': 0.7272364217252396, 'toilet_rec_0.50': 0.8551724137931035, 'desk_rec_0.50': 0.6981934112646121, 'dresser_rec_0.50': 0.8669724770642202, 'night_stand_rec_0.50': 0.9450980392156862, 'bookshelf_rec_0.50': 0.851063829787234, 'bathtub_rec_0.50': 0.8571428571428571, 'mAR_0.50': 0.7953639545755846}
1.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>>>>>>>>>>>>>>] 312/312, 10.4 task/s, elapsed: 30s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0077 | 0.2742 | 0.0052 | 0.2419 |
| bed | 0.0003 | 0.0370 | 0.0000 | 0.0247 |
| chair | 0.0193 | 0.2661 | 0.0138 | 0.2354 |
| sofa | 0.0015 | 0.0825 | 0.0005 | 0.0825 |
| table | 0.0016 | 0.1429 | 0.0011 | 0.1229 |
| door | 0.0104 | 0.2441 | 0.0079 | 0.2313 |
| window | 0.0028 | 0.1773 | 0.0021 | 0.1631 |
| bookshelf | 0.0081 | 0.1558 | 0.0042 | 0.1429 |
| picture | 0.0028 | 0.2658 | 0.0025 | 0.2568 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0012 | 0.0866 | 0.0010 | 0.0709 |
| curtain | 0.0000 | 0.0448 | 0.0000 | 0.0448 |
| refrigerator | 0.0000 | 0.0351 | 0.0000 | 0.0000 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| sink | 0.0000 | 0.0102 | 0.0000 | 0.0000 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0048 | 0.2113 | 0.0039 | 0.1811 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0034 | 0.1130 | 0.0024 | 0.0999 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.007672073319554329, 'bed_AP_0.25': 0.0003270927118137479, 'chair_AP_0.25': 0.019300922751426697, 'sofa_AP_0.25': 0.0015425414312630892, 'table_AP_0.25': 0.0016294954111799598, 'door_AP_0.25': 0.010358083993196487, 'window_AP_0.25': 0.0027569225057959557, 'bookshelf_AP_0.25': 0.008131678216159344, 'picture_AP_0.25': 0.002778079127892852, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.0011750475969165564, 'curtain_AP_0.25': 2.2466720110969618e-05, 'refrigerator_AP_0.25': 1.3386352293309756e-05, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': 0.0, 'sink_AP_0.25': 1.440034111510613e-06, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.00475729675963521, 'mAP_0.25': 0.003359251655638218, 'cabinet_rec_0.25': 0.27419354838709675, 'bed_rec_0.25': 0.037037037037037035, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25': 0.08247422680412371, 'table_rec_0.25': 0.14285714285714285, 'door_rec_0.25': 0.24411134903640258, 'window_rec_0.25': 0.1773049645390071, 'bookshelf_rec_0.25': 0.15584415584415584, 'picture_rec_0.25': 0.26576576576576577, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.08661417322834646, 'curtain_rec_0.25': 0.04477611940298507, 'refrigerator_rec_0.25': 0.03508771929824561, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.0, 'sink_rec_0.25': 0.01020408163265306, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.21132075471698114, 'mAR_0.25': 0.11298182832749847, 'cabinet_AP_0.50': 0.005249418318271637, 'bed_AP_0.50': 7.024568276392529e-06, 'chair_AP_0.50': 0.013795703649520874, 'sofa_AP_0.50': 0.0005232820985838771, 'table_AP_0.50': 0.001064108102582395, 'door_AP_0.50': 0.007914243265986443, 'window_AP_0.50': 0.0021169877145439386, 'bookshelf_AP_0.50': 0.0042017120867967606, 'picture_AP_0.50': 0.0025250331964343786, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.0010463166981935501, 'curtain_AP_0.50': 2.2466720110969618e-05, 'refrigerator_AP_0.50': 0.0, 'showercurtrain_AP_0.50': 0.0, 'toilet_AP_0.50': 0.0, 'sink_AP_0.50': 0.0, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.003941362723708153, 'mAP_0.50': 0.0023559811525046825, 'cabinet_rec_0.
50': 0.24193548387096775, 'bed_rec_0.50': 0.024691358024691357, 'chair_rec_0.50': 0.23538011695906433, 'sofa_rec_0.50': 0.08247422680412371,
'table_rec_0.50': 0.12285714285714286, 'door_rec_0.50': 0.23126338329764454, 'window_rec_0.50': 0.16312056737588654, 'bookshelf_rec_0.50':
0.14285714285714285, 'picture_rec_0.50': 0.25675675675675674, 'counter_rec_0.50': 0.0, 'desk_rec_0.50': 0.07086614173228346, 'curtain_rec_0.
50': 0.04477611940298507, 'refrigerator_rec_0.50': 0.0, 'showercurtrain_rec_0.50': 0.0, 'toilet_rec_0.50': 0.0, 'sink_rec_0.50': 0.0, 'batht
ub_rec_0.50': 0.0, 'garbagebin_rec_0.50': 0.1811320754716981, 'mAR_0.50': 0.0998950286339104}
2.Evaluation results for the model trained by myself：(12 epoches)
2.1 sunrgbd
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/5050, elapsed: [>>>>>>>>>>>>>>>] 5050/5050, 0.3 task/s, elapsed: 16300s, ETA: 0s
+-------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+-------------+---------+---------+---------+---------+
| bed | 0.5560 | 0.9903 | 0.4873 | 0.8738 |
| table | 0.3110 | 0.9817 | 0.2184 | 0.7785 |
| sofa | 0.4583 | 0.9841 | 0.4016 | 0.8740 |
| chair | 0.5615 | 0.9633 | 0.4967 | 0.8643 |
| toilet | 0.6838 | 0.9931 | 0.6078 | 0.8966 |
| desk | 0.1938 | 0.9644 | 0.0907 | 0.6791 |
| dresser | 0.2366 | 0.9541 | 0.1876 | 0.7752 |
| night_stand | 0.4379 | 0.9765 | 0.3945 | 0.8824 |
| bookshelf | 0.2321 | 0.9184 | 0.0987 | 0.5426 |
| bathtub | 0.3912 | 0.9796 | 0.3296 | 0.8367 |
+-------------+---------+---------+---------+---------+
| Overall | 0.4062 | 0.9705 | 0.3313 | 0.8003 |
+-------------+---------+---------+---------+---------+
{'bed_AP_0.25': 0.5559849143028259, 'table_AP_0.25': 0.3109968602657318, 'sofa_AP_0.25': 0.4582793712615967, 'chair_AP_0.25': 0.5614634156227112, 'toilet_AP_0.25': 0.6837515830993652, 'desk_AP_0.25': 0.1938285529613495, 'dresser_AP_0.25': 0.23655451834201813, 'night_stand_AP_0.25': 0.43787485361099243, 'bookshelf_AP_0.25': 0.2321251481771469, 'bathtub_AP_0.25': 0.3912363648414612, 'mAP_0.25': 0.4062095582485199, 'bed_rec_0.25': 0.9902912621359223, 'table_rec_0.25': 0.981686541737649, 'sofa_rec_0.25': 0.9840510366826156, 'chair_rec_0.25': 0.963258785942492, 'toilet_rec_0.25': 0.993103448275862, 'desk_rec_0.25': 0.9643995749202976, 'dresser_rec_0.25': 0.9541284403669725, 'night_stand_rec_0.25': 0.9764705882352941, 'bookshelf_rec_0.25': 0.9184397163120568, 'bathtub_rec_0.25': 0.9795918367346939, 'mAR_0.25': 0.9705421231343856, 'bed_AP_0.50': 0.4872896671295166, 'table_AP_0.50': 0.21839171648025513, 'sofa_AP_0.50': 0.40156134963035583, 'chair_AP_0.50': 0.4966593384742737, 'toilet_AP_0.50': 0.607752799987793, 'desk_AP_0.50': 0.09067006409168243, 'dresser_AP_0.50': 0.18762624263763428, 'night_stand_AP_0.50': 0.39452967047691345, 'bookshelf_AP_0.50': 0.09874459356069565, 'bathtub_AP_0.50': 0.32958272099494934, 'mAP_0.50': 0.33128082752227783, 'bed_rec_0.50': 0.8737864077669902, 'table_rec_0.50': 0.778534923339012, 'sofa_rec_0.50': 0.8740031897926634, 'chair_rec_0.50': 0.8643170926517572, 'toilet_rec_0.50': 0.896551724137931, 'desk_rec_0.50': 0.6790648246546227, 'dresser_rec_0.50': 0.7752293577981652, 'night_stand_rec_0.50': 0.8823529411764706, 'bookshelf_rec_0.50': 0.5425531914893617, 'bathtub_rec_0.50': 0.8367346938775511, 'mAR_0.50': 0.8003128346684527}
2.2 scannet
Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tun$
the variable for optimal performance in your application as needed.
Use load_from_local loader
Use load_from_local loader
[ ] 0/312, elapsed: 0[>>>>>>> ] 116/312, 0.2 task/s, elapsed[ ] 117/312, 0.2 t$
sk/s, elapsed: 547s,[ ] 118/312, 0.2 task/s, elapsed: 547s, ETA: 89[>>>>>>>>>>>>> ] 240/312, [> ] 241/312, 0.2 task/s, elapsed: 1262$
[> ] 242/312, 0.2 task/s, elapsed: 1262s[> ] 243/312, 0.2 task/s, elap[>>>>>>>>>>>>>>>>>>] 312/312, 0.2 task/s, elapsed: 1531s, ETA: 0s
+----------------+---------+---------+---------+---------+
| classes | AP_0.25 | AR_0.25 | AP_0.50 | AR_0.50 |
+----------------+---------+---------+---------+---------+
| cabinet | 0.0089 | 0.2500 | 0.0071 | 0.2419 |
| bed | 0.0022 | 0.0494 | 0.0005 | 0.0123 |
| chair | 0.0195 | 0.2661 | 0.0131 | 0.2383 |
| sofa | 0.0009 | 0.0825 | 0.0000 | 0.0309 |
| table | 0.0030 | 0.1400 | 0.0021 | 0.1257 |
| door | 0.0072 | 0.2698 | 0.0066 | 0.2548 |
| window | 0.0030 | 0.1844 | 0.0020 | 0.1525 |
| bookshelf | 0.0041 | 0.1429 | 0.0031 | 0.1429 |
| picture | 0.0049 | 0.2523 | 0.0045 | 0.2342 |
| counter | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| desk | 0.0041 | 0.0709 | 0.0041 | 0.0551 |
| curtain | 0.0001 | 0.0597 | 0.0001 | 0.0597 |
| refrigerator | 0.0001 | 0.0877 | 0.0001 | 0.0877 |
| showercurtrain | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| toilet | 0.0005 | 0.0345 | 0.0001 | 0.0345 |
| sink | 0.0000 | 0.0408 | 0.0000 | 0.0408 |
| bathtub | 0.0000 | 0.0000 | 0.0000 | 0.0000 |
| garbagebin | 0.0076 | 0.2038 | 0.0063 | 0.1887 |
+----------------+---------+---------+---------+---------+
| Overall | 0.0037 | 0.1186 | 0.0028 | 0.1056 |
+----------------+---------+---------+---------+---------+
{'cabinet_AP_0.25': 0.0088879419490695, 'bed_AP_0.25': 0.002196365036070347, 'chair_AP_0.25': 0.019513951614499092, 'sofa_AP_0.25': 0.00092$
0919025614858, 'table_AP_0.25': 0.0029890856239944696, 'door_AP_0.25': 0.007219596300274134, 'window_AP_0.25': 0.002968191634863615, 'books$
elf_AP_0.25': 0.0041231452487409115, 'picture_AP_0.25': 0.004887890536338091, 'counter_AP_0.25': 0.0, 'desk_AP_0.25': 0.004088526125997305,
'curtain_AP_0.25': 0.0001449216069886461, 'refrigerator_AP_0.25': 0.00010905037925112993, 'showercurtrain_AP_0.25': 0.0, 'toilet_AP_0.25': $
.0005089639453217387, 'sink_AP_0.25': 3.358132016728632e-05, 'bathtub_AP_0.25': 0.0, 'garbagebin_AP_0.25': 0.007584888022392988, 'mAP_0.25'$
0.003676677355542779, 'cabinet_rec_0.25': 0.25, 'bed_rec_0.25': 0.04938271604938271, 'chair_rec_0.25': 0.26608187134502925, 'sofa_rec_0.25$
: 0.08247422680412371, 'table_rec_0.25': 0.14, 'door_rec_0.25': 0.2698072805139186, 'window_rec_0.25': 0.18439716312056736, 'bookshelf_rec_0
.25': 0.14285714285714285, 'picture_rec_0.25': 0.25225225225225223, 'counter_rec_0.25': 0.0, 'desk_rec_0.25': 0.07086614173228346, 'curtain_
rec_0.25': 0.05970149253731343, 'refrigerator_rec_0.25': 0.08771929824561403, 'showercurtrain_rec_0.25': 0.0, 'toilet_rec_0.25': 0.034482758
620689655, 'sink_rec_0.25': 0.04081632653061224, 'bathtub_rec_0.25': 0.0, 'garbagebin_rec_0.25': 0.2037735849056604, 'mAR_0.25': 0.118589569
75081056, 'cabinet_AP_0.50': 0.007079910486936569, 'bed_AP_0.50': 0.0005367686389945447, 'chair_AP_0.50': 0.013134335167706013, 'sofa_AP_0.5
0': 4.525698022916913e-05, 'table_AP_0.50': 0.0021134275011718273, 'door_AP_0.50': 0.00658177025616169, 'window_AP_0.50': 0.0019997931085526
943, 'bookshelf_AP_0.50': 0.0031221157405525446, 'picture_AP_0.50': 0.004526467993855476, 'counter_AP_0.50': 0.0, 'desk_AP_0.50': 0.00407800
916582346, 'curtain_AP_0.50': 0.0001449216069886461, 'refrigerator_AP_0.50': 0.0001018800976453349, 'showercurtrain_AP_0.50': 0.0, 'toilet_A
P_0.50': 5.409148434409872e-05, 'sink_AP_0.50': 3.358132016728632e-05, 'bathtub_AP_0.50': 0.0, 'garbagebin_AP_0.50': 0.006303954869508743, '
mAP_0.50': 0.0027697933837771416, 'cabinet_rec_0.50': 0.24193548387096775, 'bed_rec_0.50': 0.012345679012345678, 'chair_rec_0.50': 0.2383040
9356725146, 'sofa_rec_0.50': 0.030927835051546393, 'table_rec_0.50': 0.12571428571428572, 'door_rec_0.50': 0.25481798715203424, 'window_rec_
0.50': 0.1524822695035461, 'bookshelf_rec_0.50': 0.14285714285714285, 'picture_rec_0.50': 0.23423423423423423, 'counter_rec_0.50': 0.0, 'des
k_rec_0.50': 0.05511811023622047, 'curtain_rec_0.50': 0.05970149253731343, 'refrigerator_rec_0.50': 0.08771929824561403, 'showercurtrain_rec
396 Dec 01, 2022
181 Dec 09, 2022
8 Aug 18, 2022
73 Oct 24, 2022
7 Dec 01, 2022
1.3k Dec 25, 2022
1 Feb 16, 2022
8 Oct 21, 2022
82 Dec 29, 2022
540 Dec 30, 2022
2 Oct 25, 2022
2.4k Jan 06, 2023
11 Dec 03, 2022
1.1k Jan 01, 2023
6 Jun 29, 2022
11 Dec 01, 2022
14 Oct 27, 2022
35 Dec 30, 2022
5.5k Jan 04, 2023
42 Jan 15, 2022