I tried to train on my custom data in coco format but no matter how I train for 10 epochs or 300 epochs I still get mAP=0,my custom data is originally in YOLOV5 format so I use this https://github.com/RapidAI/YOLO2COCO to convert my YOLOV5 format data into coco format and I also check the label which is correct.Besides I also have changed the classes in yolox/data/datasets/coco_classes.py to my own classes.However the result disappointed me again,then I tried to use the datasets https://drive.google.com/file/d/16N3u36ycNd70m23IM7vMuRQXejAJY9Fs/view?usp=sharing you guys suggest in the train_custom_data.md,but for well-known reasons in China I can hardly download it,so I also suggest you provide a BaiduYun version which is more friendly to our students. Here is my train_log.txt: 2021-08-15 21:54:11.090 | INFO | yolox.core.trainer:before_train:126 - args: Namespace(experiment_name='606', name=None, dist_backend='nccl', dist_url=None, batch_size=8, devices=0, exp_file='exps/example/custom/606.py', resume=False, ckpt='yolox_s.pth', start_epoch=None, num_machines=1, machine_rank=0, fp16=True, occupy=False, opts=[]) 2021-08-15 21:54:11.091 | INFO | yolox.core.trainer:before_train:127 - exp value: ╒══════════════════╤════════════════════════════╕ │ keys │ values │ ╞══════════════════╪════════════════════════════╡ │ seed │ None │ ├──────────────────┼────────────────────────────┤ │ output_dir │ './YOLOX_outputs' │ ├──────────────────┼────────────────────────────┤ │ print_interval │ 10 │ ├──────────────────┼────────────────────────────┤ │ eval_interval │ 1 │ ├──────────────────┼────────────────────────────┤ │ num_classes │ 8 │ ├──────────────────┼────────────────────────────┤ │ depth │ 0.33 │ ├──────────────────┼────────────────────────────┤ │ width │ 0.5 │ ├──────────────────┼────────────────────────────┤ │ data_num_workers │ 0 │ ├──────────────────┼────────────────────────────┤ │ input_size │ (640, 640) │ ├──────────────────┼────────────────────────────┤ │ random_size │ (14, 26) │ ├──────────────────┼────────────────────────────┤ │ data_dir │ 'datasets/coco128' │ ├──────────────────┼────────────────────────────┤ │ train_ann │ 'instances_train2017.json' │ ├──────────────────┼────────────────────────────┤ │ val_ann │ 'instances_val2017.json' │ ├──────────────────┼────────────────────────────┤ │ degrees │ 10.0 │ ├──────────────────┼────────────────────────────┤ │ translate │ 0.1 │ ├──────────────────┼────────────────────────────┤ │ scale │ (0.1, 2) │ ├──────────────────┼────────────────────────────┤ │ mscale │ (0.8, 1.6) │ ├──────────────────┼────────────────────────────┤ │ shear │ 2.0 │ ├──────────────────┼────────────────────────────┤ │ perspective │ 0.0 │ ├──────────────────┼────────────────────────────┤ │ enable_mixup │ True │ ├──────────────────┼────────────────────────────┤ │ warmup_epochs │ 5 │ ├──────────────────┼────────────────────────────┤ │ max_epoch │ 10 │ ├──────────────────┼────────────────────────────┤ │ warmup_lr │ 0 │ ├──────────────────┼────────────────────────────┤ │ basic_lr_per_img │ 0.00015625 │ ├──────────────────┼────────────────────────────┤ │ scheduler │ 'yoloxwarmcos' │ ├──────────────────┼────────────────────────────┤ │ no_aug_epochs │ 15 │ ├──────────────────┼────────────────────────────┤ │ min_lr_ratio │ 0.05 │ ├──────────────────┼────────────────────────────┤ │ ema │ True │ ├──────────────────┼────────────────────────────┤ │ weight_decay │ 0.0005 │ ├──────────────────┼────────────────────────────┤ │ momentum │ 0.9 │ ├──────────────────┼────────────────────────────┤ │ exp_name │ '606' │ ├──────────────────┼────────────────────────────┤ │ test_size │ (640, 640) │ ├──────────────────┼────────────────────────────┤ │ test_conf │ 0.01 │ ├──────────────────┼────────────────────────────┤ │ nmsthre │ 0.65 │ ╘══════════════════╧════════════════════════════╛ 2021-08-15 21:54:11.239 | INFO | yolox.core.trainer:before_train:132 - Model Summary: Params: 8.94M, Gflops: 26.65 2021-08-15 21:54:13.208 | INFO | apex.amp.frontend:initialize:328 - Selected optimization level O1: Insert automatic casts around Pytorch functions and Tensor methods. 2021-08-15 21:54:13.208 | INFO | apex.amp.frontend:initialize:329 - Defaults for this optimization level are: 2021-08-15 21:54:13.209 | INFO | apex.amp.frontend:initialize:331 - enabled : True 2021-08-15 21:54:13.209 | INFO | apex.amp.frontend:initialize:331 - opt_level : O1 2021-08-15 21:54:13.209 | INFO | apex.amp.frontend:initialize:331 - cast_model_type : None 2021-08-15 21:54:13.210 | INFO | apex.amp.frontend:initialize:331 - patch_torch_functions : True 2021-08-15 21:54:13.210 | INFO | apex.amp.frontend:initialize:331 - keep_batchnorm_fp32 : None 2021-08-15 21:54:13.210 | INFO | apex.amp.frontend:initialize:331 - master_weights : None 2021-08-15 21:54:13.210 | INFO | apex.amp.frontend:initialize:331 - loss_scale : dynamic 2021-08-15 21:54:13.211 | INFO | apex.amp.frontend:initialize:336 - Processing user overrides (additional kwargs that are not None)... 2021-08-15 21:54:13.211 | INFO | apex.amp.frontend:initialize:354 - After processing overrides, optimization options are: 2021-08-15 21:54:13.211 | INFO | apex.amp.frontend:initialize:356 - enabled : True 2021-08-15 21:54:13.212 | INFO | apex.amp.frontend:initialize:356 - opt_level : O1 2021-08-15 21:54:13.213 | INFO | apex.amp.frontend:initialize:356 - cast_model_type : None 2021-08-15 21:54:13.214 | INFO | apex.amp.frontend:initialize:356 - patch_torch_functions : True 2021-08-15 21:54:13.215 | INFO | apex.amp.frontend:initialize:356 - keep_batchnorm_fp32 : None 2021-08-15 21:54:13.216 | INFO | apex.amp.frontend:initialize:356 - master_weights : None 2021-08-15 21:54:13.217 | INFO | apex.amp.frontend:initialize:356 - loss_scale : dynamic 2021-08-15 21:54:13.221 | INFO | apex.amp.scaler:init:64 - Warning: multi_tensor_applier fused unscale kernel is unavailable, possibly because apex was installed without --cuda_ext --cpp_ext. Using Python fallback. Original ImportError was: ModuleNotFoundError("No module named 'amp_C'") 2021-08-15 21:54:13.223 | INFO | yolox.core.trainer:resume_train:292 - loading checkpoint for fine tuning 2021-08-15 21:54:13.351 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.0.weight in checkpoint is torch.Size([80, 128, 1, 1]), while shape of head.cls_preds.0.weight in model is torch.Size([8, 128, 1, 1]). 2021-08-15 21:54:13.352 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.0.bias in checkpoint is torch.Size([80]), while shape of head.cls_preds.0.bias in model is torch.Size([8]). 2021-08-15 21:54:13.352 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.1.weight in checkpoint is torch.Size([80, 128, 1, 1]), while shape of head.cls_preds.1.weight in model is torch.Size([8, 128, 1, 1]). 2021-08-15 21:54:13.352 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.1.bias in checkpoint is torch.Size([80]), while shape of head.cls_preds.1.bias in model is torch.Size([8]). 2021-08-15 21:54:13.352 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.2.weight in checkpoint is torch.Size([80, 128, 1, 1]), while shape of head.cls_preds.2.weight in model is torch.Size([8, 128, 1, 1]). 2021-08-15 21:54:13.353 | WARNING | yolox.utils.checkpoint:load_ckpt:24 - Shape of head.cls_preds.2.bias in checkpoint is torch.Size([80]), while shape of head.cls_preds.2.bias in model is torch.Size([8]). 2021-08-15 21:54:13.377 | INFO | yolox.data.datasets.coco:init:43 - loading annotations into memory... 2021-08-15 21:54:13.405 | INFO | yolox.data.datasets.coco:init:43 - Done (t=0.03s) 2021-08-15 21:54:13.406 | INFO | pycocotools.coco:init:89 - creating index... 2021-08-15 21:54:13.407 | INFO | pycocotools.coco:init:89 - index created! 2021-08-15 21:54:13.433 | INFO | yolox.core.trainer:before_train:153 - init prefetcher, this might take one minute or less... 2021-08-15 21:54:13.746 | INFO | yolox.data.datasets.coco:init:43 - loading annotations into memory... 2021-08-15 21:54:13.749 | INFO | yolox.data.datasets.coco:init:43 - Done (t=0.00s) 2021-08-15 21:54:13.749 | INFO | pycocotools.coco:init:89 - creating index... 2021-08-15 21:54:13.749 | INFO | pycocotools.coco:init:89 - index created! 2021-08-15 21:54:13.759 | INFO | yolox.core.trainer:before_train:183 - Training start... 2021-08-15 21:54:13.762 | INFO | yolox.core.trainer:before_train:184 - YOLOX( (backbone): YOLOPAFPN( (backbone): CSPDarknet( (stem): Focus( (conv): BaseConv( (conv): Conv2d(12, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (dark2): Sequential( (0): BaseConv( (conv): Conv2d(32, 64, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): CSPLayer( (conv1): BaseConv( (conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(64, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(32, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) ) (dark3): Sequential( (0): BaseConv( (conv): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): CSPLayer( (conv1): BaseConv( (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (1): Bottleneck( (conv1): BaseConv( (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (2): Bottleneck( (conv1): BaseConv( (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) ) (dark4): Sequential( (0): BaseConv( (conv): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): CSPLayer( (conv1): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (1): Bottleneck( (conv1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (2): Bottleneck( (conv1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) ) (dark5): Sequential( (0): BaseConv( (conv): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(512, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): SPPBottleneck( (conv1): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): ModuleList( (0): MaxPool2d(kernel_size=5, stride=1, padding=2, dilation=1, ceil_mode=False) (1): MaxPool2d(kernel_size=9, stride=1, padding=4, dilation=1, ceil_mode=False) (2): MaxPool2d(kernel_size=13, stride=1, padding=6, dilation=1, ceil_mode=False) ) (conv2): BaseConv( (conv): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(512, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (2): CSPLayer( (conv1): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(512, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) ) ) (upsample): Upsample(scale_factor=2.0, mode=nearest) (lateral_conv0): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (C3_p4): CSPLayer( (conv1): BaseConv( (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) (reduce_conv1): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (C3_p3): CSPLayer( (conv1): BaseConv( (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(64, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) (bu_conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (C3_n3): CSPLayer( (conv1): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) (bu_conv1): BaseConv( (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (C3_n4): CSPLayer( (conv1): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv3): BaseConv( (conv): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(512, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (m): Sequential( (0): Bottleneck( (conv1): BaseConv( (conv): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (conv2): BaseConv( (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(256, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) ) ) (head): YOLOXHead( (cls_convs): ModuleList( (0): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (1): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (2): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) (reg_convs): ModuleList( (0): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (1): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (2): Sequential( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) ) (cls_preds): ModuleList( (0): Conv2d(128, 8, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(128, 8, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(128, 8, kernel_size=(1, 1), stride=(1, 1)) ) (reg_preds): ModuleList( (0): Conv2d(128, 4, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(128, 4, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(128, 4, kernel_size=(1, 1), stride=(1, 1)) ) (obj_preds): ModuleList( (0): Conv2d(128, 1, kernel_size=(1, 1), stride=(1, 1)) (1): Conv2d(128, 1, kernel_size=(1, 1), stride=(1, 1)) (2): Conv2d(128, 1, kernel_size=(1, 1), stride=(1, 1)) ) (stems): ModuleList( (0): BaseConv( (conv): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (1): BaseConv( (conv): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) (2): BaseConv( (conv): Conv2d(512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False) (bn): BatchNorm2d(128, eps=0.001, momentum=0.03, affine=True, track_running_stats=True) (act): SiLU(inplace=True) ) ) (l1_loss): L1Loss() (bcewithlog_loss): BCEWithLogitsLoss() (iou_loss): IOUloss() ) ) 2021-08-15 21:54:13.766 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch1 2021-08-15 21:54:13.766 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:54:13.767 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:54:16.905 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 32768.0 2021-08-15 21:54:17.337 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 16384.0 2021-08-15 21:54:17.771 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 8192.0 2021-08-15 21:54:18.230 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 4096.0 2021-08-15 21:54:18.684 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 2048.0 2021-08-15 21:54:19.135 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 1024.0 2021-08-15 21:54:20.592 | INFO | apex.amp.handle:skip_step:138 - Gradient overflow. Skipping step, loss scaler 0 reducing loss scale to 512.0 2021-08-15 21:54:21.105 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 10/88, mem: 3017Mb, iter_time: 0.733s, data_time: 0.208s, total_loss: 21.5, iou_loss: 4.2, l1_loss: 3.2, conf_loss: 12.5, cls_loss: 1.5, lr: 6.457e-07, size: 640, ETA: 0:10:38 2021-08-15 21:54:25.868 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 20/88, mem: 3017Mb, iter_time: 0.476s, data_time: 0.201s, total_loss: 18.7, iou_loss: 4.2, l1_loss: 2.9, conf_loss: 10.2, cls_loss: 1.5, lr: 2.583e-06, size: 640, ETA: 0:08:39 2021-08-15 21:54:33.831 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 30/88, mem: 3881Mb, iter_time: 0.796s, data_time: 0.254s, total_loss: 16.9, iou_loss: 4.1, l1_loss: 3.2, conf_loss: 8.0, cls_loss: 1.6, lr: 5.811e-06, size: 704, ETA: 0:09:28 2021-08-15 21:54:41.413 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 40/88, mem: 3881Mb, iter_time: 0.758s, data_time: 0.219s, total_loss: 14.7, iou_loss: 3.9, l1_loss: 2.7, conf_loss: 6.7, cls_loss: 1.3, lr: 1.033e-05, size: 672, ETA: 0:09:40 2021-08-15 21:54:46.458 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 50/88, mem: 3881Mb, iter_time: 0.504s, data_time: 0.219s, total_loss: 13.2, iou_loss: 3.5, l1_loss: 2.6, conf_loss: 6.0, cls_loss: 1.2, lr: 1.614e-05, size: 672, ETA: 0:09:02 2021-08-15 21:54:51.804 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 60/88, mem: 3881Mb, iter_time: 0.534s, data_time: 0.146s, total_loss: 12.2, iou_loss: 3.7, l1_loss: 2.3, conf_loss: 5.3, cls_loss: 0.9, lr: 2.324e-05, size: 512, ETA: 0:08:39 2021-08-15 21:54:55.538 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 70/88, mem: 3881Mb, iter_time: 0.373s, data_time: 0.143s, total_loss: 11.0, iou_loss: 3.5, l1_loss: 1.9, conf_loss: 4.8, cls_loss: 0.8, lr: 3.164e-05, size: 512, ETA: 0:08:02 2021-08-15 21:55:02.818 | INFO | yolox.core.trainer:after_iter:242 - epoch: 1/10, iter: 80/88, mem: 3881Mb, iter_time: 0.728s, data_time: 0.184s, total_loss: 8.8, iou_loss: 2.8, l1_loss: 1.4, conf_loss: 3.9, cls_loss: 0.7, lr: 4.132e-05, size: 576, ETA: 0:08:10 2021-08-15 21:55:10.084 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:55:17.377 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:55:17.451 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:55:17.473 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.02s) 2021-08-15 21:55:17.474 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:55:17.475 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:55:17.546 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.26 ms, Average NMS time: 1.11 ms, Average inference time: 7.37 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:55:17.547 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:55:17.696 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch2 2021-08-15 21:55:17.696 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:55:17.697 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:55:22.867 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 10/88, mem: 3881Mb, iter_time: 0.517s, data_time: 0.223s, total_loss: 7.6, iou_loss: 2.6, l1_loss: 1.3, conf_loss: 3.1, cls_loss: 0.7, lr: 6.201e-05, size: 640, ETA: 0:08:10 2021-08-15 21:55:26.814 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 20/88, mem: 3881Mb, iter_time: 0.394s, data_time: 0.143s, total_loss: 7.0, iou_loss: 2.6, l1_loss: 1.2, conf_loss: 2.5, cls_loss: 0.7, lr: 7.531e-05, size: 512, ETA: 0:07:47 2021-08-15 21:55:32.197 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 30/88, mem: 3881Mb, iter_time: 0.538s, data_time: 0.140s, total_loss: 5.2, iou_loss: 2.0, l1_loss: 0.8, conf_loss: 1.9, cls_loss: 0.6, lr: 8.990e-05, size: 544, ETA: 0:07:36 2021-08-15 21:55:40.492 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 40/88, mem: 3881Mb, iter_time: 0.829s, data_time: 0.271s, total_loss: 6.5, iou_loss: 2.1, l1_loss: 1.2, conf_loss: 2.6, cls_loss: 0.6, lr: 1.058e-04, size: 800, ETA: 0:07:44 2021-08-15 21:55:45.579 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 50/88, mem: 3881Mb, iter_time: 0.508s, data_time: 0.204s, total_loss: 5.5, iou_loss: 2.0, l1_loss: 0.9, conf_loss: 2.1, cls_loss: 0.5, lr: 1.230e-04, size: 640, ETA: 0:07:32 2021-08-15 21:55:53.270 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 60/88, mem: 3881Mb, iter_time: 0.769s, data_time: 0.252s, total_loss: 4.2, iou_loss: 1.6, l1_loss: 0.8, conf_loss: 1.5, cls_loss: 0.5, lr: 1.414e-04, size: 736, ETA: 0:07:34 2021-08-15 21:55:57.769 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 70/88, mem: 3881Mb, iter_time: 0.449s, data_time: 0.175s, total_loss: 5.2, iou_loss: 2.2, l1_loss: 0.9, conf_loss: 1.5, cls_loss: 0.6, lr: 1.612e-04, size: 544, ETA: 0:07:20 2021-08-15 21:56:01.607 | INFO | yolox.core.trainer:after_iter:242 - epoch: 2/10, iter: 80/88, mem: 3881Mb, iter_time: 0.383s, data_time: 0.136s, total_loss: 3.3, iou_loss: 1.3, l1_loss: 0.5, conf_loss: 1.0, cls_loss: 0.4, lr: 1.822e-04, size: 512, ETA: 0:07:04 2021-08-15 21:56:04.585 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:56:10.799 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:56:10.809 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:56:10.818 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 21:56:10.819 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:56:10.819 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:56:10.856 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.30 ms, Average NMS time: 1.09 ms, Average inference time: 7.39 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:56:10.856 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:56:11.022 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch3 2021-08-15 21:56:11.023 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:56:11.023 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:56:16.385 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 10/88, mem: 3881Mb, iter_time: 0.536s, data_time: 0.226s, total_loss: 7.6, iou_loss: 2.2, l1_loss: 1.1, conf_loss: 3.6, cls_loss: 0.7, lr: 2.234e-04, size: 800, ETA: 0:06:44 2021-08-15 21:56:21.538 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 20/88, mem: 3881Mb, iter_time: 0.514s, data_time: 0.203s, total_loss: 5.3, iou_loss: 2.3, l1_loss: 1.0, conf_loss: 1.4, cls_loss: 0.6, lr: 2.480e-04, size: 576, ETA: 0:06:36 2021-08-15 21:56:26.631 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 30/88, mem: 3881Mb, iter_time: 0.509s, data_time: 0.218s, total_loss: 6.6, iou_loss: 2.5, l1_loss: 1.5, conf_loss: 1.9, cls_loss: 0.7, lr: 2.740e-04, size: 736, ETA: 0:06:28 2021-08-15 21:56:31.465 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 40/88, mem: 3881Mb, iter_time: 0.483s, data_time: 0.191s, total_loss: 5.9, iou_loss: 2.5, l1_loss: 1.0, conf_loss: 1.7, cls_loss: 0.8, lr: 3.012e-04, size: 544, ETA: 0:06:19 2021-08-15 21:56:35.555 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 50/88, mem: 3881Mb, iter_time: 0.408s, data_time: 0.150s, total_loss: 3.5, iou_loss: 1.5, l1_loss: 0.5, conf_loss: 1.1, cls_loss: 0.5, lr: 3.298e-04, size: 544, ETA: 0:06:09 2021-08-15 21:56:40.253 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 60/88, mem: 3881Mb, iter_time: 0.469s, data_time: 0.191s, total_loss: 4.6, iou_loss: 2.1, l1_loss: 1.0, conf_loss: 0.8, cls_loss: 0.7, lr: 3.596e-04, size: 640, ETA: 0:06:01 2021-08-15 21:56:44.927 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 70/88, mem: 3881Mb, iter_time: 0.467s, data_time: 0.189s, total_loss: 3.2, iou_loss: 1.6, l1_loss: 0.6, conf_loss: 0.4, cls_loss: 0.5, lr: 3.907e-04, size: 576, ETA: 0:05:53 2021-08-15 21:56:49.526 | INFO | yolox.core.trainer:after_iter:242 - epoch: 3/10, iter: 80/88, mem: 3881Mb, iter_time: 0.459s, data_time: 0.126s, total_loss: 9.3, iou_loss: 3.6, l1_loss: 2.1, conf_loss: 2.8, cls_loss: 0.8, lr: 4.231e-04, size: 448, ETA: 0:05:45 2021-08-15 21:56:52.664 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:56:58.751 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:56:58.770 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:56:58.779 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 21:56:58.779 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:56:58.780 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:56:58.851 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.29 ms, Average NMS time: 1.10 ms, Average inference time: 7.39 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:56:58.852 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:56:59.009 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch4 2021-08-15 21:56:59.010 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:56:59.010 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:57:03.834 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 10/88, mem: 3881Mb, iter_time: 0.482s, data_time: 0.194s, total_loss: 3.5, iou_loss: 1.5, l1_loss: 0.6, conf_loss: 0.9, cls_loss: 0.5, lr: 4.847e-04, size: 640, ETA: 0:05:30 2021-08-15 21:57:08.106 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 20/88, mem: 3881Mb, iter_time: 0.427s, data_time: 0.159s, total_loss: 11.3, iou_loss: 4.4, l1_loss: 3.9, conf_loss: 2.2, cls_loss: 0.8, lr: 5.208e-04, size: 448, ETA: 0:05:22 2021-08-15 21:57:12.441 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 30/88, mem: 3881Mb, iter_time: 0.433s, data_time: 0.108s, total_loss: 5.2, iou_loss: 2.3, l1_loss: 0.9, conf_loss: 1.5, cls_loss: 0.6, lr: 5.581e-04, size: 480, ETA: 0:05:15 2021-08-15 21:57:16.038 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 40/88, mem: 3881Mb, iter_time: 0.359s, data_time: 0.125s, total_loss: 4.3, iou_loss: 2.0, l1_loss: 0.8, conf_loss: 0.8, cls_loss: 0.7, lr: 5.967e-04, size: 544, ETA: 0:05:06 2021-08-15 21:57:19.877 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 50/88, mem: 3881Mb, iter_time: 0.384s, data_time: 0.136s, total_loss: 2.7, iou_loss: 1.3, l1_loss: 0.4, conf_loss: 0.6, cls_loss: 0.4, lr: 6.366e-04, size: 512, ETA: 0:04:58 2021-08-15 21:57:24.603 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 60/88, mem: 3881Mb, iter_time: 0.472s, data_time: 0.196s, total_loss: 11.9, iou_loss: 3.5, l1_loss: 2.2, conf_loss: 5.3, cls_loss: 0.9, lr: 6.778e-04, size: 800, ETA: 0:04:52 2021-08-15 21:57:30.386 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 70/88, mem: 3881Mb, iter_time: 0.578s, data_time: 0.243s, total_loss: 7.8, iou_loss: 2.7, l1_loss: 1.4, conf_loss: 3.1, cls_loss: 0.7, lr: 7.203e-04, size: 576, ETA: 0:04:47 2021-08-15 21:57:34.968 | INFO | yolox.core.trainer:after_iter:242 - epoch: 4/10, iter: 80/88, mem: 3881Mb, iter_time: 0.457s, data_time: 0.187s, total_loss: 5.8, iou_loss: 2.3, l1_loss: 1.2, conf_loss: 1.8, cls_loss: 0.6, lr: 7.640e-04, size: 704, ETA: 0:04:41 2021-08-15 21:57:38.888 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:57:45.180 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:57:45.223 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:57:45.241 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.02s) 2021-08-15 21:57:45.241 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:57:45.242 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:57:45.331 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.29 ms, Average NMS time: 1.14 ms, Average inference time: 7.43 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:57:45.332 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:57:45.498 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch5 2021-08-15 21:57:45.498 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:57:45.498 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:57:50.643 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 10/88, mem: 3881Mb, iter_time: 0.513s, data_time: 0.117s, total_loss: 12.8, iou_loss: 3.9, l1_loss: 2.5, conf_loss: 5.6, cls_loss: 0.8, lr: 8.461e-04, size: 608, ETA: 0:04:31 2021-08-15 21:57:58.209 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 20/88, mem: 3881Mb, iter_time: 0.756s, data_time: 0.216s, total_loss: 16.1, iou_loss: 4.5, l1_loss: 4.9, conf_loss: 5.5, cls_loss: 1.2, lr: 8.935e-04, size: 832, ETA: 0:04:29 2021-08-15 21:58:04.978 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 30/88, mem: 3881Mb, iter_time: 0.676s, data_time: 0.296s, total_loss: 20.1, iou_loss: 4.8, l1_loss: 5.6, conf_loss: 8.5, cls_loss: 1.2, lr: 9.422e-04, size: 448, ETA: 0:04:25 2021-08-15 21:58:08.970 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 40/88, mem: 3881Mb, iter_time: 0.398s, data_time: 0.132s, total_loss: 8.5, iou_loss: 3.6, l1_loss: 2.0, conf_loss: 2.1, cls_loss: 0.8, lr: 9.921e-04, size: 544, ETA: 0:04:18 2021-08-15 21:58:13.747 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 50/88, mem: 3881Mb, iter_time: 0.477s, data_time: 0.198s, total_loss: 18.3, iou_loss: 4.0, l1_loss: 2.4, conf_loss: 10.6, cls_loss: 1.3, lr: 1.043e-03, size: 832, ETA: 0:04:12 2021-08-15 21:58:20.429 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 60/88, mem: 3881Mb, iter_time: 0.667s, data_time: 0.292s, total_loss: 11.5, iou_loss: 4.4, l1_loss: 2.9, conf_loss: 3.1, cls_loss: 1.1, lr: 1.096e-03, size: 544, ETA: 0:04:09 2021-08-15 21:58:24.532 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 70/88, mem: 3881Mb, iter_time: 0.410s, data_time: 0.155s, total_loss: 7.2, iou_loss: 3.2, l1_loss: 1.0, conf_loss: 2.1, cls_loss: 0.7, lr: 1.150e-03, size: 608, ETA: 0:04:02 2021-08-15 21:58:28.869 | INFO | yolox.core.trainer:after_iter:242 - epoch: 5/10, iter: 80/88, mem: 3881Mb, iter_time: 0.433s, data_time: 0.164s, total_loss: 11.0, iou_loss: 4.2, l1_loss: 1.8, conf_loss: 4.1, cls_loss: 1.0, lr: 1.205e-03, size: 480, ETA: 0:03:56 2021-08-15 21:58:31.620 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:58:38.010 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:58:38.093 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:58:38.158 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.06s) 2021-08-15 21:58:38.158 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:58:38.160 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:58:38.224 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.54 ms, Average NMS time: 1.14 ms, Average inference time: 7.68 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:58:38.225 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:58:38.376 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch6 2021-08-15 21:58:38.376 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:58:38.377 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:58:44.452 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 10/88, mem: 3881Mb, iter_time: 0.607s, data_time: 0.277s, total_loss: 10.6, iou_loss: 4.4, l1_loss: 2.5, conf_loss: 3.0, cls_loss: 0.7, lr: 6.250e-05, size: 768, ETA: 0:03:46 2021-08-15 21:58:47.853 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 20/88, mem: 3881Mb, iter_time: 0.340s, data_time: 0.101s, total_loss: 5.3, iou_loss: 2.6, l1_loss: 0.6, conf_loss: 1.4, cls_loss: 0.7, lr: 6.250e-05, size: 448, ETA: 0:03:39 2021-08-15 21:58:54.795 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 30/88, mem: 3881Mb, iter_time: 0.693s, data_time: 0.325s, total_loss: 12.1, iou_loss: 4.3, l1_loss: 2.6, conf_loss: 4.5, cls_loss: 0.7, lr: 6.250e-05, size: 832, ETA: 0:03:35 2021-08-15 21:58:58.460 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 40/88, mem: 3881Mb, iter_time: 0.366s, data_time: 0.117s, total_loss: 5.1, iou_loss: 2.0, l1_loss: 0.4, conf_loss: 2.1, cls_loss: 0.6, lr: 6.250e-05, size: 480, ETA: 0:03:28 2021-08-15 21:59:04.487 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 50/88, mem: 3881Mb, iter_time: 0.602s, data_time: 0.278s, total_loss: 9.1, iou_loss: 3.9, l1_loss: 1.8, conf_loss: 2.7, cls_loss: 0.7, lr: 6.250e-05, size: 768, ETA: 0:03:24 2021-08-15 21:59:10.483 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 60/88, mem: 3881Mb, iter_time: 0.599s, data_time: 0.269s, total_loss: 8.2, iou_loss: 3.2, l1_loss: 1.3, conf_loss: 3.0, cls_loss: 0.7, lr: 6.250e-05, size: 768, ETA: 0:03:19 2021-08-15 21:59:15.368 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 70/88, mem: 3881Mb, iter_time: 0.488s, data_time: 0.196s, total_loss: 4.9, iou_loss: 2.3, l1_loss: 0.6, conf_loss: 1.4, cls_loss: 0.6, lr: 6.250e-05, size: 640, ETA: 0:03:14 2021-08-15 21:59:21.506 | INFO | yolox.core.trainer:after_iter:242 - epoch: 6/10, iter: 80/88, mem: 3881Mb, iter_time: 0.613s, data_time: 0.283s, total_loss: 5.8, iou_loss: 2.4, l1_loss: 0.8, conf_loss: 2.0, cls_loss: 0.6, lr: 6.250e-05, size: 736, ETA: 0:03:09 2021-08-15 21:59:26.364 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:59:33.032 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 21:59:33.053 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 21:59:33.080 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.03s) 2021-08-15 21:59:33.080 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 21:59:33.081 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 21:59:33.134 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.32 ms, Average NMS time: 1.17 ms, Average inference time: 7.49 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 21:59:33.134 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 21:59:33.312 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch7 2021-08-15 21:59:33.313 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 21:59:33.313 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 21:59:38.130 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 10/88, mem: 3881Mb, iter_time: 0.481s, data_time: 0.192s, total_loss: 6.9, iou_loss: 3.5, l1_loss: 1.4, conf_loss: 1.3, cls_loss: 0.8, lr: 6.250e-05, size: 576, ETA: 0:03:00 2021-08-15 21:59:43.472 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 20/88, mem: 3881Mb, iter_time: 0.533s, data_time: 0.231s, total_loss: 4.0, iou_loss: 1.7, l1_loss: 0.5, conf_loss: 1.2, cls_loss: 0.6, lr: 6.250e-05, size: 704, ETA: 0:02:54 2021-08-15 21:59:48.087 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 30/88, mem: 3881Mb, iter_time: 0.461s, data_time: 0.181s, total_loss: 4.7, iou_loss: 2.4, l1_loss: 0.7, conf_loss: 1.1, cls_loss: 0.6, lr: 6.250e-05, size: 576, ETA: 0:02:49 2021-08-15 21:59:53.363 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 40/88, mem: 3881Mb, iter_time: 0.527s, data_time: 0.228s, total_loss: 5.8, iou_loss: 2.7, l1_loss: 0.9, conf_loss: 1.6, cls_loss: 0.6, lr: 6.250e-05, size: 736, ETA: 0:02:44 2021-08-15 21:59:58.929 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 50/88, mem: 3881Mb, iter_time: 0.556s, data_time: 0.244s, total_loss: 4.1, iou_loss: 1.7, l1_loss: 0.5, conf_loss: 1.3, cls_loss: 0.5, lr: 6.250e-05, size: 736, ETA: 0:02:38 2021-08-15 22:00:05.210 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 60/88, mem: 3881Mb, iter_time: 0.628s, data_time: 0.278s, total_loss: 3.8, iou_loss: 1.6, l1_loss: 0.5, conf_loss: 1.3, cls_loss: 0.5, lr: 6.250e-05, size: 800, ETA: 0:02:34 2021-08-15 22:00:09.728 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 70/88, mem: 3881Mb, iter_time: 0.451s, data_time: 0.167s, total_loss: 8.5, iou_loss: 3.8, l1_loss: 1.8, conf_loss: 2.0, cls_loss: 0.8, lr: 6.250e-05, size: 544, ETA: 0:02:28 2021-08-15 22:00:14.975 | INFO | yolox.core.trainer:after_iter:242 - epoch: 7/10, iter: 80/88, mem: 3881Mb, iter_time: 0.524s, data_time: 0.225s, total_loss: 3.0, iou_loss: 1.1, l1_loss: 0.3, conf_loss: 1.0, cls_loss: 0.6, lr: 6.250e-05, size: 736, ETA: 0:02:23 2021-08-15 22:00:19.952 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:00:25.995 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 22:00:26.010 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 22:00:26.018 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 22:00:26.018 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 22:00:26.019 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 22:00:26.058 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.28 ms, Average NMS time: 1.08 ms, Average inference time: 7.36 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 22:00:26.058 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:00:26.221 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch8 2021-08-15 22:00:26.221 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 22:00:26.222 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 22:00:31.045 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 10/88, mem: 3881Mb, iter_time: 0.482s, data_time: 0.183s, total_loss: 10.0, iou_loss: 3.8, l1_loss: 2.1, conf_loss: 3.2, cls_loss: 0.9, lr: 6.250e-05, size: 512, ETA: 0:02:13 2021-08-15 22:00:35.180 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 20/88, mem: 3881Mb, iter_time: 0.413s, data_time: 0.155s, total_loss: 5.6, iou_loss: 2.6, l1_loss: 1.0, conf_loss: 1.4, cls_loss: 0.7, lr: 6.250e-05, size: 640, ETA: 0:02:08 2021-08-15 22:00:39.925 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 30/88, mem: 3881Mb, iter_time: 0.474s, data_time: 0.191s, total_loss: 3.9, iou_loss: 1.8, l1_loss: 0.5, conf_loss: 1.0, cls_loss: 0.6, lr: 6.250e-05, size: 672, ETA: 0:02:02 2021-08-15 22:00:44.020 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 40/88, mem: 3881Mb, iter_time: 0.409s, data_time: 0.150s, total_loss: 6.4, iou_loss: 3.3, l1_loss: 1.2, conf_loss: 1.1, cls_loss: 0.8, lr: 6.250e-05, size: 512, ETA: 0:01:57 2021-08-15 22:00:48.270 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 50/88, mem: 3881Mb, iter_time: 0.424s, data_time: 0.167s, total_loss: 4.1, iou_loss: 2.0, l1_loss: 0.6, conf_loss: 1.0, cls_loss: 0.6, lr: 6.250e-05, size: 672, ETA: 0:01:51 2021-08-15 22:00:52.262 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 60/88, mem: 3881Mb, iter_time: 0.398s, data_time: 0.140s, total_loss: 8.9, iou_loss: 3.9, l1_loss: 1.8, conf_loss: 2.4, cls_loss: 0.9, lr: 6.250e-05, size: 448, ETA: 0:01:45 2021-08-15 22:00:57.304 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 70/88, mem: 3881Mb, iter_time: 0.504s, data_time: 0.216s, total_loss: 5.0, iou_loss: 2.3, l1_loss: 0.8, conf_loss: 1.3, cls_loss: 0.7, lr: 6.250e-05, size: 768, ETA: 0:01:40 2021-08-15 22:01:03.148 | INFO | yolox.core.trainer:after_iter:242 - epoch: 8/10, iter: 80/88, mem: 3881Mb, iter_time: 0.584s, data_time: 0.259s, total_loss: 2.8, iou_loss: 1.1, l1_loss: 0.3, conf_loss: 0.9, cls_loss: 0.5, lr: 6.250e-05, size: 736, ETA: 0:01:35 2021-08-15 22:01:07.368 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:01:13.748 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 22:01:13.761 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 22:01:13.771 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 22:01:13.771 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 22:01:13.772 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 22:01:13.809 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.27 ms, Average NMS time: 1.13 ms, Average inference time: 7.40 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 22:01:13.810 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:01:13.969 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch9 2021-08-15 22:01:13.969 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 22:01:13.969 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 22:01:19.724 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 10/88, mem: 3881Mb, iter_time: 0.575s, data_time: 0.261s, total_loss: 5.1, iou_loss: 2.2, l1_loss: 0.7, conf_loss: 1.5, cls_loss: 0.6, lr: 6.250e-05, size: 768, ETA: 0:01:26 2021-08-15 22:01:25.460 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 20/88, mem: 3881Mb, iter_time: 0.573s, data_time: 0.249s, total_loss: 5.0, iou_loss: 2.4, l1_loss: 0.8, conf_loss: 1.1, cls_loss: 0.7, lr: 6.250e-05, size: 640, ETA: 0:01:21 2021-08-15 22:01:30.151 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 30/88, mem: 3881Mb, iter_time: 0.469s, data_time: 0.185s, total_loss: 4.9, iou_loss: 2.4, l1_loss: 0.7, conf_loss: 1.1, cls_loss: 0.6, lr: 6.250e-05, size: 544, ETA: 0:01:16 2021-08-15 22:01:33.860 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 40/88, mem: 3881Mb, iter_time: 0.370s, data_time: 0.127s, total_loss: 5.9, iou_loss: 2.6, l1_loss: 0.7, conf_loss: 2.0, cls_loss: 0.6, lr: 6.250e-05, size: 480, ETA: 0:01:10 2021-08-15 22:01:38.131 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 50/88, mem: 3881Mb, iter_time: 0.427s, data_time: 0.167s, total_loss: 8.6, iou_loss: 3.4, l1_loss: 1.6, conf_loss: 2.9, cls_loss: 0.8, lr: 6.250e-05, size: 768, ETA: 0:01:05 2021-08-15 22:01:43.483 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 60/88, mem: 3881Mb, iter_time: 0.534s, data_time: 0.224s, total_loss: 4.4, iou_loss: 2.3, l1_loss: 0.5, conf_loss: 0.9, cls_loss: 0.6, lr: 6.250e-05, size: 576, ETA: 0:01:00 2021-08-15 22:01:48.007 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 70/88, mem: 3881Mb, iter_time: 0.452s, data_time: 0.178s, total_loss: 3.8, iou_loss: 1.8, l1_loss: 0.5, conf_loss: 1.0, cls_loss: 0.6, lr: 6.250e-05, size: 672, ETA: 0:00:54 2021-08-15 22:01:52.385 | INFO | yolox.core.trainer:after_iter:242 - epoch: 9/10, iter: 80/88, mem: 3881Mb, iter_time: 0.437s, data_time: 0.165s, total_loss: 8.9, iou_loss: 3.7, l1_loss: 1.6, conf_loss: 2.7, cls_loss: 1.0, lr: 6.250e-05, size: 448, ETA: 0:00:49 2021-08-15 22:01:55.374 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:02:01.640 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 22:02:01.653 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 22:02:01.660 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 22:02:01.661 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 22:02:01.661 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 22:02:01.701 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.29 ms, Average NMS time: 1.07 ms, Average inference time: 7.36 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 22:02:01.701 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:02:01.865 | INFO | yolox.core.trainer:before_epoch:192 - ---> start train epoch10 2021-08-15 22:02:01.866 | INFO | yolox.core.trainer:before_epoch:195 - --->No mosaic aug now! 2021-08-15 22:02:01.866 | INFO | yolox.core.trainer:before_epoch:197 - --->Add additional L1 loss now! 2021-08-15 22:02:06.901 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 10/88, mem: 3881Mb, iter_time: 0.503s, data_time: 0.206s, total_loss: 3.8, iou_loss: 1.9, l1_loss: 0.5, conf_loss: 0.8, cls_loss: 0.5, lr: 6.250e-05, size: 608, ETA: 0:00:40 2021-08-15 22:02:11.762 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 20/88, mem: 3881Mb, iter_time: 0.486s, data_time: 0.204s, total_loss: 6.7, iou_loss: 3.0, l1_loss: 1.2, conf_loss: 1.7, cls_loss: 0.8, lr: 6.250e-05, size: 768, ETA: 0:00:34 2021-08-15 22:02:17.996 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 30/88, mem: 3881Mb, iter_time: 0.623s, data_time: 0.285s, total_loss: 7.2, iou_loss: 3.4, l1_loss: 1.5, conf_loss: 1.5, cls_loss: 0.8, lr: 6.250e-05, size: 832, ETA: 0:00:29 2021-08-15 22:02:24.433 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 40/88, mem: 3881Mb, iter_time: 0.643s, data_time: 0.276s, total_loss: 6.7, iou_loss: 3.4, l1_loss: 1.4, conf_loss: 1.2, cls_loss: 0.7, lr: 6.250e-05, size: 576, ETA: 0:00:24 2021-08-15 22:02:28.987 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 50/88, mem: 3881Mb, iter_time: 0.455s, data_time: 0.186s, total_loss: 4.9, iou_loss: 2.3, l1_loss: 0.8, conf_loss: 1.1, cls_loss: 0.6, lr: 6.250e-05, size: 832, ETA: 0:00:19 2021-08-15 22:02:35.567 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 60/88, mem: 3881Mb, iter_time: 0.658s, data_time: 0.286s, total_loss: 4.9, iou_loss: 2.4, l1_loss: 0.8, conf_loss: 1.1, cls_loss: 0.6, lr: 6.250e-05, size: 640, ETA: 0:00:14 2021-08-15 22:02:40.259 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 70/88, mem: 3881Mb, iter_time: 0.469s, data_time: 0.189s, total_loss: 3.3, iou_loss: 1.5, l1_loss: 0.4, conf_loss: 0.9, cls_loss: 0.5, lr: 6.250e-05, size: 640, ETA: 0:00:09 2021-08-15 22:02:45.042 | INFO | yolox.core.trainer:after_iter:242 - epoch: 10/10, iter: 80/88, mem: 3881Mb, iter_time: 0.478s, data_time: 0.196s, total_loss: 3.7, iou_loss: 1.9, l1_loss: 0.6, conf_loss: 0.6, cls_loss: 0.5, lr: 6.250e-05, size: 704, ETA: 0:00:04 2021-08-15 22:02:49.305 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:02:55.436 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:171 - Evaluate in main process... 2021-08-15 22:02:55.449 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - Loading and preparing results... 2021-08-15 22:02:55.457 | INFO | yolox.evaluators.coco_evaluator:evaluate_prediction:204 - DONE (t=0.01s) 2021-08-15 22:02:55.457 | INFO | pycocotools.coco:loadRes:365 - creating index... 2021-08-15 22:02:55.458 | INFO | pycocotools.coco:loadRes:365 - index created! 2021-08-15 22:02:55.498 | INFO | yolox.core.trainer:evaluate_and_save_model:315 - Average forward time: 6.26 ms, Average NMS time: 1.10 ms, Average inference time: 7.36 ms Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 1 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 10 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = -1.000 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = -1.000

2021-08-15 22:02:55.499 | INFO | yolox.core.trainer:save_ckpt:324 - Save weights to ./YOLOX_outputs\606 2021-08-15 22:02:55.667 | INFO | yolox.core.trainer:after_train:187 - Training of experiment is done and the best AP is 0.00

As you can see I only train for 10 epochs just for testing but the mAP is 0.00 and I get the same result even when I train for 300 epochs,and this is my own Exp file: `#!/usr/bin/env python3

-- coding:utf-8 --

Copyright (c) Megvii, Inc. and its affiliates.

import os

from yolox.exp import Exp as MyExp

class Exp(MyExp): def init(self): super(Exp, self).init() self.depth = 0.33 self.width = 0.50 self.exp_name = os.path.split(os.path.realpath(file))[1].split(".")[0]

    # Define yourself dataset path
    self.data_dir = "datasets/coco128"
    self.train_ann = "instances_train2017.json"
    self.val_ann = "instances_val2017.json"

    self.num_classes = 8

    self.max_epoch = 10
    self.data_num_workers = 0
    self.eval_interval = 1`

and the coco_classes.py: `#!/usr/bin/env python3

-- coding:utf-8 --

Copyright (c) Megvii, Inc. and its affiliates.

COCO_CLASSES = ( "blue1", "red1", "blue2", "red2", "blue3", "red3", "blue4", "red4", )` So what's wrong with my code?I have struggled into this a whole day,Honestly,I'm about to be crazy now!!!

gtx1080运行训练demo提示错误如下 2021-07-27 13:21:04 | ERROR | yolox.models.yolo_head:287 - OOM RuntimeError is raised due to the huge memory cost during label assignment. CPU mode is applied in this batch. If you want to avoid this issue, try to reduce the batch size or image size. 2021-07-27 13:21:04 | INFO | yolox.core.trainer:183 - Training of experiment is done and the best AP is 0.00 2021-07-27 13:21:04 | ERROR | yolox.core.launch:68 - An error has been caught in function 'launch', process 'MainProcess' (8953), thread 'MainThread' (139988140537664): ... RuntimeError: CUDA error: device-side assert triggered

During training, RAM usage continues to grow. Finaily, the training process stopped. It is a bug?

2021-07-23 14:46:56 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2000/3905, mem: 1730Mb, iter_time: 0.165s, data_time: 0.001s, total_loss: 3.3, iou_loss: 1.7, l1_loss: 0.0, conf_loss: 1.2, cls_loss: 0.5, lr: 9.955e-03, size: 320, ETA: 16:27:19 2021-07-23 14:47:04 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2050/3905, mem: 1730Mb, iter_time: 0.166s, data_time: 0.001s, total_loss: 3.0, iou_loss: 1.8, l1_loss: 0.0, conf_loss: 0.7, cls_loss: 0.5, lr: 9.955e-03, size: 320, ETA: 16:27:11 2021-07-23 14:47:13 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2100/3905, mem: 1730Mb, iter_time: 0.165s, data_time: 0.001s, total_loss: 2.9, iou_loss: 1.8, l1_loss: 0.0, conf_loss: 0.6, cls_loss: 0.5, lr: 9.954e-03, size: 320, ETA: 16:27:02 2021-07-23 14:47:21 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2150/3905, mem: 1730Mb, iter_time: 0.166s, data_time: 0.001s, total_loss: 3.3, iou_loss: 1.9, l1_loss: 0.0, conf_loss: 1.0, cls_loss: 0.5, lr: 9.954e-03, size: 320, ETA: 16:26:54 2021-07-23 14:47:30 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2200/3905, mem: 1730Mb, iter_time: 0.177s, data_time: 0.002s, total_loss: 2.3, iou_loss: 1.3, l1_loss: 0.0, conf_loss: 0.6, cls_loss: 0.4, lr: 9.954e-03, size: 320, ETA: 16:26:51 2021-07-23 14:47:38 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2250/3905, mem: 1730Mb, iter_time: 0.166s, data_time: 0.001s, total_loss: 2.9, iou_loss: 1.7, l1_loss: 0.0, conf_loss: 0.7, cls_loss: 0.5, lr: 9.953e-03, size: 320, ETA: 16:26:43 2021-07-23 14:47:47 | INFO | yolox.core.trainer:237 - epoch: 9/100, iter: 2300/3905, mem: 1730Mb, iter_time: 0.168s, data_time: 0.001s, total_loss: 2.4, iou_loss: 1.5, l1_loss: 0.0, conf_loss: 0.5, cls_loss: 0.4, lr: 9.953e-03, size: 320, ETA: 16:26:36 ------------------------stopped here--------------------

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 440.33.01 Driver Version: 440.33.01 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce RTX 208... On | 00000000:3D:00.0 Off | N/A | | 28% 50C P2 109W / 250W | 2050MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 1 GeForce RTX 208... On | 00000000:3E:00.0 Off | N/A | | 27% 49C P2 103W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 2 GeForce RTX 208... On | 00000000:41:00.0 Off | N/A | | 25% 48C P2 117W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 3 GeForce RTX 208... On | 00000000:42:00.0 Off | N/A | | 28% 50C P2 113W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 4 GeForce RTX 208... On | 00000000:44:00.0 Off | N/A | | 16% 27C P8 21W / 250W | 11MiB / 11019MiB | 0% Default | +-------------------------------+----------------------+----------------------+ | 5 GeForce RTX 208... On | 00000000:45:00.0 Off | N/A | | 28% 50C P2 110W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 6 GeForce RTX 208... On | 00000000:46:00.0 Off | N/A | | 24% 47C P2 95W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+ | 7 GeForce RTX 208... On | 00000000:47:00.0 Off | N/A | | 26% 49C P2 99W / 250W | 2086MiB / 11019MiB | 100% Default | +-------------------------------+----------------------+----------------------+

是RAM持续增长，然后溢出，导致程序停止？

when I try to train,run this ''' python3 tools/train.py -f exps/example/yolox_voc/yolox_voc_s.py -d 0 -b 2 --fp16 -c yolox_s.pth ''' it was wrong when there occurs 2021-07-28 19:50:56 | INFO | yolox.core.trainer:245 - epoch: 1/300, iter: 240/405, mem: 2898Mb, iter_time: 0.104s, data_time: 0.000s, total_loss: 4.7, iou_loss: 1.6, l1_loss: 0.0, conf_loss: 2.3, cls_loss: 0.8, lr: 4.390e-06, size: 608, ETA: 7:17:45

here are the details: 2021-07-28 19:50:55 | INFO | yolox.core.trainer:245 - epoch: 1/300, iter: 230/405, mem: 2898Mb, iter_time: 0.104s, data_time: 0.000s, total_loss: 4.8, iou_loss: 2.0, l1_loss: 0.0, conf_loss: 1.5, cls_loss: 1.3, lr: 4.031e-06, size: 544, ETA: 7:27:43 2021-07-28 19:50:56 | INFO | yolox.core.trainer:245 - epoch: 1/300, iter: 240/405, mem: 2898Mb, iter_time: 0.104s, data_time: 0.000s, total_loss: 4.7, iou_loss: 1.6, l1_loss: 0.0, conf_loss: 2.3, cls_loss: 0.8, lr: 4.390e-06, size: 608, ETA: 7:17:45 2021-07-28 19:50:57 | INFO | yolox.core.trainer:184 - Training of experiment is done and the best AP is 0.00 2021-07-28 19:50:57 | ERROR | yolox.core.launch:68 - An error has been caught in function 'launch', process 'MainProcess' (3173), thread 'MainThread' (140583223760704): Traceback (most recent call last):

File "tools/train.py", line 115, in dist_url=dist_url, args=(exp, args) │ │ └ Namespace(batch_size=2, ckpt='yolox_s.pth', devices=0, dist_backend='nccl', dist_url=None, exp_file='exps/example/yolox_voc/y... │ └ ╒══════════════════╤═════════════════════════════════════════════════════════════════════════════════════════════════════════... └ 'auto'

File "/home/yan/YOLOX/yolox/core/launch.py", line 68, in launch main_func(*args) │ └ (╒══════════════════╤════════════════════════════════════════════════════════════════════════════════════════════════════════... └ <function main at 0x7fdb434ab598>

File "tools/train.py", line 101, in main trainer.train() │ └ <function Trainer.train at 0x7fdb55bb3400> └ <yolox.core.trainer.Trainer object at 0x7fdc11a8fda0>

File "/home/yan/YOLOX/yolox/core/trainer.py", line 70, in train self.train_in_epoch() │ └ <function Trainer.train_in_epoch at 0x7fdbaa11d620> └ <yolox.core.trainer.Trainer object at 0x7fdc11a8fda0>

File "/home/yan/YOLOX/yolox/core/trainer.py", line 79, in train_in_epoch self.train_in_iter() │ └ <function Trainer.train_in_iter at 0x7fdbaa1249d8> └ <yolox.core.trainer.Trainer object at 0x7fdc11a8fda0>

File "/home/yan/YOLOX/yolox/core/trainer.py", line 85, in train_in_iter self.train_one_iter() │ └ <function Trainer.train_one_iter at 0x7fdbaa124a60> └ <yolox.core.trainer.Trainer object at 0x7fdc11a8fda0>

File "/home/yan/YOLOX/yolox/core/trainer.py", line 91, in train_one_iter inps, targets = self.prefetcher.next() │ │ └ <function DataPrefetcher.next at 0x7fdb480e4b70> │ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x7fdb390250f0> └ <yolox.core.trainer.Trainer object at 0x7fdc11a8fda0>

File "/home/yan/YOLOX/yolox/data/data_prefetcher.py", line 48, in next self.preload() │ └ <function DataPrefetcher.preload at 0x7fdb480e46a8> └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x7fdb390250f0>

File "/home/yan/YOLOX/yolox/data/data_prefetcher.py", line 30, in preload self.next_input, self.next_target, _, _ = next(self.loader) │ │ │ │ │ └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7fdb55564a20> │ │ │ │ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x7fdb390250f0> │ │ │ └ tensor([[[0., 0., 0., 0., 0.], │ │ │ [0., 0., 0., 0., 0.], │ │ │ [0., 0., 0., 0., 0.], │ │ │ ..., │ │ │ [0., 0., ... │ │ └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x7fdb390250f0> │ └ tensor([[[[-0.1657, -0.1657, -0.1657, ..., -0.1657, -0.1657, -0.1657], │ [-0.1657, -0.1657, -0.1657, ..., -0.1657, ... └ <yolox.data.data_prefetcher.DataPrefetcher object at 0x7fdb390250f0>

File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 521, in next data = self._next_data() │ └ <function _MultiProcessingDataLoaderIter._next_data at 0x7fdb555a5268> └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7fdb55564a20> File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data return self._process_data(data) │ │ └ <torch._utils.ExceptionWrapper object at 0x7fdb3af557b8> │ └ <function _MultiProcessingDataLoaderIter._process_data at 0x7fdb555a5378> └ <torch.utils.data.dataloader._MultiProcessingDataLoaderIter object at 0x7fdb55564a20> File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data data.reraise() │ └ <function ExceptionWrapper.reraise at 0x7fdc10ccad90> └ <torch._utils.ExceptionWrapper object at 0x7fdb3af557b8> File "/home/yan/.local/lib/python3.6/site-packages/torch/_utils.py", line 425, in reraise raise self.exc_type(msg) │ │ └ Caught KeyError in DataLoader worker process 0. │ │ Original Traceback (most recent call last): │ │ File "/home/yan/.local/lib/pyth... │ └ <class 'KeyError'> └ <torch._utils.ExceptionWrapper object at 0x7fdb3af557b8>

KeyError: Caught KeyError in DataLoader worker process 0. Original Traceback (most recent call last): File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop data = fetcher.fetch(index) File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in fetch data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/yan/.local/lib/python3.6/site-packages/torch/utils/data/_utils/fetch.py", line 44, in data = [self.dataset[idx] for idx in possibly_batched_index] File "/home/yan/YOLOX/yolox/data/datasets/datasets_wrapper.py", line 121, in wrapper ret_val = getitem_fn(self, index) File "/home/yan/YOLOX/yolox/data/datasets/mosaicdetection.py", line 91, in getitem img, _labels, _, _ = self._dataset.pull_item(index) File "/home/yan/YOLOX/yolox/data/datasets/voc.py", line 145, in pull_item target = self.load_anno(index) File "/home/yan/YOLOX/yolox/data/datasets/voc.py", line 126, in load_anno target = self.target_transform(target) File "/home/yan/YOLOX/yolox/data/datasets/voc.py", line 64, in call label_idx = self.class_to_ind[name] KeyError: 'dw'

你好，我直接下载官方提供的yolox-nano模型跑demo程序，测试dog.jpg，发现输出结果稍微有点问题，多了一个目标。调用代码如下： python tools/demo.py image -n yolox-nano -c ./models/yolox_nano.pth.tar --path assets/dog.jpg --conf 0.3 --nms 0.65 --tsize 416 --save_result 显示结果如下图所示： 我打印输出outputs的结果如下： tensor([[ 71.5272, 116.8364, 172.1584, 295.3528, 0.9587, 0.8872, 16.0000], [ 60.9691, 75.9739, 307.4900, 231.2223, 0.9442, 0.8800, 1.0000], [252.5838, 41.6913, 375.4042, 91.2227, 0.7677, 0.8870, 2.0000], [252.0094, 41.5889, 375.6715, 93.2385, 0.4352, 0.8493, 7.0000]], device='cuda:0') 另外，比较奇怪的是，我把pytorch模型转换成ncnn之后检测结果正常。直接使用的是官方提供的yolox.cpp文件，参数设置和上面相同，检测结果如下图所示： 16 = 0.84702 at 132.07 215.84 185.92 x 329.64 1 = 0.82856 at 113.21 140.14 454.67 x 286.03 2 = 0.68373 at 466.35 77.03 226.64 x 91.41 非常感谢您的工程！

epochs set to 300: when training to about 200 epochs, the map starts to drop. epochs set to 200: when training to about 120 epochs, the map starts to drop. Any suggestions? thx!

I trained yolox_L model for 100 epochs, the detection for big objects is good but is bad for small objects. The confidence for small targets is too low that cannot show the bounding box with a normal conf and nms value. Could anyone provide idea on how to improve it? Thanks a lot

I have pulled the latest code. training command is: 'python3 tools/train.py -f exps/default/yolox_l.py -d 1 -b 8 --fp16 -o -c yolox/weights/yolox_l.pth.tar'

2021-08-05 19:03:18.410 | INFO | yolox.core.trainer:before_epoch:193 - ---> start train epoch300 2021-08-05 19:03:21.105 | INFO | yolox.core.trainer:after_iter:246 - epoch: 300/300, iter: 10/27, mem: 10161Mb, iter_time: 0.269s, data_time: 0.002s, total_loss: 5.5, iou_loss: 1.9, l1_loss: 0.8, conf_loss: 2.0, cls_loss: 0.8, lr: 6.250e-05, size: 672, ETA: 0:00:05 2021-08-05 19:03:24.223 | INFO | yolox.core.trainer:after_iter:246 - epoch: 300/300, iter: 20/27, mem: 10161Mb, iter_time: 0.311s, data_time: 0.001s, total_loss: 6.2, iou_loss: 2.3, l1_loss: 1.0, conf_loss: 2.2, cls_loss: 0.8, lr: 6.250e-05, size: 576, ETA: 0:00:02 2021-08-05 19:03:26.326 | INFO | yolox.core.trainer:save_ckpt:322 - Save weights to ./YOLOX_outputs/yolox_voc_s 2021-08-05 19:03:27.583 | INFO | yolox.evaluators.voc_evaluator:evaluate_prediction:161 - Evaluate in main process... 2021-08-05 19:03:28.087 | INFO | yolox.core.trainer:evaluate_and_save_model:313 - Average forward time: 10.16 ms, Average NMS time: 0.79 ms, Average inference time: 10.96 ms

2021-08-05 19:03:28.087 | INFO | yolox.core.trainer:save_ckpt:322 - Save weights to ./YOLOX_outputs/yolox_voc_s 2021-08-05 19:03:28.344 | INFO | yolox.core.trainer:after_train:186 - Training of experiment is done and the best AP is 0.00

WARNING: Discarding file:///D:/LH/code/YOLOX-main. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

image有保存的dog结果 但是video的结果为空白

~/dev/YOLOX$ python3 tools/demo.py video -n yolox-s -c pretrained_models/yolox_s.pth.tar --path /assets/ch14_0616-0625.mp4 --conf 0.3 --nms 0.65 --tsize 640 --save_result 2021-07-22 14:34:23 | INFO | main:219 - Args: Namespace(camid=0, ckpt='pretrained_models/yolox_s.pth.tar', conf=0.3, demo='video', exp_file=None, experiment_name='yolox_s', fp16=False, fuse=False, name='yolox-s', nms=0.65, path='/assets/ch14_0616-0625.mp4', save_result=True, trt=False, tsize=640) /usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at /pytorch/c10/core/TensorImpl.h:1156.) return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode) 2021-07-22 14:34:23 | INFO | main:229 - Model Summary: Params: 8.97M, Gflops: 26.81 2021-07-22 14:34:28 | INFO | main:240 - loading checkpoint 2021-07-22 14:34:29 | INFO | main:245 - loaded checkpoint done. 2021-07-22 14:34:29 | INFO | main:183 - video save_path is ./YOLOX_outputs/yolox_s/vis_res/2021_07_22_14_34_29/ch14_0616-0625.mp4 但是检查该目录下 并没有mp4文件。

Background images are images with no objects that are added to a dataset to reduce False Positives (FP). When training Yolox, which percentage of backgound in our dataset could improve performance.

To install yolox I did these two:

git clone [email protected]:Megvii-BaseDetection/YOLOX.git
cd YOLOX

Then pip3 install -v -e . is not working. Here is the log of this command.

Using pip 22.0.2 from /usr/lib/python3/dist-packages/pip (python 3.10)
Defaulting to user installation because normal site-packages is not writeable
Obtaining file:///mnt/Shared/azimjon/YOLOX
  Running command python setup.py egg_info
  running egg_info
  creating /tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info
  writing /tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/PKG-INFO
  writing dependency_links to /tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/dependency_links.txt
  writing requirements to /tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/requires.txt
  writing top-level names to /tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/top_level.txt
  writing manifest file '/tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/SOURCES.txt'
  /home/azimjon/.local/lib/python3.10/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
    warnings.warn(msg.format('we could not find ninja.'))
  reading manifest file '/tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching '*.cu' under directory 'yolox'
  warning: no files found matching '*.cuh' under directory 'yolox'
  warning: no files found matching '*.cc' under directory 'yolox'
  adding license file 'LICENSE'
  writing manifest file '/tmp/pip-pip-egg-info-ypvf739l/yolox.egg-info/SOURCES.txt'
  Preparing metadata (setup.py) ... done
Collecting loguru
  Using cached loguru-0.6.0-py3-none-any.whl (58 kB)
Collecting ninja
  Using cached ninja-1.11.1-py2.py3-none-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (145 kB)
Collecting numpy
  Using cached numpy-1.24.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
Collecting onnx-simplifier==0.4.1
  Using cached onnx-simplifier-0.4.1.tar.gz (19.8 MB)
  Running command python setup.py egg_info
  fatal: not a git repository (or any of the parent directories): .git
  fatal: not a git repository (or any of the parent directories): .git
  Traceback (most recent call last):
    File "<string>", line 2, in <module>
    File "<pip-setuptools-caller>", line 34, in <module>
    File "/tmp/pip-install-q2xkxl0n/onnx-simplifier_795fe5484b37426fb7feb48837b76bd9/setup.py", line 64, in <module>
      assert CMAKE, 'Could not find "cmake" executable!'
  AssertionError: Could not find "cmake" executable!
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> See above for output.
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  full command: /usr/bin/python3 -c '
  exec(compile('"'"''"'"''"'"'
  # This is <pip-setuptools-caller> -- a caller that pip uses to run setup.py
  #
  # - It imports setuptools before invoking setup.py, to enable projects that directly
  #   import from `distutils.core` to work with newer packaging standards.
  # - It provides a clear error message when setuptools is not installed.
  # - It sets `sys.argv[0]` to the underlying `setup.py`, when invoking `setup.py` so
  #   setuptools doesn'"'"'t think the script is `-c`. This avoids the following warning:
  #     manifest_maker: standard file '"'"'-c'"'"' not found".
  # - It generates a shim setup.py, for handling setup.cfg-only projects.
  import os, sys, tokenize
  
  try:
      import setuptools
  except ImportError as error:
      print(
          "ERROR: Can not execute `setup.py` since setuptools is not available in "
          "the build environment.",
          file=sys.stderr,
      )
      sys.exit(1)
  
  __file__ = %r
  sys.argv[0] = __file__
  
  if os.path.exists(__file__):
      filename = __file__
      with tokenize.open(__file__) as f:
          setup_py_code = f.read()
  else:
      filename = "<auto-generated setuptools caller>"
      setup_py_code = "from setuptools import setup; setup()"
  
  exec(compile(setup_py_code, filename, "exec"))
  '"'"''"'"''"'"' % ('"'"'/tmp/pip-install-q2xkxl0n/onnx-simplifier_795fe5484b37426fb7feb48837b76bd9/setup.py'"'"',), "<pip-setuptools-caller>", "exec"))' egg_info --egg-base /tmp/pip-pip-egg-info-nazgrqy3
  cwd: /tmp/pip-install-q2xkxl0n/onnx-simplifier_795fe5484b37426fb7feb48837b76bd9/
  Preparing metadata (setup.py) ... error
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

My PC is Window10. I entered python tools/train.py -f exps/my/custom_yolox_s.py -c yolox_s.pth --occupy --cache --devices 0 --batch-size 16 in anaconda prompt. I trained my custom dataset, and training worked well. However, when loading and preparing results during train interval, the error appeared...

I checked ninja installed in my env. Also, I even add the ninja path to system path

I saw other issues in YOLOX, so I checked from yolox.layers import FastCOCOEvalOp, but it had some problems.

Please help me... I read other questions, but there's no solved.

When I train my own dataset, I always report this result at the beginning. I don't know exactly what went wrong. I'm going crazy. Please help me think about it

2022-12-16 22:46:39 | INFO | yolox.core.trainer:203 - ---> start train epoch1 C:\cb\pytorch_1000000000000\work\aten\src\ATen\native\cuda\ScatterGatherKernel.cu:276: block: [0,0,0], thread: [0,0,0] Assertion idx_dim >= 0 && idx_dim < index_size && "index out of bounds" failed. 2022-12-16 22:46:42 | INFO | yolox.core.trainer:195 - Training of experiment is done and the best AP is 0.00 2022-12-16 22:46:42 | ERROR | yolox.core.launch:98 - An error has been caught in function 'launch', process 'MainProcess' (12104), thread 'MainThread' (15040): Traceback (most recent call last):

File "tools\train.py", line 133, in launch(

(x) H:\YOLOX-0.3.0>python tools/train.py -f exps/example/custom/yolox_s.py -d 1 -b 8