Evaluation Metric on KITTI

[baibaibai-star]

Hello! Thanks for your work!
I trained your code for 60 epochs with batch size 2 on a single 2080Ti GPU. The val results are not very good. I don't know where the problem is. Could you please show me the evaluation 3D APs that you got from the three categories that you got? Or is it possible to publish the pretraining weights? Thank you very much!

[jialeli1]

Try to train the detection model for Car class only using tools/cfgs/kitti_models/MGAF-3DSSD/mgaf-3dssd.yaml. The Car-based model is usually better than the 3 classes-based model. The paper also reports the results of the Car-based model.

[jialeli1]

Also, we really recommend a larger batchsize for better results.

[baibaibai-star]

Try to train the detection model for Car class only using tools/cfgs/kitti_models/MGAF-3DSSD/mgaf-3dssd.yaml. The Car-based model is usually better than the 3 classes-based model. The paper also reports the results of the Car-based model.

[baibaibai-star]

Thanks for your reply! I regret that I can't set a larger batch size because of CUDA memory. Could you tell me the AP gap between val and test on three categories when using 3 classes-based model? Thank you!

[jialeli1]

Like most methods, we submitted the results of car model and ped&cyc model, respectively. So, we can't offer you the performance gap of 3-class model on val and testing set. You can refer to the ablation experiments on the car class about the gap.

The 3-class model and the car model are similar on car class (within 1 AP), but more difference on ped/cyc. If batchsize is limited to a single GPU, you can train it on multiple GPUs. A larger batchsize will have better results.

[baibaibai-star]

Thank you for being patient with me. I will continue to try. Since I only have one GPU, considering overfitting case during the train, I will try to train only 30 epochs with batch size 2 and LR max 0.003.