Differences in learning outcomes between a self-defined model and the TIMM-provided model, despite utilizing the same weights

[Hu-nie]

I am currently utilizing the vit_large_patch16_224.augreg_in21k pretrained model from the TIMM library.

While attempting to fine-tune the model with my custom dataset, I have encountered some difficulties.

Could you kindly provide guidance on why there might be differences in the learning outcomes between using the TIMM-pretrained model and a self-defined one?

For your reference, I have included the code I implemented below.

I would greatly appreciate your assistance in resolving this issue.

VIT.zip