Something about stage2 model

[EeyoreLee]

all-MiniLM-L6-v2

基本10个epoch达到LB F2最优

Non-futher pretrain & fine-tuning

Epoch	CV	LB
8	\	0.52
10	0.544	0.532

futher pretrain & fine-tuning

Epoch	Threshold	CV	LB
10	0.12	0.552	0.533

将Stage1未召回的正样本加入训练(further pretrain)

Epoch	Threshold	CV	LB
8	0.12	0.547	0.533
9	0.1	0.546	\
10	0.12	0.547	\

将召回id加入文本(non-futher pretrain)

Epoch	Threshold	CV	LB
8	0.06	0.60	0.543

paraphrase-multilingual-mpnet-base-v2

Non-futher pretrain & fine-tuning

Epoch	Threshold	CV	LB
8	0.11	0.552	0.554

将召回id加入文本(non-futher pretrain)(lr_scheduler="cosin_and_restarts")

Epoch	Threshold	CV	LB
8	0.21	0.582	0.527