Number of SwAV prototypes

[siemdejong]

Research question
What is the optimal number of prototypes for the SwAV model?
What is the optimal number of hidden dimension parameters?
What is the optimal number of output dimension parameters?

Hypothesis
prototypes
The number of prototypes that works best for classifying medullo/pilo = 6.
Reason: SwAV was previously trained on imagenet with 3000 prototypes for 1000 classes. Scaling it down linerarly to 2 classes means 6 prototypes.

hidden
?

output
?

Method
Perform a search with the multivariate Tree-structured Parzen Estimator algorithm to find the lowest loss in the loss landscape spanned by the SwAV parameters.

Why is this experiment worthwhile?
Presumably, the higher the contrast between tile representations, the better stage 2 will classify tumours.

Note
Probably, the choice of hidden en output dimension influence the optimal number of prototypes.

Result comments on this should include results and a discussion.