Cannot use load_from_checkpoint in an offline environment

[zzaebok]

Motivation

Due to the policy of my company, I cannot access to an online server (e.g., huggingface).
So I tried to use load_from_checkpoint function after I manually downloaded wmt22-comet-da checkpoint.
However, it makes an SSL error "SSLError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded".

This error is raised because of missing local_files_only param to each model's from_pretrained method.

🚀 Feature

I think COMET needs to support local_files_only parameter to load_from_checkpoint function not to raise http connection error.

Below is my current solution to the problem.

• Add local_files_only param to load_from_checkpoint.
• Pass it to str2model value classes' load_from_checkpoint(https://github.com/Unbabel/COMET/blob/master/comet/models/__init__.py#L88)
  • it automatically passes the param to model instantiation through LightningModule's load_from_checkpoint using kwargs.
• Add local_files_only param to CometModel.__init__() (https://github.com/Unbabel/COMET/blob/master/comet/models/base.py#L94)
• Pass it to str2encoder value classes from_pretrained (https://github.com/Unbabel/COMET/blob/master/comet/models/base.py#L119)
• Add local_files_only param to each encoder classes' __init__ such as

self.model = XLMRobertaModel(
    XLMRobertaConfig.from_pretrained(pretrained_model, local_files_only=local_files_only),
    add_pooling_layer=False,
)

Can I make a PR?
Please let me know if there is better way.

[ricardorei]

yep, this seems good to me!

[zzaebok]

I made a PR for this issue, please check it :)

Additionally, I think it is also needed to add cache_dir param too. (If you thinks it's okay, I will make another PR for this)

[ricardorei]

Hi @zzaebok thanks for the PR!

I am on vocations and going to ACL next week so it might take me some time but I will look at it ASAP.

[vergilus]

Hi @zzaebok thanks for the PR!

I am on vocations and going to ACL next week so it might take me some time but I will look at it ASAP.

any updates?

[flashbear]

i need this update as well :)

[flashbear]

do i simply place the downloaded (eg. xlm-roberta-large) and place it on the parent folder?

[vergilus]

do i simply place the downloaded (eg. xlm-roberta-large) and place it on the parent folder?

I modified the local comet configs hparam.yaml, where there is a path for the infoxlm ckpt path (pretrained_model). Then switch on the reload_hparams during comet loading:
comet_scorer = comet.load_from_checkpoint(args.comet_ckpt, reload_hparams=True)

[flashbear]

@vergilus appreciate your feedback 👍 working now :)