How to deploy LoRA adapter on NIM based on checkpoint created during llm.finetune with NeMo Run?

[mikolajMacioszczyk]

I successfully followed the NeMo tutorial to fine-tune Llama 3 with LoRA using nemo:24.12. This process generated a checkpoint stored in the directory:
./results/nemo2_peft/checkpoints/nemo2_peft--reduced_train_loss=0.2321-epoch=0-consumed_samples=160.0-last.

Checkpoint Structure

The checkpoint directory contains the following structure:
├── weights
│ ├── __0_0.distcp
│ ├── common.pt
│ ├── __0_1.distcp
│ ├── .metadata
│ ├── metadata.json
│ └── adapter_metadata.json
├── context
│ ├── 3c51d8ae-8292-48ac-9bb3-d316c248b82f
│ ├── model.yaml
│ ├── io.json
│ └── nemo_tokenizer
│ ├── tokenizer_config.json
│ ├── special_tokens_map.json
│ └── tokenizer.json

NIM's Required Structure

Now, I want to deploy this LoRA adapter on NIM, but NIM requires a different file structure for adapters:
loras
├── llama3-8b-math
│ └── llama3_8b_math.nemo
├── llama3-8b-math-hf
│ ├── adapter_config.json
│ └── adapter_model.bin
├── llama3-8b-squad
│ └── squad.nemo
└── llama3-8b-squad-hf
├── adapter_config.json
└── adapter_model.safetensors

Problem

I am unable to find a way to convert my checkpoint into a llama3-8b-custom.nemo file that matches NIM's requirements.

Questions

1. How can I convert my existing LoRA checkpoint into a llama3-8b-custom.nemo file for NIM?
2. Is there an alternative approach to deploying this adapter on NIM?

Any guidance on this would be greatly appreciated!

[mikolajMacioszczyk]

I found a soultion how to convert it to hf format, which is also supported by NIM with llm.peft.export_lora:

def configure_adapter_checkpoint_hf_conversion():
    return run.Partial(
        llm.peft.export_lora,
        lora_checkpoint_path=peft_ckpt_path,
        output_path=adapter_hf_outpout_path
    )

convert_ckpt_hf = configure_adapter_checkpoint_hf_conversion()
# define your executor
local_executor = run.LocalExecutor()

run.run(convert_ckpt_hf, executor=local_executor)