Eval bug: c4ai-command-r7b-12-2024 unable to use

[rhjdvsgsgks]

Name and Version

caf773f

Operating systems

Linux

GGML backends

Vulkan

Hardware

not relevant

Models

c4ai-command-r7b-12-2024 q8

Problem description & steps to reproduce

run it with llama server. got error

First Bad Commit

maybe 6171c9d

Relevant log output

terminate called after throwing an instance of 'std::runtime_error'
  what():  Expected block keyword at row 152, column 17:
        {%- else %}
            {%- break %}
                ^
        {%- endif %}

[dranger003]

Looks like you are using the jinja template option, which I don't believe supports the template for this model (at least not yet).

[rhjdvsgsgks]

Looks like you are using the jinja template option, which I don't believe supports the template for this model (at least not yet).

i didn't enable it explicitly. is it enabled by default?

[dranger003]

Can you try --chat-template command-r maybe? The help says --jinja should be disabled by default.

[arch-btw]

Having the same issue when I haven't enabled jinja. But like @dranger003 said --chat-template command-r works.

[rhjdvsgsgks]

Can you try --chat-template command-r maybe? The help says --jinja should be disabled by default.

yes. it solved the error

[ochafik]

Hey @rhjdvsgsgks, @dranger003, thanks for reporting / investigating this! Some quick thoughts:

• Even when --jinja isn't enabled, minja is used to parse the template. It made the code of Add Jinja template support #11016 simpler, but that's an unfortunate coupling. I'll look into options to address this.
• Minja doesn't support {% break %} yet, tracking in Support CohereForAI/c4ai-command-r7b-12-2024 ({% break %}...) google/minja#38