Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Failure rendering a simple template #252

Open
pavel-esir opened this issue May 31, 2024 · 2 comments
Open

Failure rendering a simple template #252

pavel-esir opened this issue May 31, 2024 · 2 comments

Comments

@pavel-esir
Copy link

pavel-esir commented May 31, 2024

Hi Dear Jinja2cpp developers,

i notices something unexpected when i try to apply a chatbot template.

This "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}" fails with an error:

terminate called after throwing an instance of 'nonstd::expected_lite::bad_expected_access<jinja2::ErrorInfoTpl<char> >'
  what():  bad_expected_access

When i remove the last {% if %} then it's ok. Attached a minimal reproducer. Looks like a bug, but idk maybe my template is wrong. Could you please take a look?

#include <jinja2cpp/template.h>
#include <jinja2cpp/template_env.h>

using namespace std;
using namespace jinja2;

int main() {
    TemplateEnv env;
    env.GetSettings().lstripBlocks = true;
    env.GetSettings().trimBlocks = true;
    Template tpl(&env);
       
    string broken_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}{% if add_generation_prompt %}{{'<start_of_turn>model'}}{% endif %}";
    string working_tpl = "{% for message in messages %}{{ '<start_of_turn>' + message['role'] + message['content'] }}{% endfor %}";
      
    tpl.Load(broken_tpl);
    
    jinja2::ValuesMap params_1 {{"role", "system"}, {"content", "You are a friendly chatbot who always responds in the style of a pirate"}};
    jinja2::ValuesMap params_2 {{"role", "user"}, {"content", "1+1="}};
    jinja2::ValuesMap params = {
        {"messages", jinja2::ValuesList({params_1, params_2})},
        {"add_generation_prompt", true},
        {"bos_token", "<bos>"},
        {"eos_token", "<eos>"},
        {"unk_token", "<unk>"},
        {"pad_token", "<pad>"}
    };

    cout << tpl.RenderAsString(params).value() << endl;

    return 0;
}

broken_tpl is an abbreviated version, just to reproduce the failure. The full template which i would loke to render is from gemma chatbot:

{{ bos_token }}

{% if messages[0]['role'] == 'system' %}
  {{ raise_exception('System role not supported') }}
{% endif %}

{% for message in messages %}
  {% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}
    {{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}
  {% endif %}

  {% if message['role'] == 'assistant' %}
    {% set role = 'model' %}
  {% else %}
    {% set role = message['role'] %}
  {% endif %}

  {{ '<start_of_turn>' + role + '\n' + message['content'] | trim + '<end_of_turn>\n' }}
{% endfor %}

{% if add_generation_prompt %}
  {{ '<start_of_turn>model\n' }}
{% endif %}

@pavel-esir
Copy link
Author

i localized a bit more, when i replace {{ with { it works on a full template

@pavel-esir
Copy link
Author

I mean replacing {{'<start_of_turn>model'}} with {'<start_of_turn>model'} fixes the problem.

Looks like this bug can be easily fixed. Could you please take a look?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant