Parse string values for add_special_tokens in vLLM

[eldarkurtic]

When trying to evaluate reasoning models on e.g. AIME task, we would run evals like this:

MODEL=deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B
MODEL_ARGS="pretrained=$MODEL,dtype=bfloat16,max_model_length=32768,gpu_memory_utilization=0.8,generation_parameters={max_new_tokens:32768,temperature:0.6,top_p:0.95}"
OUTPUT_DIR=data/evals/$MODEL

# AIME 2024
TASK=aime24
lighteval vllm $MODEL_ARGS "custom|$TASK|0|0" \
    --custom-tasks src/open_r1/evaluate.py \
    --use-chat-template \
    --output-dir $OUTPUT_DIR

As part of MODEL_ARGS we would like to be able to specify value for add_special_tokens to control whether BOS token is added or not by tokenizer. At the moment this is not possible because the given value will be interpreted as string whereas the codebase relies on boolean. This PR enables this by parsing add_special_tokens like other vLLM params are parsed in the class VLLMModel.

[lewtun]

Thanks for the PR @eldarkurtic! This LGTM but I'll let @NathanHB comment on whether to merge or not

[lewtun]

For viz @NathanHB this is related to an issue in vLLM where double BOS tokens are added if one uses the generate() method with strings instead of token IDs.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[clefourrier]

LGTM, but I'm not fond of the nested logic there, I would actually expand it into a

var = False
if config_var is not None:
   if isinstance(config_var, bool)
   elif isinstance(config_var, str)

for legibility

Or even add an utils function to parse bool strings to bool (which I thought we had?)

[NathanHB]

Hey ! Thanks for the PR, there is a PR open to use Pydantic as model configs and will remove the need for converting the strings to bool, good to merge though !