Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

deepsparse.TextGeneration doesn't accept trust_remote_code as an arg anymore #1639

Open
mgoin opened this issue Mar 26, 2024 · 1 comment
Open
Assignees
Labels
bug Something isn't working

Comments

@mgoin
Copy link
Member

mgoin commented Mar 26, 2024

The legacy text-generation pipeline supported trust_remote_code as an argument, which was inherited from the Transformers base pipeline here:

trust_remote_code: bool = False,

This was useful for models with custom code related to their tokenizer: #1176

Testing code:

from deepsparse import TextGeneration
pipeline = TextGeneration(model="hf:mgoin/llama2.c-stories15M-quant-ds", trust_remote_code=True)
print(pipeline("hello").generations[0].text)

Output:

Fetching 7 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:00<00:00, 28042.15it/s]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 724, in text_generation_pipeline
    return Pipeline.create("text_generation", **kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 148, in create
    raise e
  File "/home/mgoin/code/deepsparse/src/deepsparse/pipeline.py", line 140, in create
    pipeline = Operator.create(task=task, **new_kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/operators/operator.py", line 137, in create
    return operator_constructor(**kwargs)
  File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/pipeline.py", line 175, in __init__
    single_engine_operator = NLEngineOperator(
  File "/home/mgoin/code/deepsparse/src/deepsparse/transformers/pipelines/text_generation/nl_engine_operator.py", line 132, in __init__
    super().__init__(**kwargs)
TypeError: EngineOperator.__init__() got an unexpected keyword argument 'trust_remote_code'
@mgoin mgoin added the bug Something isn't working label Mar 26, 2024
@mgoin
Copy link
Member Author

mgoin commented Apr 24, 2024

Hey @bfineran could we get this addressed?

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants