Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Fix fused models for tf >= 4.39 #418

Merged
merged 1 commit into from
Apr 6, 2024

Conversation

TechxGenus
Copy link
Contributor

Fix #407 and #417

The latest main branch of transformer has solved the problems of quantization and inference. This pr solves the error reporting using fused models.

@casper-hansen
Copy link
Owner

Is this needed with the 4.39.3 patch? And should we also apply the same properties to other fused models?

@TechxGenus
Copy link
Contributor Author

Yes. It requires the 4.39.3 patch.
Other fused models do not need to be modified. Currently, only Llama, Gemma and Cohere (not yet supported by AutoAWQ) models are affected.
I'm not sure whether skipping the 4.39.0 to 4.39.2 versions of transformers in the requirements, as other models like Mistral work well.

@casper-hansen casper-hansen mentioned this pull request Apr 6, 2024
13 tasks
@casper-hansen casper-hansen merged commit 5d7b050 into casper-hansen:main Apr 6, 2024
# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AttributeError: 'Catcher' object has no attribute 'self_attn'
2 participants