-
-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? # to your account
feat(transformer): transformer-based backend #1015
Labels
Comments
mudler
added a commit
that referenced
this issue
Sep 9, 2023
Related to: #1015 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler
added a commit
that referenced
this issue
Sep 9, 2023
Related to: #1015 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
mudler
added a commit
that referenced
this issue
Sep 10, 2023
Is Vllm fully integrated? vLLM begins to seem to be many AWQ quantized model's first choice recently |
yep, going into the next release soon! |
mudler
added a commit
that referenced
this issue
Sep 22, 2023
This particularly useful to set AWQ **Description** Follow up of #1015 **Notes for Reviewers** **[Signed commits](../CONTRIBUTING.md#signing-off-on-commits-developer-certificate-of-origin)** - [ ] Yes, I signed my commits. <!-- Thank you for contributing to LocalAI! Contributing Conventions: 1. Include descriptive PR titles with [<component-name>] prepended. 2. Build and test your changes before submitting a PR. 3. Sign your commits By following the community's contribution conventions upfront, the review process will be accelerated and your PR merged more quickly. --> --------- Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
1 task
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Is your feature request related to a problem? Please describe.
It makes sense to have a backend which is tied to https://github.com/huggingface/transformers. This would increase the range of support for LLM that LocalAI already offers
Describe the solution you'd like
Ideally would be preferred to use a wrapper around transformers and xformers to avoid duplicating efforts.
Describe alternatives you've considered
Additional context
The text was updated successfully, but these errors were encountered: