Check input model and return if the model does not exist in modelPath #2314
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for localai canceled.
|
… specified but it doen´t exist in modelPath Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
* auto select cpu variant Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * remove cuda target for now Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * fix metal Signed-off-by: Sertac Ozercan <sozercan@gmail.com> * fix path Signed-off-by: Sertac Ozercan <sozercan@gmail.com> --------- Signed-off-by: Sertac Ozercan <sozercan@gmail.com> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
feat(llama.cpp): add flash_attn and no_kv_offload Signed-off-by: Ettore Di Giacinto <mudler@localai.io> Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
Modified the message to report that the specified model doesn´t exist in the model Path. Signed-off-by: Jose Luis López López <joseluisll@gmail.com>
|
Models are asynchronously loaded, and I don´t see a direct access to the configuration structures that define the models and the backends from the http fiber App. This PR will be closed and i need to think on a more elaborated solution. ¿May be inject a service into the App fiber from the go APP in orden to get the list of model names, model file names and check if they are loaded or not? |
# for free
to join this conversation on GitHub.
Already have an account?
# to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR addresses the issue #1076 . This PR checks if the model specified in the request exists in the model Path. If it doesn´t exist, then the request returns an error, the same way that it is done for the model specified in the bearer token context when the model doesn´t exist.
I chose to modify the fiber.go in the http context as it is common for all endpoints.
I have not introduce any change in backend to model mapping, or backend behaviour, and simply adds a check reusing and existing function.
Notes for Reviewers
Signed commits