Skip to content

Check input model and return if the model does not exist in modelPath #2314

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Closed
wants to merge 10 commits into from
Closed

Check input model and return if the model does not exist in modelPath #2314

Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
0e4602a
Return quickly if there is no model in bearer token, an inputModel is…
joseluisll May 13, 2024
1766dee
feat: auto select llama-cpp cpu variant (#2305)
sozercan May 13, 2024
2e38c4e
models(gallery): add aura-llama-Abliterated (#2309)
mudler May 13, 2024
2f01e6f
models(gallery): add Bunny-llama (#2311)
mudler May 13, 2024
78eba71
models(gallery): add lumimaidv2 (#2312)
mudler May 13, 2024
118ff66
models(gallery): add orthocopter (#2313)
mudler May 13, 2024
52bd3dc
feat(llama.cpp): add `flash_attention` and `no_kv_offloading` (#2310)
mudler May 13, 2024
fa02b56
better message
joseluisll May 13, 2024
9a5e1ed
Merge branch 'master' into 1076-CheckInputModel
joseluisll May 13, 2024
fe15986
Merge branch 'master' into 1076-CheckInputModel
joseluisll May 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions core/http/ctx/fiber.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,17 @@ func ModelFromContext(ctx *fiber.Ctx, loader *model.ModelLoader, modelInput stri
if bearerExists {
log.Debug().Msgf("Using model from bearer token: %s", bearer)
modelInput = bearer
//Takes precedence, then return inmediatly
return modelInput,nil
}

//if a model is specified in modelInput, and there is no bearertoken,but it doesn´t exists in the model path, return QUICKLY. https://github.com/mudler/LocalAI/issues/1076
modelExists := modelInput != "" && loader.ExistsInModelPath(modelInput)
if !modelExists{
log.Debug().Msgf("The specified model does not exist in the model PATH, returning error")
return "", fmt.Errorf("invalid model filename")
}

//A model is specified, or the first available model is picked
return modelInput, nil
}
Loading