Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

configuration of llms not consistent #33

Closed
behrica opened this issue Sep 9, 2023 · 3 comments
Closed

configuration of llms not consistent #33

behrica opened this issue Sep 9, 2023 · 3 comments

Comments

@behrica
Copy link
Contributor

behrica commented Sep 9, 2023

Between gen/complete-template:

(def synopsis
  (gen/complete-template
    synopsis-template
    {:title "Mr. X" :genre "crime"}
    {:play   :llm/openai
     :review :llm/cohere}))

and gen/complete

(def review (gen/complete play 
    {:title "Mr. X" :genre "crime"}
    {:synopsis {:llm-generate {:impl :openai :api-key "<my-key>"}}
     :review   {:llm-generate {:impl :openai :api-key "<my-key>"}}}))

Maybe we mix here the "new style" and the "old style".
This relates as well to #30

So maybe the code in 0.3.6 is still in a transition to the new style, as introduced by the change to "Config management for LLM components"

So I will wait for your feedback, before looking deeper into #30.
I think, that the code as in 0.3.6 cannot work for Azure with "gen/complete-template"

@behrica
Copy link
Contributor Author

behrica commented Sep 9, 2023

It took me a bit, but I finally realized that gen/complete-template has a subset of functionality then gen/complete, correct ?
So all I can do with gen/complete-template can be done by using gen/complete. If this is true, I would suggest to remove gen/complete-template .This should make teh code simpler.

@zmedelis
Copy link
Owner

Added a fix for the complete-template configuration. Both complete and complete-template accept the last param with LLM service and model options (see comment section in bosquet.llm.generator)

  (complete-template
   "You are a playwright. Given the play's title and its genre write a synopsis for that play.
     Title: {{title}}
     Genre: {{genre}}
     Playwright: This is a synopsis for the above play: {% gen var-name=text %}"
   {:title "Mr. X" :genre "crime"}
   {:text {:bosquet.llm/service [:llm/openai :provider/openai]
           :bosquet.llm/model-parameters {:temperature 0 :model "gpt-4"}}})

I will be doing an iteration or two on this parameterization approach and thinking about more or less freezing it for v0.4.0

If this is true, I would suggest to remove gen/complete-template .This should make teh code simpler.

See the comment I added in the complete-template. There might be a benefit in having a slightly simpler version of the complete function, but you are probably right, not worth added complexity. Will consider removing it.

In any case, those are big interface changes. Need way better documentation. Will start work on that instead of dumping everything into Readme and notebooks.

@zmedelis
Copy link
Owner

Not to hold the other issues, closing it. Addressed in v0.3.7 release.

# for free to join this conversation on GitHub. Already have an account? # to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants