Skip to content
New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

#

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Jump to bottom

It would be cool if one could access self-hosted generative models (Llama, WizardLLM, etc.) #15

Open
darkhog opened this issue Apr 21, 2024 · 3 comments · May be fixed by #18
Open

It would be cool if one could access self-hosted generative models (Llama, WizardLLM, etc.) #15

darkhog opened this issue Apr 21, 2024 · 3 comments · May be fixed by #18

Comments

@darkhog
Copy link

darkhog commented Apr 21, 2024

Either over the network via Llamacpp/Kobold AI interfaces or running locally on your machine.

With the subscription prices being as they are, hosting your own model can be cheaper.
@Efoi
Copy link

Efoi commented Apr 22, 2024

I've forked it and added some of that functionality. Its not complete, so I'm not making a PR for it yet, and I'm not sure he wants the changes :P

@jjhaggar
Copy link

jjhaggar commented Jun 2, 2024

It'd be great if we could host and run our own models with Ollama (https://github.com/ollama/ollama).
They also host lots of models (https://ollama.com/library), it would be great if they would have a finetuned model for Godot 4.x code generation ^^

@minosvasilias I have just seen that you have several Godot code finetunes from about a year ago! :D Would you recommend them over newer models like LLaMA 3? Are you planning on finetuning any of the recently launched models? (and maybe adding it to the Ollama model library?) I'm somewhat baffled that the official Godot devs aren't supporting this initiative more actively ^^U By lowering the entrance barrier and enabling more people to code in Godot, we could all benefit!

@darkhog
Copy link
Author

darkhog commented Jun 2, 2024

I've forked it and added some of that functionality. Its not complete, so I'm not making a PR for it yet, and I'm not sure he wants the changes :P

That's amazing @Efoi! Will follow and star your fork - even if the PR won't be accepted here, I'll just use the fork.

ticklecatisback added a commit to ticklecatisback/godot-copilot that referenced this issue Oct 18, 2024
@ticklecatisback
Add support for Llama and WizardLLM models
9c07e8a 
Fixes minosvasilias#15

Add support for self-hosted generative models Llama and WizardLLM.

* **Add Llama model support:**
  - Create `addons/copilot/Llama.gd` script.
  - Implement `_get_models`, `_set_model`, `_send_user_prompt` functions.
  - Define constants and helper functions for Llama model.

* **Add WizardLLM model support:**
  - Create `addons/copilot/WizardLLM.gd` script.
  - Implement `_get_models`, `_set_model`, `_send_user_prompt` functions.
  - Define constants and helper functions for WizardLLM model.

* **Update Copilot.gd:**
  - Modify `populate_models` function to include Llama and WizardLLM models.
  - Add Llama and WizardLLM to the `models` dictionary.

* **Update CopilotUI.tscn:**
  - Add Llama and WizardLLM nodes to the `LLMs` node.
  - Set the script for Llama and WizardLLM nodes to the respective new scripts.
  - Add signal connections for Llama and WizardLLM nodes.

---

For more details, open the [Copilot Workspace session](https://copilot-workspace.githubnext.com/minosvasilias/godot-copilot/issues/15?shareId=XXXX-XXXX-XXXX-XXXX).
@ticklecatisback ticklecatisback linked a pull request Oct 18, 2024 that will close this issue
Open
# for free to join this conversation on GitHub. Already have an account? # to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add support for Llama and WizardLLM models ticklecatisback/godot-copilot
3 participants
@darkhog @jjhaggar @Efoi