Tools, tools & more tools!

[ggozad]

Hey there,
I have recently released the 0.6.* series of oterm and now we support Tools!
I am generally happy with what oterm does already and I think tools is the way forward to extend what is possible to do with it.
Here are some thoughts and I would really appreciate feedback 👀

I think oterm should remain minimal and usable with a small feature-set. Tools could be in the form of one or more additional packages that complement it. Thus I am thinking of adding for example an optional dependency to oterm something in the lines of otermtools. Thus oterm can be kept slim without extra dependencies, complex setups etc. which was the original intention for it.

Here are some tool ideas that I have been working on, would love to hear your ideas as well:

1. Web access. This is pretty trivial to do, I have a working tool for it.
2. RAG. This is a bit more complex as we will need some vector db, chunking, retrieval, perhaps some small LLM use too. The potential is endless though as you can index your documents locally and query them easily.
3. Personal assistant tools. Think email, calendar etc.
4. Image generation capabilities probably through Stable Diffusion.

Keep in mind that I would like to keep oterm cross-platform compatible, which complicates some of the above.

Another thing I've been wondering about is how to go to about allowing you to integrate your own tools. This is possible now of course already, you can monkey-patch the available tools in oterm and provide your own. I think this might be daunting for the average developer though and looking into alternative "plugin" solutions. Again happy to have feedback on this point about what would be convenient for you!

Thanks for using oterm, I have had a lot of fun building this with your help and looking forward to seeing what the future will bring ❤️

[draeician]

Love this, ollama run is great, but it has badly needed some features. Thanks for this!!

[garland3]

Very cool. It would be nice to have some config file where more tools are listed that are accessible. https://ollama.com/blog/functions-as-tools The newest python ollama makes it super easy easy to add python functions as tools. Like if I had some code that is

def do_my_cool_job_1(args, kwargs):
    print("I'm doing cool work")

def do_my_cool_job_2(num1, num2):
    print("I'm doing cool work but v2")
    print(f"{num1} + {num2} = { num1+num2}")


tools = [do_my_cool_job_1,do_my_cool_job_2]

Then in oterm, you could point it at the file with the tools var to get more tools.

If you point me in the right direction, I could take a crack at adding this feature.

[ggozad]

I have started oterm.tools.rag.
I think this is ambitious but progresses well. The idea is to build a full RAG service that works with local documents and make it usable as a tool to Ollama.

Would be happy to receive input or PRs from people interested in this, I know you are out there :)

[ggozad]

Hey @garland3
It is not very difficult to come up with something. If you have the time to invest, I am happy to help out. I am always struggling to keep a balance between my paying job and open-source so not able to build stuff as fast as I would have liked.

To the point, in principle one could add some path to python scripts that contain tools in the same format that oterm uses. Then oterm could inspect and import the scripts making them available.
The complexity with the above is that users might be running oterm as an "app", not as a python package that is available in their virtual env. For example MacOS users who install with brew, or others who installed oterm with a package manager will be running oterm in a "sandboxed" virtual envioronment that could make it difficult to install packages in. Simple functions/tools though that have no dependencies should be easy.

[ggozad]

Here we go! #151 does the job :)