• 11 Posts
  • 10 Comments
Joined 3 years ago
cake
Cake day: July 8th, 2023

help-circle


  • Unfortunately, yes. It’s one reason I’m trying to figure out a good mechanism to maybe do something like multiple ollama hosts. So like: you can specify what model to use specifically in an agent. But if an agent delegates to a sub-agent, it unloads that model and loads the new one. I’m trying to figure out if there’s a way to “alternate” between multiple hosts (say, ollama running locally and one running on your server), so that when a switch happens, it does it on the secondary host while also looking ahead to see what needs to be switched, if anything, on the primary host.

    It supports multiple Ollama hosts right now as-is so what I’ve honestly been doing for the time being is specify which model on which host each agent uses so there’s only loading of one model at the beginning of a session. Then there’s no unloading/loading/etc. The other thing I’ve been trying is to see how small I can get the models to be without losing performance. While the tricks implemented in Loki help dramatically, I know there’s still a lot more I can do to improve it further.








  • To answer your question, I built it for a few reasons:

    • I wanted to learn Rust, so I used this project to do that
    • I really love TUI’s and I pretty much live in my command line at work, and since I already automate everything I can to make my work life easier, I wanted to be able to do the same with my homelab
    • I think it looks cool
    • For fun. If no one else ever gets use out of it, that’s okay! I just really enjoyed building it and I’m excited to build out more of it.

    But also: Why not? ☺️

    So really, you would only use this if you like TUIs or want a command line tool for interacting with your Servarrs. If you have no use for it, that’s totally fine too!

    The one actual use I’ve found people say they use it for is people who don’t want to expose the web interface to interact with their Servarrs and only interact via SSH. Then they like to use this instead.