Nexctloud generally hasn’t pushed too hard for it have they?

In your defense, I’ve thought the same joke every time I’ve seen it lol

Funny to see someone else with an active distaste for his videos. He sets off predatory alarm bells in my head and feels smarmy to me.

I just run the llama-swap docker container with a config file mounted, set to listen for config changes so I don’t have to restart it to add new models. I don’t have a guide besides the README for llama-swap.

Especially from a 7b model

run a local LLM like Claude!

Look inside

“Run ollama”

Ollama will almost always be slower than running vllm or llama.cpp, nobody should be suggesting it for anything agentic. On most consumer hardware, the availability of llama.cpp’s --cpu-moe flag alone is absurdly good and worth the effort to familiarize yourself with llamacpp instead of ollama.

I’ve used Xpra for similar

Most of them are written in heavy legalese to cover asses