A llama.cpp fronted gui that runs on Windows? (Without extra steps)
I now have llama.cpp running pretty well for my needs, but the inability to quickly set/swap models and system prompts isn't ideal. Lm-studio let's you save system prompts and settings in a drop down and also per-model and thats great. But (so far) in all my…
I now have llama.cpp running pretty well for my needs, but the inability to quickly set/swap models and system prompts isn't ideal. Lm-studio let's you save system prompts and settings in a drop down and also per-model and thats great. But (so far) in all my testing theres some issue where using the same settings and same sized quants in lm-studio results in it going significantly slower (probably out of memory) and I can't find why. Will do a couple more tests but if it can't handle what llama.cpp by itself can do then its a fail. Llama-swap is fine for model swapping but no way to system prompt save or swap. That I know of. I currently have the different prompts and cli launch parameters as various .text files that I will copy and paste when needed. But this it 2026. Even using the cli shouldnt be necessary anymore, not sure why its the standard for cutting edge applications. I suspect its a dev thing. So are there any frontend options out there that will work? I considered open Web ui but from what i can tell its only available as a docker install. I can do it, but I just prefer not to (on windows, I have dozens of docker containers on my server).
Collected discussion
Llama-server (in router mode, with a models.ini file). It's built-in llama.cpp
Isnt that just the llama.cpp web ui? Thats what ive been using, but it doesnt allow for model swapping and it doesn't allow for saving or swapping system prompts...
Isnt that just the llama.cpp web ui? Thats what ive been using, but it doesnt allow for model swapping and it doesn't allow for saving or swapping system prompts... Though you seem to say it does allow for model swaps. I thought llama-swap was required for that. Ill look into the models.ini file
It allows model swapping if you load it in router mode, which is what's being done in the script I've provided (by loading llama-server WITH a models.ini). As for system prompts I don't swap them myself, but IIRC the docs should have some info about how to do that with jinja templates.
It does. Add to launch arguments: --models-max 1 --models-preset ./configs/models.ini Then in ./configs/models.ini: [qwen-3.6-27b] model = ./models/qwen3.6/Qwen3.6-27B-Q4_K_L.gguf mmproj = ./models/qwen3.6/Qwen3.6-27B-mmproj-bf16.gguf fit-ctx = 65536 ctx-size = 65536 predict = 49152 image-min-tokens = 1024 image-max-tokens = 1024 spec-type = draft-mtp spec-draft-n-max = 11 spec-draft-p-min = 0.1 temp = 0.6 top-k = 20 top-p = 0.95 min-p = 0.0 presence-penalty = 0.0 repeat-penalty = 1.0 chat-template-kwargs = "{\"enable_thinking\": true,\"preserve_thinking\": true}" ...and whatever other settings you want to set. Instead of the --no-* flags like --no-mmap, you would set mmap = off in the config. [*] fit = off Means fit is off for all models.
Thats a linux command.
No it isn’t? It comes with python. You should be able to do this on windows.
Have you tried llama-server? Just open http://127.0.0.1:8080/ when it's loaded .\llama-server.exe' --models-preset '.\models.ini' --tools 'all' -np 1 -fa on -lv 4 --no-mmap --models-max 1 --ui-mcp-proxy --host 0.0.0.0 A sample from my models.ini: [byteshape/Qwen3.6-35B-A3B-GGUF:Qwen3.6-35B-A3B-IQ4_XS-4.15bpw] hf = byteshape/Qwen3.6-35B-A3B-GGUF:Qwen3.6-35B-A3B-IQ4_XS-4.15bpw temp = 0.6 min-p = 0.05 top-p = 0.95 top-k = 20 repeat-penalty = 1.05 no-mmproj = on c = 128000 ctk = q8_0 ctv = q8_0 ngl = 41 n-cpu-moe = 29 b = 4096 ub = 4096 #fitt = 64 chat-template-kwargs = {"preserve_thinking":true} Bear in mind you'll need to adjust some settings depending on your setup
This comment was deleted.
llama-server --models-preset presets.ini --models-max 1 In the webui, it'll show the list of available models to choose from. If you have limited vram, with --models-max 1 it'll unload the previous model if you load a new one. It works with the api too. Check the documentation on how to write a proper presets.ini file.
You don’t need docker for openwebui. pip install open-webui open-webui serve Then hit localhost:8080 and it should just work. I don’t know if it can load manage models in the way you are describing, though.
Unsloth has a neat windows compatible stack for inference, training etc.
models.ini
Try https://github.com/alekk89/llama-cpp-windows-manager
If you are interested, checkout LlamaForge. Its still being built, so in early preview. But is already very powerful and useful. It's here: https://github.com/dadwritestech/LlamaForge https://dadwritestech.github.io/LlamaForge/whats-new.html Thanks!
I’ve also been looking for something like this a purpose built windows installer without a crap ton of external dependencies that runs an actual windows app. Not a browser local port and not in a terminal. I haven’t seen many options. Open to ideas as well.
Try turbollm https://github.com/mohitsoni48/TurboLLM
https://jan.ai is open source, runs on Llama.cpp, has good UI, model swapping and projects. You can create an assistant (prompt) and set it as default for the project and create new chat inside that project. model picker is in the top left with all para
Koboldcpp
If you don't want the web ui just use the Ollama app
You can prompt your model for the powershell command to read the llama-server command line. That's what lm studio is running behind the scenes. Unfortunately you don't have complete control of the command line in the settings. But you can give your model both commands, your hand tuned one and the lm studio one, and it may suggest changes to improve performance that you can implement in lm studio settings. Worked for me.