Voice conversation setup (smarphone to own server)?

So I have... Moderate machine for models (2x 7900 xtx), runs gemma 4 decently comfy via llama.cpp vulkan. Though I do not used it too much, I'd like to set it up so that via VPN I could voice chat to my ring from my phone (preferably from headset) to hopefully get some more use out of my HW. So VPN is ofc non issue, I had it set up, no advice needed. But not sure if there is something to do what I want on phone and what I should set up on my "server"? Recently looked a bit into localAI thingie/tool/app, as it seemed to sorta claim to have all the shit with easy setup. But somehow was not that impressed, honestly had some of "ollama's bad smell" type of thing for me. But maybe it's not that bad? In any case, does anyone have such (or close) setup? What you guys using?