What is the Current State of the Art way to Develop with AI in a Selfhosted VM ?
Hey there, Not a Software dev, just a Techintereste Network dude trying to build a Proper SotA AI dev Setup. Ive got a Proxmox Box with a Ryzen 5 2600, 64GB DDR4 and a GTX1080 sitting in it which i use to self host diverse Services (Immich, Jellyfin etc.) I…
Hey there, Not a Software dev, just a Techintereste Network dude trying to build a Proper SotA AI dev Setup. Ive got a Proxmox Box with a Ryzen 5 2600, 64GB DDR4 and a GTX1080 sitting in it which i use to self host diverse Services (Immich, Jellyfin etc.) I currently "Vibe Code" With Claude Code on my Main Gaming PC (I also use it for trying GenAI since it has a 4080 Super and 64 GB DDR5) and i want to change that so that i have a 24/7 up VM Enviorment where i can remote into and which is completle Seperated from my Daily Driver Workspace so the Agents can also Spin up Docker Containers with Databases etc. without interfering my day to day Workspace, I also currently have to leave my PC on when leaving the house so the DB of my Current Project stays up etc. and i wanna move all that to my "Cloud". My Question however is What is the Current Best way to do exactly that? I heard Ubunut was the best distro to go with for AI Development? Couple more things im trying to figure out while im at it. How much RAM and Cores would such a VM actually need, or does that depend way too much on what im running for anyone to give a real number? And Security wise, since the Agents will be spinning up Containers and touching Databases on their own, how do you Sandbox that properly so it cant reach the rest of your Homelab? Kinda paranoid about this after reading OpenAI had one of their own Models escape a Sandbox and get into Hugging Faces Production Servers last month, dont want something like that happening on my own Network. (but TBH i dont expect do be able to stop a Current Frontier Model wanting to escape if it wants xd) Also would it make more sense to stick with Claude Code for this or go with something like OpenCode instead since it can run other Models too, im currently looking into moving more towards Kimi K3 and GPT 5.6-Sol and would want a Setup where i can actually use Multiple Subscriptions (Claude, GPT, Kimi) on the same Project instead of committing to just one, basically running it like a Dirigent towards other Frontier Models, so for example Fable does the Planning, Kimi K3 does the actual Building and GPT does the Review, using each Model for what its actually good at instead of paying for three Subscriptions and only ever touching one of them. Not sure if OpenCode is actually built for wiring Subscriptions together like that or if people just say that and it falls apart once you try it for real, i keep seeing people mention it but not sure if its worth the switch for someone who just wants stuff to work. And is the GTX1080 8GB any usefull in my setup ?, like could it actually pull weight for local Models or Comfy UI stuff (something like a render queue or smthing ?, or is it better off doing something else on that Box entirely and i should just rent GPU compute when i need it. AH Also i heard a LOT of glazing towards Hermes would that be usefull in my use case ? Would rather get this right the first time instead of rebuilding it in a few months. Whats your Setup look like ?
Collected discussion
Just spin up opencode inside a docker container. That way you can lock down what access you give it. There's very little that has changed as far as VMs in the last 5-10 years. It's all either fully hypervisor "real" VMs or Docker/Kubernetes for paravirtualization.
That defenitly sounds like a fairly simple and save solution , but how does that actually work once the containerized agent needs to spin up more containers itself? My current project for example has Claude Code managing a stack with its own DB and web UI containers, so it's not just "the agent touches files," it needs to create and manage other containers as part of the workflow.
Docker supports a new sandbox mode (sbx) that allows it to use a micro vm inside the docker container which can also spawn other docker containers insid it. Feel free to check it out.
Just spin up opencode inside a d̶o̶c̶k̶e̶r̶ rootless-podman container. That way you can lock down what access you give it. ftfy, but yes
I just gave my docker container for opencode full system access (it's a sandbox host so nothing else on it other than development), so opencode can spin up and down docker containers as it needs, it can basically do anything it needs to get something done.
I’d make the VM the security boundary, not the container the agent happens to run in. Once you mount the host Docker socket so it can create other containers, that container can effectively control the host anyway. Give the agent a disposable VM where it can own Docker, but put that VM on its own network segment with no route to your hypervisor, NAS, management interfaces or the rest of the homelab. Keep the git remote and backups outside it, snapshot the VM, and only expose the services it needs. Then if it wrecks the environment, you restore the VM instead of trying to prove every tool call was safe. I’d get that boundary working with one coding agent first, then add the planner, worker and reviewer setup after the isolation is solid.
Hemm... Opencode comes with support / is optimized for these models: https://opencode.ai/workspace/wrk_01KZ10MKFXPR6FSFS30QDFMTM8/go That's because they offer those in their plan so you can bet they have templates for those.