Opening
There is a familiar 2am moment for anyone running a homelab. Plex stops streaming, the Home Assistant dashboard shows a red banner, and the fix is SSH-ing into a Proxmox node in the dark to grep journalctl by feel. An XDA writer recently handed that whole loop to a local LLM and reported that a Qwen-class model with tool calling could restart the right container, patch a bad automation, and report back before the coffee finished brewing. The pattern has a name now: an AI-managed homelab, where a local model reads system state and runs the shell and API commands you would run by hand. We tested eight of the best apps for running an AI-managed homelab on Linux, Windows, and macOS.
What to look for
A useful AI-managed homelab stack needs five things. First, local inference. The model should run on your GPU or an on-device runtime, not a remote API you cannot audit. Second, real tool calling. The model must produce structured JSON tool calls and the app must dispatch them to shell commands, HTTP endpoints, or a Model Context Protocol server. Third, sandboxed execution. Every irreversible action, a container restart, a firewall rule change, a delete, should surface an approval prompt before it runs. Fourth, self-hostable. If the control plane phones home, half the point of running a homelab is gone. Fifth, community plugins. Proxmox, Docker, and Home Assistant already have MCP servers or REST wrappers; the app that plugs into them wins the weekend.
Quick comparison
| App | Best for | Platforms | License | Tool calling |
|---|---|---|---|---|
| Ollama | Local model host with native tool calls | Linux, Windows, macOS | MIT | Yes |
| LM Studio | GUI model host with MCP servers | Linux, Windows, macOS | Free, closed source | Yes (MCP) |
| Home Assistant | LLM-driven house automation | Linux, Windows, macOS, HAOS | Apache 2.0 | Via Assist |
| Continue | IDE agent for editing infra-as-code | VS Code, JetBrains | Apache 2.0 | Yes |
| Cline | VS Code agent with terminal access | Linux, Windows, macOS | Apache 2.0 | Yes (MCP) |
| Aider | Git-first terminal pair programmer | Linux, Windows, macOS | Apache 2.0 | Yes |
| OpenHands | Autonomous end-to-end agent | Linux, Windows, macOS | MIT | Yes |
| LocalAI | OpenAI-compatible server for the rack | Linux, Windows, macOS | MIT | Yes |
1. Ollama for hosting local models with tool calling
Ollama runs local LLMs behind an OpenAI-compatible HTTP endpoint and has shipped native tool calling since v0.3. Point it at Qwen3 8B or a recent Gemma release and the model returns structured tool calls that a homelab agent can dispatch to Proxmox, Docker, or a Home Assistant REST call. The June 2026 v0.30.8 release added an upgraded Apple Silicon MLX engine and broader GGUF hardware support, so the same runtime handles a Mac mini and a spare rack GPU.
Where it falls short: Ollama has no built-in agent loop or approval UI. It hosts the model and returns tool calls; you still need something on top that executes them and gates the destructive ones.
Pricing: Free.
Platforms: Linux, Windows, macOS.
Download: Ollama
Bottom line: The default model host for anyone running an AI-managed homelab. Every other app on this list can point at an Ollama endpoint.
2. LM Studio for a GUI model host with MCP servers
LM Studio pairs a polished local model UI with Model Context Protocol support, which shipped in v0.3.17 and matured through 2026. You install an MCP server for the tool you care about, Docker, Home Assistant, the filesystem, and the model can call it from a chat session with a confirmation dialog on every tool call. The Agent Orchestrator beta lets multiple specialized agents coordinate on one task, useful when the fix spans a container restart and a Home Assistant automation edit.
Where it falls short: The app itself is closed source. That matters more in a homelab context than for a chat client, since the tool now touches your infrastructure.
Pricing: Free for personal and internal work use.
Platforms: Linux, Windows, macOS.
Download: LM Studio
Bottom line: The fastest way to get an approval-gated agent talking to your Docker daemon without writing a single wrapper.
3. Home Assistant for LLM-driven house automation
Home Assistant added a native Ollama conversation agent in the 2025.9 release, and by mid-2026 the pattern is a set piece. Run Ollama with a tool-capable model, register it under Settings, and wire it into an Assist pipeline. The model then maps natural language to real service calls, restart a media server, arm the alarm, change scenes, without touching a cloud. Community integrations extend Assist to Proxmox and Docker monitoring so the same voice command can reach past the smart plugs.
Where it falls short: Only tool-capable models work with Assist. Older 7B models without native tool calling will happily describe the action and never call it, which reads as a broken assistant.
Pricing: Free.
Platforms: Linux, Windows, macOS (also Home Assistant OS on Raspberry Pi and other single-board hardware).
Download: Home Assistant
Bottom line: The homelab component that most directly benefits from a local agent, and the one where a bad tool call is the least catastrophic.
4. Continue for an IDE agent that edits infra-as-code
Continue runs as a VS Code and JetBrains extension and turns the editor into an agent that reads and edits your Ansible playbooks, docker-compose files, and Terraform. It supports both cloud APIs and a local Ollama endpoint, so a homelab agent that touches nothing outside your network is one config change. The rules system scopes coding standards per file, which helps when the same repo holds production infra and throwaway experiments.
Where it falls short: Continue was acquired by Cursor and the team now points solo users to a new CLI. The JetBrains plugin has moved to community maintenance, so pace of updates is quieter than the VS Code side.
Pricing: Free, open source. Team and Company plans add shared private agents, SSO, and BYOK.
Platforms: Linux, Windows, macOS (VS Code and JetBrains).
Download: Continue
Bottom line: The right pick when the homelab work looks less like restarting a container and more like editing the compose file that defines it.
5. Cline for a VS Code agent with terminal access
Cline is an autonomous agent that runs as a VS Code sidebar (and now in JetBrains, Cursor, Zed, and a preview CLI) with Plan and Act modes that separate design from execution. Every file edit and terminal command surfaces an approval prompt unless you flip on auto-approve. Cline was one of the first coding tools to support MCP natively, so an MCP server for Proxmox or Docker plugs straight in and the model can call it during a chat.
Where it falls short: Cline is model-agnostic and bills through your provider keys, so a heavy homelab session on Claude 4.7 can add up. Point it at a local Ollama endpoint to keep costs at zero.
Pricing: Free extension; you bring your own API key or point at a local model. Teams pay $20/month per user after Q1 2026.
Platforms: Linux, Windows, macOS.
Download: Cline
Bottom line: The best pick when the homelab work happens inside the editor and the agent needs a terminal to test its own changes.
6. Aider for a git-first terminal pair programmer
Aider runs in the terminal, so it sits next to the tmux session where you already SSH into a Proxmox node. Add a file to the chat, describe the change, and Aider edits it, runs the linter and tests, and commits the result to git with a message that describes the change. Watch mode lets you scatter comments in files and have Aider react without leaving the editor, which fits the way most homelab tinkerers actually work.
Where it falls short: Aider is code-first. It edits files well and runs shell commands on request, but the agent loop is lighter than what Cline or OpenHands ship, so autonomous multi-step work needs more hand-holding.
Pricing: Free, open source. You bring your own API key; a local Ollama endpoint drops the running cost to nothing.
Platforms: Linux, Windows, macOS.
Download: Aider
Bottom line: The pick when the AI-managed homelab work happens in a terminal and every change should end as a git commit.
7. OpenHands for autonomous end-to-end tasks
OpenHands is the most-watched open-source coding agent on GitHub, and v1.6 landed in March 2026 with Kubernetes support and a Planning Mode beta. The default surface is now Agent Canvas, a browser UI that runs on your desktop and hands the agent a sandboxed environment to write code, run tests, debug failures, and open a PR without step-by-step approvals. The 1.6 release also added ACP compatibility, so you can plug in Claude Code, Codex, Gemini, or a local model as the reasoning engine.
Where it falls short: OpenHands is stronger on software tasks than on ops. Restarting a container works; a full weekend of Proxmox rebalancing needs guardrails and a checklist you write yourself.
Pricing: Free to self-host. OpenHands Cloud has a free tier using the MiniMax model.
Platforms: Linux, Windows, macOS.
Download: OpenHands
Bottom line: The right pick when the task is “fix this and open a PR” and you want the agent to work while you sleep.
8. LocalAI for an OpenAI-compatible server that scales the rack
LocalAI presents an HTTP API that mirrors OpenAI’s chat, embeddings, image, and audio endpoints, then runs the model on whatever hardware you have. The March 2026 v4.0 release added native agentic orchestration through the Agenthub community hub, and v4.3 turned on the llama.cpp prompt cache by default so repeated system prompts fall from minutes to seconds. Distributed mode routes requests across replicas with VRAM-aware autoscaling, which matters the day you add a second GPU to the rack.
Where it falls short: LocalAI’s surface area is now large enough that a fresh install rewards a weekend of reading before you point production traffic at it.
Pricing: Free, open source.
Platforms: Linux, Windows, macOS.
Download: LocalAI
Bottom line: The pick when the homelab has grown past a single GPU and you need one endpoint that every agent, IDE, and Home Assistant install can share.
How to pick the right one
If we were starting an AI-managed homelab from scratch tomorrow, we would install Ollama first, wire it into Home Assistant, and use LM Studio on a laptop as the day-to-day chat surface with the household MCP servers loaded. That covers the “listen to me and run the house” part cleanly.
If the homelab is really an ops sandbox, put Cline in the editor with a local Ollama endpoint and let Plan mode design a change before Act mode touches anything. Aider is the pick when the workflow is git-first and every change should end as a commit that you can revert with a single command. Continue is the right fit when the work looks like editing docker-compose and Ansible files inside the IDE you already own.
If we wanted a single agent to fix an issue while we slept, OpenHands in Agent Canvas mode is the one to reach for, ideally against a repo where the tests actually run. And once the second GPU shows up, LocalAI becomes the shared endpoint that keeps every app on the same model routing plane.
FAQ
What is an AI-managed homelab?
An AI-managed homelab is a self-hosted setup where a local or self-hosted LLM reads system state and runs shell or API commands to manage infrastructure. Instead of chat-only assistants, the model uses tool calling to restart containers, adjust Home Assistant automations, check backups, and report status on your behalf.
Can Ollama run agents that control Proxmox or Docker?
Ollama hosts the model and returns tool calls, but it does not execute them. Pair it with a client that dispatches those calls, LM Studio with an MCP server, Cline, OpenHands, or a small Python glue script, and the same model can call the Proxmox API or the Docker socket end to end.
Do you need a GPU for a local LLM in a homelab?
A tool-capable 7B to 8B model at 4-bit quantization runs comfortably on 8 GB of VRAM, which lines up with a used consumer GPU or an Apple Silicon Mac. CPU-only inference works but is too slow for voice control or a snappy agent loop.
Is it safe to give an LLM shell access to a Proxmox node?
Only with an approval step in front of every irreversible action. LM Studio, Cline, and OpenHands all support a human-in-the-loop mode where the model proposes a command and you confirm before it runs. Auto-approve exists in most of these tools; we would not turn it on for anything that touches production data or a backup target.
What is the difference between Cline and Aider for homelab work?
Cline is a VS Code sidebar with a strong plan-and-act loop and native MCP support, so it fits editor-heavy workflows and can drive an MCP server directly. Aider is a terminal tool that treats every change as a git commit, which fits the SSH-and-tmux workflow most homelab operators already use.
Which local model handles tool calling best in 2026?
Community benchmarks point at Qwen3 8B as the most consistent tool caller among models you can run on a homelab GPU, followed by recent Gemma releases with native tool support. Older models without native tool training will describe the action instead of calling it, which fails the whole agent loop.
