feat(local-llm): B0.5 — llama-swap + lokale Modellauswahl in Diagnostic

Mehrere lokale Modelle, on-demand geladen/geswappt, in Diagnostic waehlbar.
Design: das Brain schickt den Modellnamen (aus local_llm.json) im llm_request
mit -> Adapter -> llama-swap laedt/swappt. Keine separate Gamebox-Config noetig.

- xtts: `llama`-Container -> `llama-swap` (unified-cuda), config.yaml mit
  qwen3-8b (Standard) + qwen3-4b; Auto-Download via -hf, Cache /models geteilt
  (qwen3-8b schon da). Adapter -> llama-swap:8080, Timeout 600s (Erst-Download).
- adapter: `model` aus dem Request an llama-swap durchreichen (Fallback env).
- brain: router.load_config liest localLlmModel; local_llm_chat(model=...);
  agent gibt cfg-Modell mit; bridge reicht model durch (_local_llm + Route).
- diagnostic: /api/local-models-list (aus /shared/config/local_models.json,
  seeded), local-llm-config um localLlmModel erweitert; Dropdown "Lokales
  Modell" im Settings-Block + Erst-Download-Hinweis.

BLIND gebaut (Gamebox nicht testbar hier): llama-swap CLI/Config-Pfad beim
ersten Start via `docker logs aria-llama-swap` pruefen. Live-Lade-Status
(Adapter->Diagnostic) ist B0.5-2 (Folgeschritt).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-11 14:11:59 +02:00
co-authored by Claude Opus 4.8
parent 3ddcf665f0
commit 8ae20a9bd8
9 changed files with 174 additions and 46 deletions
+3 -1
View File
@@ -1171,6 +1171,7 @@ class Agent:
if not router_mod.should_try_local(user_message, cfg):
return None
local_only = bool(cfg.get("localOnly"))
local_model = cfg.get("localLlmModel") or "qwen3-8b" # B0.5: llama-swap-Key
tools = self._build_local_tools() # B1b: kuratierte Tools
sys_prompt = router_mod.build_local_system_prompt(IDENTITY_ANCHOR,
@@ -1185,7 +1186,8 @@ class Agent:
# Spotify aufrufen. Ergebnisse zurueck, bis es final (ohne tool_calls) antwortet.
final = ""
for _ in range(self._LOCAL_TOOL_ITERATIONS):
res = local_llm_chat(messages, max_tokens=500, temperature=0.5, tools=tools)
res = local_llm_chat(messages, max_tokens=500, temperature=0.5,
tools=tools, model=local_model)
if not res.get("ok"):
logger.info("[router] lokal fehlgeschlagen (%s) — %s", res.get("error"),
"kein Fallback (localOnly)" if local_only else "→ Claude")
+7 -4
View File
@@ -29,11 +29,12 @@ LOCAL_LLM_HTTP_TIMEOUT_SEC = float(os.environ.get("LOCAL_LLM_HTTP_TIMEOUT_SEC",
def local_llm_chat(messages: list, *, max_tokens: int = 512,
temperature: float = 0.7, stop=None, tools=None) -> dict:
temperature: float = 0.7, stop=None, tools=None,
model=None) -> dict:
"""Ein Chat-Call ans lokale LLM. messages = [{role, content}, ...].
tools (B1b): optionale OpenAI-Tool-Defs; das Ergebnis kann dann
result['tool_calls'] enthalten. Blockierend (urllib) — chat() laeuft
ohnehin im Executor-Thread."""
model (B0.5): welches Modell llama-swap laden soll. tools (B1b): optionale
OpenAI-Tool-Defs; das Ergebnis kann dann result['tool_calls'] enthalten.
Blockierend (urllib) — chat() laeuft ohnehin im Executor-Thread."""
if not isinstance(messages, list) or not messages:
return {"ok": False, "error": "messages leer/ungueltig"}
req = {"messages": messages, "max_tokens": max_tokens, "temperature": temperature}
@@ -41,6 +42,8 @@ def local_llm_chat(messages: list, *, max_tokens: int = 512,
req["stop"] = stop
if tools:
req["tools"] = tools
if model:
req["model"] = model
try:
body = json.dumps(req).encode("utf-8")
http_req = urllib.request.Request(
+5 -1
View File
@@ -30,7 +30,8 @@ CONFIG_PATH = os.environ.get("LOCAL_LLM_CONFIG", "/shared/config/local_llm.json"
ESCALATE_MARKER = "<<ESCALATE>>"
DEFAULT_CONFIG = {"enabled": False, "localOnly": False, "toolVariant": "slim"}
DEFAULT_CONFIG = {"enabled": False, "localOnly": False,
"toolVariant": "slim", "localLlmModel": "qwen3-8b"}
def load_config() -> dict:
@@ -42,6 +43,9 @@ def load_config() -> dict:
"enabled": bool(data.get("enabled", False)),
"localOnly": bool(data.get("localOnly", False)),
"toolVariant": data.get("toolVariant", "slim") or "slim",
# Welches lokale Modell llama-swap laden soll (B0.5). Muss zu einem
# Key in xtts/llama-swap/config.yaml passen.
"localLlmModel": (data.get("localLlmModel") or "qwen3-8b").strip(),
}
except (FileNotFoundError, json.JSONDecodeError):
return dict(DEFAULT_CONFIG)