feat(local-llm): B0.5 — llama-swap + lokale Modellauswahl in Diagnostic
Mehrere lokale Modelle, on-demand geladen/geswappt, in Diagnostic waehlbar. Design: das Brain schickt den Modellnamen (aus local_llm.json) im llm_request mit -> Adapter -> llama-swap laedt/swappt. Keine separate Gamebox-Config noetig. - xtts: `llama`-Container -> `llama-swap` (unified-cuda), config.yaml mit qwen3-8b (Standard) + qwen3-4b; Auto-Download via -hf, Cache /models geteilt (qwen3-8b schon da). Adapter -> llama-swap:8080, Timeout 600s (Erst-Download). - adapter: `model` aus dem Request an llama-swap durchreichen (Fallback env). - brain: router.load_config liest localLlmModel; local_llm_chat(model=...); agent gibt cfg-Modell mit; bridge reicht model durch (_local_llm + Route). - diagnostic: /api/local-models-list (aus /shared/config/local_models.json, seeded), local-llm-config um localLlmModel erweitert; Dropdown "Lokales Modell" im Settings-Block + Erst-Download-Hinweis. BLIND gebaut (Gamebox nicht testbar hier): llama-swap CLI/Config-Pfad beim ersten Start via `docker logs aria-llama-swap` pruefen. Live-Lade-Status (Adapter->Diagnostic) ist B0.5-2 (Folgeschritt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -29,11 +29,12 @@ LOCAL_LLM_HTTP_TIMEOUT_SEC = float(os.environ.get("LOCAL_LLM_HTTP_TIMEOUT_SEC",
|
||||
|
||||
|
||||
def local_llm_chat(messages: list, *, max_tokens: int = 512,
|
||||
temperature: float = 0.7, stop=None, tools=None) -> dict:
|
||||
temperature: float = 0.7, stop=None, tools=None,
|
||||
model=None) -> dict:
|
||||
"""Ein Chat-Call ans lokale LLM. messages = [{role, content}, ...].
|
||||
tools (B1b): optionale OpenAI-Tool-Defs; das Ergebnis kann dann
|
||||
result['tool_calls'] enthalten. Blockierend (urllib) — chat() laeuft
|
||||
ohnehin im Executor-Thread."""
|
||||
model (B0.5): welches Modell llama-swap laden soll. tools (B1b): optionale
|
||||
OpenAI-Tool-Defs; das Ergebnis kann dann result['tool_calls'] enthalten.
|
||||
Blockierend (urllib) — chat() laeuft ohnehin im Executor-Thread."""
|
||||
if not isinstance(messages, list) or not messages:
|
||||
return {"ok": False, "error": "messages leer/ungueltig"}
|
||||
req = {"messages": messages, "max_tokens": max_tokens, "temperature": temperature}
|
||||
@@ -41,6 +42,8 @@ def local_llm_chat(messages: list, *, max_tokens: int = 512,
|
||||
req["stop"] = stop
|
||||
if tools:
|
||||
req["tools"] = tools
|
||||
if model:
|
||||
req["model"] = model
|
||||
try:
|
||||
body = json.dumps(req).encode("utf-8")
|
||||
http_req = urllib.request.Request(
|
||||
|
||||
Reference in New Issue
Block a user