feat(local-llm): B0.5 — llama-swap + lokale Modellauswahl in Diagnostic
Mehrere lokale Modelle, on-demand geladen/geswappt, in Diagnostic waehlbar. Design: das Brain schickt den Modellnamen (aus local_llm.json) im llm_request mit -> Adapter -> llama-swap laedt/swappt. Keine separate Gamebox-Config noetig. - xtts: `llama`-Container -> `llama-swap` (unified-cuda), config.yaml mit qwen3-8b (Standard) + qwen3-4b; Auto-Download via -hf, Cache /models geteilt (qwen3-8b schon da). Adapter -> llama-swap:8080, Timeout 600s (Erst-Download). - adapter: `model` aus dem Request an llama-swap durchreichen (Fallback env). - brain: router.load_config liest localLlmModel; local_llm_chat(model=...); agent gibt cfg-Modell mit; bridge reicht model durch (_local_llm + Route). - diagnostic: /api/local-models-list (aus /shared/config/local_models.json, seeded), local-llm-config um localLlmModel erweitert; Dropdown "Lokales Modell" im Settings-Block + Erst-Download-Hinweis. BLIND gebaut (Gamebox nicht testbar hier): llama-swap CLI/Config-Pfad beim ersten Start via `docker logs aria-llama-swap` pruefen. Live-Lade-Status (Adapter->Diagnostic) ist B0.5-2 (Folgeschritt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -3375,7 +3375,8 @@ class ARIABridge:
|
||||
_LLM_TIMEOUT_S = 30.0
|
||||
|
||||
async def _local_llm(self, messages: list, max_tokens: int = 512,
|
||||
temperature: float = 0.7, stop=None, tools=None) -> dict:
|
||||
temperature: float = 0.7, stop=None, tools=None,
|
||||
model=None) -> dict:
|
||||
"""Schickt einen llm_request an den llm-adapter (Gamebox), wartet auf
|
||||
llm_response. tools (B1b) werden durchgereicht; tool_calls kommen zurueck.
|
||||
Rueckgabe: {ok, content, tool_calls, model, elapsedMs} oder {ok:False, error}."""
|
||||
@@ -3399,8 +3400,10 @@ class ARIABridge:
|
||||
req_payload["stop"] = stop
|
||||
if tools:
|
||||
req_payload["tools"] = tools
|
||||
logger.info("[rvs] llm_request → llm-adapter (id=%s, msgs=%d, max_tokens=%d, tools=%d)",
|
||||
request_id[:8], len(messages), max_tokens, len(tools) if tools else 0)
|
||||
if model:
|
||||
req_payload["model"] = model
|
||||
logger.info("[rvs] llm_request → llm-adapter (id=%s, msgs=%d, max_tokens=%d, tools=%d, model=%s)",
|
||||
request_id[:8], len(messages), max_tokens, len(tools) if tools else 0, model or "-")
|
||||
ok = await self._send_to_rvs({
|
||||
"type": "llm_request",
|
||||
"payload": req_payload,
|
||||
@@ -3898,10 +3901,11 @@ class ARIABridge:
|
||||
except (TypeError, ValueError):
|
||||
temperature = 0.7
|
||||
_tools = data.get("tools") if isinstance(data.get("tools"), list) else None
|
||||
_model = data.get("model") if isinstance(data.get("model"), str) else None
|
||||
result = await self._local_llm(
|
||||
messages=messages, max_tokens=max_tokens,
|
||||
temperature=temperature, stop=data.get("stop"),
|
||||
tools=_tools,
|
||||
tools=_tools, model=_model,
|
||||
)
|
||||
status = 200 if result.get("ok") else 502
|
||||
await _send_response(writer, status, result)
|
||||
|
||||
Reference in New Issue
Block a user