Compare commits
62 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 570eb031e0 | |||
| e9615d987e | |||
| 5e95eacd11 | |||
| ece08f0f2f | |||
| 31fd0d7f7a | |||
| 263835ad74 | |||
| ab7e9801ee | |||
| 3d001a1d03 | |||
| 91760dd2e1 | |||
| 3c2e537420 | |||
| 97b6ea1b3e | |||
| 94ee0455a2 | |||
| 0bf6d49432 | |||
| 493cba36a2 | |||
| a68827fb38 | |||
| 11ca316e4e | |||
| be1d2e950a | |||
| 199297a3a1 | |||
| e99bf0b032 | |||
| 41999c2304 | |||
| 095c1e2d70 | |||
| 0145179aca | |||
| c2475ffef6 | |||
| 98982fea2f | |||
| 356f8b3171 | |||
| b4115bb345 | |||
| 02cac99ef9 | |||
| 2940ce0075 | |||
| d78b668e31 | |||
| a9115699db | |||
| f2bfd4bbc6 | |||
| b182ef5ed5 | |||
| 9818dc1867 | |||
| 543ad3c46d | |||
| 408d20a087 | |||
| 0756baa2a0 | |||
| 27c9b1af96 | |||
| 70f4ff480e | |||
| c23daf14e3 | |||
| ebfde4cd1f | |||
| 5d3e3e5e8c | |||
| 0d69e211cb | |||
| 4ea13afe60 | |||
| d12bfd0302 | |||
| 8d5991f364 | |||
| 7d16a0f3e5 | |||
| 0a859f637b | |||
| 8c1476c2ca | |||
| 7d8c411f5c | |||
| fef2a32c50 | |||
| e7fd918559 | |||
| bb3c7957aa | |||
| 89cafa6251 | |||
| 1ea7ab5ab1 | |||
| 15f95ed196 | |||
| 210ce62ffe | |||
| 298b2202a1 | |||
| 845a8b0020 | |||
| 0540c49c66 | |||
| add303970b | |||
| fb71048dfd | |||
| aaaf118cb7 |
@@ -37,6 +37,12 @@ aria-data/brain/qdrant/
|
||||
# Diagnostic-State (aktive Session etc.)
|
||||
aria-data/config/diag-state/
|
||||
|
||||
# ── Shared Volume (Bind-Mount statt Docker-managed) ──
|
||||
# Enthaelt User-Uploads, Voice-Cloning-Samples, OAuth-Tokens,
|
||||
# chat_backup.jsonl, Memory-Attachments, runtime-state. Hunderte MB,
|
||||
# enthaelt PRIVATE Daten. Backup via Diagnostic, nicht via Git.
|
||||
aria-shared/
|
||||
|
||||
# ── Node / npm ──────────────────────────────────
|
||||
node_modules/
|
||||
npm-debug.log*
|
||||
|
||||
@@ -324,6 +324,111 @@ aria-brain → Antwort → Bridge → RVS → App
|
||||
|
||||
---
|
||||
|
||||
## Skills — Architektur
|
||||
|
||||
Skills sind ARIAs wiederverwendbare Faehigkeiten. Jeder Skill ist ein
|
||||
Python-Programm in seinem eigenen `local-venv`. ARIA legt sie selbst via
|
||||
`skill_create` an, fixt Bugs mit `skill_update`, rollt zur Not zurueck
|
||||
mit `skill_rollback`.
|
||||
|
||||
### Skill-Layout
|
||||
|
||||
```
|
||||
/data/skills/<name>/
|
||||
skill.json # Manifest (Metadata + config_schema + version_history)
|
||||
run.py # Entry-Point (Python via venv-python)
|
||||
requirements.txt # pip-Pakete fuer die venv
|
||||
README.md # Beschreibung
|
||||
venv/ # automatisch erzeugt
|
||||
logs/<ts>.json # Run-Logs (append-only)
|
||||
versions/v_<ts>/ # archivierte Vorgaengerstaende (vor jedem update_skill)
|
||||
```
|
||||
|
||||
### Drei-Stufen-Daten-Modell
|
||||
|
||||
Skills muessen **niemals** Credentials hardcoden. Drei saubere Wege:
|
||||
|
||||
1. **OAuth2-Tokens** (Spotify, Google, GitHub, Reddit, …): Brain haelt
|
||||
Client-Credentials und macht den Auth-Flow. Skill ruft
|
||||
`GET {BRAIN_INTERNAL_URL}/oauth/<service>/token` und bekommt einen
|
||||
frischen access_token (Auto-Refresh < 60 s Restzeit).
|
||||
2. **Statische Werte** (API-Keys, User-IDs, Default-Geraete): Skill
|
||||
deklariert ein `config_schema` in `skill.json`, Stefan setzt die
|
||||
Werte in Diagnostic / App, Skill bekommt sie zur Laufzeit als
|
||||
`CFG_<UPPER_NAME>` ENV.
|
||||
3. **Brain-Daten** (Memories, Skills-Liste, Standort etc.): jeder Skill
|
||||
kann gegen `BRAIN_INTERNAL_URL` Endpoints wie `/memory/search`,
|
||||
`/memory/pinned`, `/skills/list` rufen — z.B. ein Wetter-Skill kann
|
||||
Stefans Standort aus Memories holen statt ihn als Arg zu erwarten.
|
||||
|
||||
### Versionierung mit Rollback
|
||||
|
||||
`update_skill` archiviert den aktuellen Stand vor jeder strukturellen
|
||||
Aenderung (entry_code, readme, pip_packages, config_schema, args) nach
|
||||
`versions/v_<ts>/`. ARIA-Tools `skill_list_versions` + `skill_rollback`
|
||||
(+ HTTP `/skills/{name}/versions` + `/rollback`) erlauben Wiederherstellung.
|
||||
Vor jedem Rollback wird der aktuelle Stand als „safety-snapshot" gesichert
|
||||
— der Rollback selbst ist also nicht destruktiv.
|
||||
|
||||
UI sowohl in Diagnostic (Skill-Detail → 📦 Versionen) als auch in der App
|
||||
(SkillBrowser → Detail-Modal).
|
||||
|
||||
### Anti-Skill-Friedhof
|
||||
|
||||
ARIA hat frueher gerne 9 Spotify-Skills mit Suffixen `-v2`, `-aria`,
|
||||
`-ctl`, `-fixed` gebaut statt einen sauberen zu pflegen.
|
||||
`skills.create_skill()` rejected jetzt hart:
|
||||
|
||||
- Versions-Suffixe (`-v\d+`, `_v\d+`, `-new`, `-fixed`, `-old`,
|
||||
`-alt`, `-copy`, `-final`, `-clean`)
|
||||
- Prefix-Kollisionen (`spotify` existiert → `spotify-aria` rejected)
|
||||
|
||||
Plus die Skill-Regeln (siehe naechster Abschnitt) erinnern ARIA bei jedem
|
||||
Chat-Turn an die richtigen Patterns.
|
||||
|
||||
### Skill-Regeln (seed_rules)
|
||||
|
||||
`aria-brain/seed_rules.py` enthaelt 20 `type=rule, pinned=true,
|
||||
source=seed`-Memories, die bei jedem Brain-Start idempotent in die
|
||||
Vector-DB geschrieben werden (`migration_key`-basiert). Sie tauchen in
|
||||
jedem Chat-Turn im Hot-Memory-Block auf:
|
||||
|
||||
- **list-before-create** — IMMER `skill_list` vor `skill_create`
|
||||
- **no-version-suffix** — keine `-v2`/`_v3`-Namen, Versionsverwaltung ist intern
|
||||
- **update-not-recreate** — defekten Skill mit `skill_update` fixen, nicht neu bauen
|
||||
- **no-hardcoded-credentials** — OAuth-Tokens via `oauth_get_token`, keine client_secrets im Code
|
||||
- **config-schema-for-settings** — statische Werte via `config_schema`, nicht hardcoded
|
||||
- **brain-internal-url** — `BRAIN_INTERNAL_URL` Endpoints inkl. `/oauth/<s>/token`, `/memory/search`, `/memory/pinned`, `/skills/list`
|
||||
- **oauth-reauth-reflex** — bei 401: ZUERST `oauth_get_token` (Auto-Refresh), nur bei dessen Fehler `oauth_authorize`
|
||||
- **no-skill-drift** — kein Drift vom Skill zu Ad-hoc-Bash-Befehlen. Skill kaputt? `skill_logs` + `skill_update`. Niemals nur SAGEN „ich baue dir einen Skill", wenn `skill_create` nicht wirklich gefeuert wird
|
||||
- **runtime-topology** (architektur) — ARIA laeuft als `claude`-CLI-Subprocess IM aria-proxy Container (alpine — kein python3/jq), NICHT im aria-brain. `/data/skills/` und `BRAIN_INTERNAL_URL` existieren dort nicht. Brain-Resources via Brain-Tools (`oauth_get_token`, `memory_search`, `run_<skill>` …), nicht via Bash. SSH zur VM-Host via `ssh aria@host` (Key liegt im Proxy)
|
||||
- **scaffold-reflex** — ARIA entscheidet selbst ob ein wiederkehrender Bash-Pattern Skill-würdig ist (parametrisierbar + wiederkehrend + nicht-exploratory). Im Zweifel fragt sie Stefan. **Kein Auto-Scaffold, kein Tracking, keine Pflege** — Skills werden bewusst angelegt, nicht magisch. Pentest/Audit/Recherche bleibt ad-hoc Bash, auch bei 100× derselbe Host.
|
||||
- **external-api-auth-strategy** — OAuth2 → `oauth_get_token`, sonst `config_schema`, NIEMALS hardcoden
|
||||
|
||||
### Skill-Scaffold (Templates)
|
||||
|
||||
Statt jedes Mal einen kompletten Skill aus dem Nichts zu generieren,
|
||||
ruft ARIA `skill_scaffold(name, template, params)` — Brain expandiert
|
||||
ein passendes Skelett. Massiv niedrigere Hürde gegen Skill-Drift.
|
||||
|
||||
Drei mitgelieferte Templates (`aria-brain/skill_templates.py`):
|
||||
|
||||
| Template | Wofür | params |
|
||||
|---|---|---|
|
||||
| `oauth-api` | Spotify, GitHub, Reddit, Google, Discord — Token aus Brain mit Auto-Refresh | `{service: "spotify", base_url?}` |
|
||||
| `apikey-api` | OpenWeather, OpenAI, Twilio — statischer Key in `config_schema` → `CFG_<NAME>` ENV | `{api_name, key_env, auth_header?, auth_prefix?, base_url}` |
|
||||
| `file-process` | PDF/Bild/JSON-Wandler — Input aus `/shared/uploads/`, Output zurueck. `process()`-Stub, danach `skill_update` mit echtem Code | `{output_ext}` |
|
||||
|
||||
HTTP: `POST /skills/scaffold` + `GET /skills/templates` (Liste mit Param-Doku).
|
||||
Nach Scaffold optional `skill_update` falls Custom-Logik gebraucht wird.
|
||||
|
||||
Im Gegensatz zu `aria-data/brain-import/` (User-Saatgut, gitignored,
|
||||
manueller Diagnostic-Klick) gehoeren seed_rules zum Brain-Code und werden
|
||||
mit jedem Deploy ausgerollt. Editieren = `SEED_RULES`-Liste anpassen,
|
||||
Brain neu starten.
|
||||
|
||||
---
|
||||
|
||||
## Diagnostic — Selbstcheck-UI und Einstellungen
|
||||
|
||||
Erreichbar unter `http://<VM-IP>:3001`. Teilt das Netzwerk mit der Bridge.
|
||||
@@ -352,7 +457,10 @@ Erreichbar unter `http://<VM-IP>:3001`. Teilt das Netzwerk mit der Bridge.
|
||||
- **Voice Export/Import**: einzelne Stimmen als `.tar.gz` zwischen Gameboxen mitnehmen
|
||||
- **Settings Export/Import**: `voice_config.json` + `highlight_triggers.json` als JSON-Bundle
|
||||
- **Claude Login**: Browser-Terminal zum Einloggen in den Proxy
|
||||
- **ARIA Live**: read-only Mirror der Claude-Code-Session — alle Tool-Calls + Inputs + Outputs live in einer Monospace-Liste, farbcodiert. Plus ⛔ **Not-Aus**-Button der per RVS einen `cancel_request` mit `hard:true` ausloest → aria-bridge ruft den proxy-internen `/cancel-all` Side-Channel → alle Claude-Subprocesses werden sofort gekillt
|
||||
- **ARIA Live**: read-only Mirror der Claude-Code-Session — alle Tool-Calls + Inputs + Outputs live in einer Monospace-Liste, farbcodiert. **Persistenz**: jeder `agent_stream`-Event wird parallel in `/shared/logs/agent_stream.jsonl` (soft-cap 50 MB) geschrieben, Live-View laedt beim Tab-Oeffnen / Page-Reload die letzten 200 Eintraege — Browser-Standby wirft nichts mehr weg. Plus ⛔ **Not-Aus**-Button der per RVS einen `cancel_request` mit `hard:true` ausloest → aria-bridge ruft den proxy-internen `/cancel-all` Side-Channel → alle Claude-Subprocesses werden sofort gekillt
|
||||
- **Debug-API ohne SSH** (Diagnostic-Server, Port 3001):
|
||||
- `GET /api/chat-backup?lines=N` — letzte N Zeilen aus `chat_backup.jsonl` (Default 200, max 5000) als geparstes JSON. Hilfreich um nachzuvollziehen was ARIA tatsaechlich gemacht hat.
|
||||
- `GET /api/agent-stream?lines=N` — gleiche Mechanik fuer den persistierten Live-Stream (Tool-Calls + Inputs + Outputs).
|
||||
- **OAuth-Callback-Pipeline**: Caddy davor terminiert TLS via Let's Encrypt, RVS hat einen HTTP-Listener auf demselben Port wie der WebSocket. Provider (Spotify/Dropbox/Discord/...) redirecten den User an `https://{RVS_HOST}/oauth/callback/{service}` → RVS broadcastet als `oauth_callback`-WS-Message → aria-bridge forwarded an Brain → Brain matched `state`, tauscht `code` gegen Token, persistiert in `/shared/config/oauth_tokens.json`. Token-Refresh laeuft automatisch. ARIA hat vier Brain-Tools: **`oauth_register_provider`** (legt URLs eines neuen Providers wie Dropbox/Discord/Notion/... on-demand in `oauth_apps.json` an — Credentials bleiben Stefans Job), `oauth_authorize`, `oauth_get_token`, `oauth_revoke`
|
||||
|
||||
---
|
||||
@@ -929,6 +1037,12 @@ docker exec aria-brain curl localhost:8080/memory/stats
|
||||
- [x] **ARIA Live (Diagnostic) + Not-Aus**: read-only Mirror der Claude-Code-Session ersetzt den SSH-Tab. Tool-Calls + Inputs + Outputs (truncated 4 KB) live, farbcodiert. Roter ⛔ Not-Aus-Button schickt `cancel_request` mit `hard:true` → Bridge ruft den proxy-internen `/cancel-all` Side-Channel (Port 3457) → alle Claude-Subprocesses sofort tot. Plus: Idle-Watchdog im Proxy (20 min Inaktivitaet → Subprocess-Kill) + httpx-Timeout-Split im Brain (connect 10s / read 24h) damit lange Pentests durchlaufen
|
||||
- [x] **OAuth2-Pipeline ueber RVS-Callback**: Caddy mit Let's Encrypt vor dem RVS, HTTP-Route `/oauth/callback/{service}` broadcastet als `oauth_callback`-WS-Message, aria-bridge forwarded an Brain, Token landet in `/shared/config/oauth_tokens.json` (mode 0600). ARIAs `oauth_register_provider`-Tool legt neue Provider on-demand an (URLs/scopes, nicht Credentials). Diagnostic + App haben beide Provider-Verwaltung inklusive Custom-Provider-Anlage
|
||||
- [x] **Skill-Mgmt-Tools fuer ARIA**: `skill_update` (Code/README/pip_packages mit venv-Rebuild) + `skill_delete` — verhindert Skill-Friedhof mit `-v2`/`-fixed`-Suffixen. Plus App-seitiger SkillBrowser (Run + Live-Output + Logs der letzten 20 Runs) in Settings → 🛠️ Skills
|
||||
- [x] **Skill-Architektur P0-P4**:
|
||||
- `seed_rules` (9 pinned rule-Memories) werden bei jedem Brain-Boot idempotent in die DB geschrieben (`source=seed`, `migration_key`-basiert). Decken Skill-Friedhof, OAuth-Auth-Strategie, no-skill-drift, BRAIN_INTERNAL_URL ab
|
||||
- Anti-Friedhof-Check in `create_skill`: rejected Versions-Suffixe + Prefix-Kollisionen hart
|
||||
- Neuer Brain-HTTP-Endpoint `/oauth/<service>/token` + `BRAIN_INTERNAL_URL` ENV-Var fuer Skills — Skill ruft Brain fuer frischen Token statt client_secret hardzucoden
|
||||
- `config_schema` in skill.json + zentrales `/shared/config/skill_configs.json` + `CFG_<NAME>` ENV beim Run + `skill_set_config` Brain-Tool + UI in Diagnostic & App (TextInput / Switch / password-Felder mit `***SET***`-Masking)
|
||||
- Versionierung: jeder `skill_update` archiviert vorherigen Stand nach `versions/v_<ts>/` (ohne venv/logs). `skill_list_versions` + `skill_rollback` Brain-Tools (mit Safety-Snapshot + auto venv-Rebuild). UI mit Rollback-Button in Diagnostic & App
|
||||
- [x] **Bridge-Hang-Schutz + Voice-Speed persistent**: 3-Schichten-Watchdog (TCP-Keepalive + Asyncio-Watchdog + File-Based Liveness mit Self-Kill), TLS-Fallback klebt nicht mehr beim Reconnect. `xttsSpeed` jetzt im voice_config.json persistiert — greift auch bei Diagnostic-Chats und nach Bridge-Restart
|
||||
- [x] **Bubble-Aktionen in der App**: Long-Press oder ⎘-Icon auf einer Chat-Bubble → Aktions-Menu mit "📋 Ganzen Text teilen" plus pro extrahierte URL/E-Mail/Telefonnummer eine eigene Teilen-Option (System-Share-Sheet → Zwischenablage / Apps / Browser)
|
||||
|
||||
|
||||
@@ -79,8 +79,8 @@ android {
|
||||
applicationId "com.ariacockpit"
|
||||
minSdkVersion rootProject.ext.minSdkVersion
|
||||
targetSdkVersion rootProject.ext.targetSdkVersion
|
||||
versionCode 10603
|
||||
versionName "0.1.6.3"
|
||||
versionCode 10806
|
||||
versionName "0.1.8.6"
|
||||
// Fallback fuer Libraries mit Product Flavors
|
||||
missingDimensionStrategy 'react-native-camera', 'general'
|
||||
}
|
||||
|
||||
@@ -9,14 +9,26 @@
|
||||
<!-- Optional: GPS-Position der Frage anhaengen (nur wenn User in Settings aktiviert) -->
|
||||
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION" />
|
||||
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION" />
|
||||
<!-- Background-Location ist OPT-IN (Settings → GPS auch im Hintergrund).
|
||||
Muss vom User explizit in Android-Einstellungen auf "Immer erlauben"
|
||||
gesetzt werden — kann nicht ueber den normalen Permission-Dialog
|
||||
angefordert werden (Android 10+). Default: aus. -->
|
||||
<uses-permission android:name="android.permission.ACCESS_BACKGROUND_LOCATION" />
|
||||
<!-- Foreground-Service damit TTS auch bei minimierter App weiterlaeuft.
|
||||
FOREGROUND_SERVICE_MICROPHONE ist Pflicht ab Android 14 wenn der
|
||||
Service waehrend des Backgrounds aufs Mikro zugreift (Wake-Word,
|
||||
Aufnahme im Gespraechsmodus). -->
|
||||
Aufnahme im Gespraechsmodus). LOCATION wird nur aktiv wenn der
|
||||
User Background-GPS in Settings einschaltet. -->
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PLAYBACK" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MICROPHONE" />
|
||||
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_LOCATION" />
|
||||
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
|
||||
<!-- WAKE_LOCK damit Wake-Word + JS-Bridge auch bei aus-Display und Doze
|
||||
arbeiten: ohne Lock pausiert Android die CPU, Native-AudioRecord
|
||||
laeuft weiter aber JS-Bridge frisst die DeviceEvents nicht mehr ->
|
||||
Wake-Word wird erkannt aber callbacks feuern erst beim App-Resume. -->
|
||||
<uses-permission android:name="android.permission.WAKE_LOCK" />
|
||||
|
||||
<application
|
||||
android:name=".MainApplication"
|
||||
@@ -52,6 +64,6 @@
|
||||
<service
|
||||
android:name=".AriaPlaybackService"
|
||||
android:exported="false"
|
||||
android:foregroundServiceType="mediaPlayback|microphone" />
|
||||
android:foregroundServiceType="mediaPlayback|microphone|location" />
|
||||
</application>
|
||||
</manifest>
|
||||
|
||||
@@ -5,9 +5,11 @@ import android.app.NotificationChannel
|
||||
import android.app.NotificationManager
|
||||
import android.app.PendingIntent
|
||||
import android.app.Service
|
||||
import android.content.Context
|
||||
import android.content.Intent
|
||||
import android.os.Build
|
||||
import android.os.IBinder
|
||||
import android.os.PowerManager
|
||||
import android.util.Log
|
||||
import androidx.core.app.NotificationCompat
|
||||
|
||||
@@ -32,15 +34,26 @@ class AriaPlaybackService : Service() {
|
||||
|
||||
private var currentReason: String = ""
|
||||
|
||||
// PARTIAL_WAKE_LOCK haelt die CPU wach solange der Foreground-Service
|
||||
// aktiv ist. Damit bleibt die JS-Bridge im Doze ansprechbar und die
|
||||
// gesamte Sprach-Pipeline (Wake → Aufnahme → POST → ARIA → TTS → wieder
|
||||
// Wake) laeuft durchgehend im Hintergrund. Ein einziger Lock fuer den
|
||||
// ganzen Foreground-Cycle, nicht pro Sub-Modul.
|
||||
private var wakeLock: PowerManager.WakeLock? = null
|
||||
|
||||
override fun onCreate() {
|
||||
super.onCreate()
|
||||
ensureNotificationChannel()
|
||||
acquireWakeLock()
|
||||
}
|
||||
|
||||
override fun onStartCommand(intent: Intent?, flags: Int, startId: Int): Int {
|
||||
val reason = intent?.getStringExtra(EXTRA_REASON) ?: ""
|
||||
currentReason = reason
|
||||
Log.i(TAG, "Foreground-Service start/update (reason=$reason)")
|
||||
// Falls der Lock zwischendurch released wurde (z.B. nach onCreate-
|
||||
// race oder OS-quirk), hier sicherheits-halber erneut anfordern.
|
||||
acquireWakeLock()
|
||||
try {
|
||||
startForeground(NOTIFICATION_ID, buildNotification(reason))
|
||||
} catch (e: Exception) {
|
||||
@@ -53,10 +66,36 @@ class AriaPlaybackService : Service() {
|
||||
}
|
||||
|
||||
override fun onDestroy() {
|
||||
releaseWakeLock()
|
||||
Log.i(TAG, "Foreground-Service gestoppt")
|
||||
super.onDestroy()
|
||||
}
|
||||
|
||||
private fun acquireWakeLock() {
|
||||
if (wakeLock?.isHeld == true) return
|
||||
try {
|
||||
val pm = getSystemService(Context.POWER_SERVICE) as PowerManager
|
||||
wakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK,
|
||||
"AriaCockpit:Pipeline").apply {
|
||||
setReferenceCounted(false)
|
||||
acquire(8 * 60 * 60 * 1000L) // 8h Sicherheits-Cap
|
||||
}
|
||||
Log.i(TAG, "WakeLock acquired (CPU bleibt wach im Hintergrund)")
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "WakeLock acquire fehlgeschlagen: ${e.message}")
|
||||
}
|
||||
}
|
||||
|
||||
private fun releaseWakeLock() {
|
||||
try {
|
||||
wakeLock?.takeIf { it.isHeld }?.release()
|
||||
if (wakeLock != null) Log.i(TAG, "WakeLock released")
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "WakeLock release fehlgeschlagen: ${e.message}")
|
||||
}
|
||||
wakeLock = null
|
||||
}
|
||||
|
||||
override fun onBind(intent: Intent?): IBinder? = null
|
||||
|
||||
private fun ensureNotificationChannel() {
|
||||
|
||||
@@ -131,6 +131,58 @@ class AudioFocusModule(reactContext: ReactApplicationContext) : ReactContextBase
|
||||
promise.resolve(true)
|
||||
}
|
||||
|
||||
/** Sanfter Spotify-Resume-Nudge: kurz USAGE_MEDIA mit TRANSIENT
|
||||
* requesten und sofort abandonen. Spotify bekommt das als
|
||||
* Focus-Frei-Signal und resumed automatisch — aber weil TRANSIENT
|
||||
* (nicht GAIN permanent), interpretiert Spotify das NICHT als
|
||||
* "user stopped" was Auto-Resume verhindert haette.
|
||||
*
|
||||
* Hintergrund: ARIA spricht TTS via USAGE_ASSISTANT GAIN_TRANSIENT,
|
||||
* Spotify pausiert. ARIA released. Spotify SOLLTE nach
|
||||
* TRANSIENT-Loss + Abandon automatisch resumen, tut es aber bei
|
||||
* manchen Versionen / Geraeten nicht zuverlaessig. Dieser Nudge
|
||||
* triggert den Focus-Stack-Refresh ohne den Spotify-Auto-Stop-Bug
|
||||
* der alten kickReleaseMedia mit GAIN permanent.
|
||||
*/
|
||||
@ReactMethod
|
||||
fun nudgeMediaResume(promise: Promise) {
|
||||
val am = audioManager()
|
||||
if (am == null) {
|
||||
promise.resolve(false)
|
||||
return
|
||||
}
|
||||
Thread {
|
||||
try {
|
||||
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.O) {
|
||||
val attrs = AudioAttributes.Builder()
|
||||
.setUsage(AudioAttributes.USAGE_MEDIA)
|
||||
.setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)
|
||||
.build()
|
||||
val nudgeListener = AudioManager.OnAudioFocusChangeListener { /* ignorieren */ }
|
||||
val nudgeReq = AudioFocusRequest.Builder(AudioManager.AUDIOFOCUS_GAIN_TRANSIENT)
|
||||
.setAudioAttributes(attrs)
|
||||
.setOnAudioFocusChangeListener(nudgeListener)
|
||||
.build()
|
||||
am.requestAudioFocus(nudgeReq)
|
||||
Thread.sleep(100)
|
||||
am.abandonAudioFocusRequest(nudgeReq)
|
||||
} else {
|
||||
val nudgeListener = AudioManager.OnAudioFocusChangeListener { /* ignorieren */ }
|
||||
@Suppress("DEPRECATION")
|
||||
am.requestAudioFocus(nudgeListener, AudioManager.STREAM_MUSIC,
|
||||
AudioManager.AUDIOFOCUS_GAIN_TRANSIENT)
|
||||
Thread.sleep(100)
|
||||
@Suppress("DEPRECATION")
|
||||
am.abandonAudioFocus(nudgeListener)
|
||||
}
|
||||
Log.i(TAG, "nudgeMediaResume: USAGE_MEDIA TRANSIENT request+abandon (Spotify-Resume-Trigger)")
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "nudgeMediaResume failed: ${e.message}")
|
||||
}
|
||||
}.start()
|
||||
promise.resolve(true)
|
||||
}
|
||||
|
||||
/** Den USAGE_MEDIA-Focus-Stack im System aufmischen, damit Spotify/YouTube
|
||||
* resumen wenn ein anderer Player (z.B. react-native-sound) seinen Focus
|
||||
* nicht ordnungsgemaess released hat. Strategie: kurz selbst USAGE_MEDIA
|
||||
@@ -140,6 +192,10 @@ class AudioFocusModule(reactContext: ReactApplicationContext) : ReactContextBase
|
||||
*
|
||||
* Workaround fuer das react-native-sound-Bug: Sound.stop()/release()
|
||||
* laesst den AudioFocusRequest haengen.
|
||||
*
|
||||
* ⚠️ ACHTUNG: nutzt AUDIOFOCUS_GAIN (permanent), Spotify kann das als
|
||||
* "user-action stopp" interpretieren und Auto-Resume verhindern.
|
||||
* Fuer Spotify-Resume nach TTS lieber nudgeMediaResume() nehmen (sanfter).
|
||||
*/
|
||||
@ReactMethod
|
||||
fun kickReleaseMedia(promise: Promise) {
|
||||
|
||||
@@ -21,6 +21,7 @@ class MainApplication : Application(), ReactApplication {
|
||||
add(ApkInstallerPackage())
|
||||
add(AudioFocusPackage())
|
||||
add(PcmStreamPlayerPackage())
|
||||
add(PcmStreamRecorderPackage())
|
||||
add(OpenWakeWordPackage())
|
||||
add(PhoneCallPackage())
|
||||
add(BackgroundAudioPackage())
|
||||
|
||||
@@ -4,6 +4,7 @@ import ai.onnxruntime.OnnxTensor
|
||||
import ai.onnxruntime.OrtEnvironment
|
||||
import ai.onnxruntime.OrtSession
|
||||
import android.Manifest
|
||||
import android.content.Context
|
||||
import android.content.pm.PackageManager
|
||||
import android.media.AudioFormat
|
||||
import android.media.AudioRecord
|
||||
@@ -11,6 +12,7 @@ import android.media.MediaRecorder
|
||||
import android.media.audiofx.AcousticEchoCanceler
|
||||
import android.media.audiofx.AutomaticGainControl
|
||||
import android.media.audiofx.NoiseSuppressor
|
||||
import android.os.PowerManager
|
||||
import android.util.Log
|
||||
import androidx.core.content.ContextCompat
|
||||
import com.facebook.react.bridge.Promise
|
||||
@@ -80,6 +82,13 @@ class OpenWakeWordModule(reactContext: ReactApplicationContext) : ReactContextBa
|
||||
private var ns: NoiseSuppressor? = null
|
||||
private var agc: AutomaticGainControl? = null
|
||||
|
||||
// PARTIAL_WAKE_LOCK damit die CPU bei aus-Display nicht in Doze geht und
|
||||
// die JS-Bridge die WakeWordDetected-Events live verarbeitet (sonst
|
||||
// queuen sich die Events nur und werden erst beim App-Foreground
|
||||
// delivered — Stefan-Beobachtung: "Spotify pausiert, aber Gong/Aufnahme
|
||||
// kommen erst wenn ich die App nach vorne hole").
|
||||
private var wakeLock: PowerManager.WakeLock? = null
|
||||
|
||||
// Inferenz-State
|
||||
private val melBuffer: ArrayList<FloatArray> = ArrayList(256) // Liste von 32-dim Frames
|
||||
private var melProcessedIdx: Int = 0
|
||||
@@ -198,6 +207,21 @@ class OpenWakeWordModule(reactContext: ReactApplicationContext) : ReactContextBa
|
||||
running.set(true)
|
||||
record.startRecording()
|
||||
|
||||
// PARTIAL_WAKE_LOCK greifen damit die CPU nicht in Doze geht und
|
||||
// die JS-Bridge die emit("WakeWordDetected")-Events live verarbeitet.
|
||||
// 8h Cap als Sicherheit gegen forgotten-release.
|
||||
try {
|
||||
val pm = reactApplicationContext.getSystemService(Context.POWER_SERVICE) as PowerManager
|
||||
wakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK,
|
||||
"AriaCockpit:WakeWordRecord").apply {
|
||||
setReferenceCounted(false)
|
||||
acquire(8 * 60 * 60 * 1000L)
|
||||
}
|
||||
Log.i(TAG, "WakeLock acquired")
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "WakeLock acquire fehlgeschlagen: ${e.message}")
|
||||
}
|
||||
|
||||
captureThread = Thread({ captureLoop() }, "OpenWakeWordCapture").apply {
|
||||
isDaemon = true
|
||||
start()
|
||||
@@ -232,6 +256,7 @@ class OpenWakeWordModule(reactContext: ReactApplicationContext) : ReactContextBa
|
||||
try { audioRecord?.release() } catch (_: Exception) {}
|
||||
audioRecord = null
|
||||
releaseAudioEffects()
|
||||
releaseWakeLock()
|
||||
Log.i(TAG, "Lauschen gestoppt")
|
||||
promise.resolve(true)
|
||||
}
|
||||
@@ -245,10 +270,21 @@ class OpenWakeWordModule(reactContext: ReactApplicationContext) : ReactContextBa
|
||||
try { audioRecord?.release() } catch (_: Exception) {}
|
||||
audioRecord = null
|
||||
releaseAudioEffects()
|
||||
releaseWakeLock()
|
||||
disposeSessions()
|
||||
promise.resolve(true)
|
||||
}
|
||||
|
||||
private fun releaseWakeLock() {
|
||||
try {
|
||||
wakeLock?.takeIf { it.isHeld }?.release()
|
||||
if (wakeLock != null) Log.i(TAG, "WakeLock released")
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "WakeLock release fehlgeschlagen: ${e.message}")
|
||||
}
|
||||
wakeLock = null
|
||||
}
|
||||
|
||||
@ReactMethod
|
||||
fun isAvailable(promise: Promise) {
|
||||
// Wake-Word ist immer verfuegbar (kein API-Key, alles on-device)
|
||||
|
||||
@@ -361,6 +361,12 @@ class PcmStreamPlayerModule(reactContext: ReactApplicationContext) : ReactContex
|
||||
writerThread = null
|
||||
val t = track
|
||||
if (t != null) {
|
||||
// pause() + flush() vor stop() — sonst spielt der Hardware-Buffer
|
||||
// (200-500ms PCM-Samples) noch hörbar weiter, nachdem der User
|
||||
// den Mute-Button gedrückt hat. Stefan-Bug-Report: "wenn ich auf
|
||||
// den Mund halten Button klicke während ARIA redet stoppt sie nicht".
|
||||
try { t.pause() } catch (_: Exception) {}
|
||||
try { t.flush() } catch (_: Exception) {}
|
||||
try { t.stop() } catch (_: Exception) {}
|
||||
try { t.release() } catch (_: Exception) {}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,246 @@
|
||||
package com.ariacockpit
|
||||
|
||||
import android.Manifest
|
||||
import android.content.Context
|
||||
import android.content.pm.PackageManager
|
||||
import android.media.AudioFormat
|
||||
import android.media.AudioRecord
|
||||
import android.media.MediaRecorder
|
||||
import android.media.audiofx.AcousticEchoCanceler
|
||||
import android.media.audiofx.AutomaticGainControl
|
||||
import android.media.audiofx.NoiseSuppressor
|
||||
import android.os.PowerManager
|
||||
import android.util.Base64
|
||||
import android.util.Log
|
||||
import androidx.core.content.ContextCompat
|
||||
import com.facebook.react.bridge.Arguments
|
||||
import com.facebook.react.bridge.Promise
|
||||
import com.facebook.react.bridge.ReactApplicationContext
|
||||
import com.facebook.react.bridge.ReactContextBaseJavaModule
|
||||
import com.facebook.react.bridge.ReactMethod
|
||||
import com.facebook.react.modules.core.DeviceEventManagerModule
|
||||
import java.util.concurrent.atomic.AtomicBoolean
|
||||
|
||||
/**
|
||||
* PCM-Streaming-Recorder fuer die Streaming-Whisper-Bridge.
|
||||
*
|
||||
* Oeffnet AudioRecord (16 kHz mono s16le, VOICE_COMMUNICATION-Source mit
|
||||
* automatischer AEC + NS) und feuert ~200ms-Chunks als base64-Event
|
||||
* "PcmStreamChunk" an die JS-Bridge.
|
||||
*
|
||||
* audio.ts schickt die Chunks via RVS direkt an die whisper-bridge die
|
||||
* dort einen ML-Endpointer laufen laesst — kein dB-VAD-Tuning mehr.
|
||||
*
|
||||
* Mic-Ownership: dieser Recorder DARF nicht gleichzeitig mit
|
||||
* OpenWakeWord laufen — beide wollen AudioRecord vom MIC. Caller
|
||||
* muss OpenWakeWord.stop() vor start() hier aufrufen und nach stop()
|
||||
* hier wieder OpenWakeWord.start() — genau wie's audio.ts ohnehin
|
||||
* macht.
|
||||
*
|
||||
* Events:
|
||||
* "PcmStreamChunk" { pcm: base64-s16le, seq: N, ts: epochMs }
|
||||
* "PcmStreamError" { error: string }
|
||||
*/
|
||||
class PcmStreamRecorderModule(reactContext: ReactApplicationContext) :
|
||||
ReactContextBaseJavaModule(reactContext) {
|
||||
|
||||
override fun getName() = "PcmStreamRecorder"
|
||||
|
||||
companion object {
|
||||
private const val TAG = "PcmStreamRecorder"
|
||||
private const val SAMPLE_RATE = 16000
|
||||
// 200ms-Chunks: gross genug fuer wenig RVS-Overhead, klein genug damit
|
||||
// der Endpointer im Whisper-Bridge granular sieht. 200ms ist auch das
|
||||
// Whisper-VAD-Frame-Hop — passt also zu downstream.
|
||||
private const val CHUNK_SAMPLES = 3200 // 200ms @ 16 kHz
|
||||
private const val BYTES_PER_SAMPLE = 2 // s16
|
||||
private const val CHUNK_BYTES = CHUNK_SAMPLES * BYTES_PER_SAMPLE
|
||||
}
|
||||
|
||||
private var audioRecord: AudioRecord? = null
|
||||
private val running = AtomicBoolean(false)
|
||||
private var captureThread: Thread? = null
|
||||
|
||||
private var aec: AcousticEchoCanceler? = null
|
||||
private var ns: NoiseSuppressor? = null
|
||||
private var agc: AutomaticGainControl? = null
|
||||
|
||||
// PARTIAL_WAKE_LOCK damit der JS-Bridge-Loop weiterlaeuft auch wenn das
|
||||
// Display aus ist — sonst sammeln sich zwar Chunks in der nativen Queue
|
||||
// an, aber emit() landet nicht zeitnah in JS und der Whisper-Bridge
|
||||
// bekommt die Audio-Chunks erst beim App-Foreground-Resume.
|
||||
private var wakeLock: PowerManager.WakeLock? = null
|
||||
|
||||
private var seq: Long = 0L
|
||||
|
||||
@ReactMethod
|
||||
fun start(promise: Promise) {
|
||||
if (running.get()) {
|
||||
promise.resolve(true)
|
||||
return
|
||||
}
|
||||
val perm = ContextCompat.checkSelfPermission(
|
||||
reactApplicationContext, Manifest.permission.RECORD_AUDIO
|
||||
)
|
||||
if (perm != PackageManager.PERMISSION_GRANTED) {
|
||||
promise.reject("NO_MIC_PERMISSION", "RECORD_AUDIO Permission fehlt")
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
val minBuf = AudioRecord.getMinBufferSize(
|
||||
SAMPLE_RATE,
|
||||
AudioFormat.CHANNEL_IN_MONO,
|
||||
AudioFormat.ENCODING_PCM_16BIT,
|
||||
).coerceAtLeast(CHUNK_BYTES * 4) // 4x Chunk-Size als Sicherheit
|
||||
|
||||
val record = AudioRecord(
|
||||
MediaRecorder.AudioSource.VOICE_COMMUNICATION,
|
||||
SAMPLE_RATE,
|
||||
AudioFormat.CHANNEL_IN_MONO,
|
||||
AudioFormat.ENCODING_PCM_16BIT,
|
||||
minBuf,
|
||||
)
|
||||
if (record.state != AudioRecord.STATE_INITIALIZED) {
|
||||
record.release()
|
||||
promise.reject("AUDIO_INIT", "AudioRecord nicht initialisiert (Mikro belegt? OpenWakeWord noch aktiv?)")
|
||||
return
|
||||
}
|
||||
audioRecord = record
|
||||
|
||||
// AEC/NS/AGC explizit anschalten — manche Geraete liefern's via
|
||||
// VOICE_COMMUNICATION zwar mit, aber Belt-and-Suspenders.
|
||||
try {
|
||||
if (AcousticEchoCanceler.isAvailable()) {
|
||||
aec = AcousticEchoCanceler.create(record.audioSessionId)?.apply { enabled = true }
|
||||
}
|
||||
} catch (e: Exception) { Log.w(TAG, "AEC failed: ${e.message}") }
|
||||
try {
|
||||
if (NoiseSuppressor.isAvailable()) {
|
||||
ns = NoiseSuppressor.create(record.audioSessionId)?.apply { enabled = true }
|
||||
}
|
||||
} catch (e: Exception) { Log.w(TAG, "NS failed: ${e.message}") }
|
||||
try {
|
||||
if (AutomaticGainControl.isAvailable()) {
|
||||
agc = AutomaticGainControl.create(record.audioSessionId)?.apply { enabled = true }
|
||||
}
|
||||
} catch (e: Exception) { Log.w(TAG, "AGC failed: ${e.message}") }
|
||||
|
||||
seq = 0L
|
||||
running.set(true)
|
||||
record.startRecording()
|
||||
|
||||
try {
|
||||
val pm = reactApplicationContext.getSystemService(Context.POWER_SERVICE) as PowerManager
|
||||
wakeLock = pm.newWakeLock(PowerManager.PARTIAL_WAKE_LOCK,
|
||||
"AriaCockpit:PcmStreamRecord").apply {
|
||||
setReferenceCounted(false)
|
||||
acquire(8 * 60 * 60 * 1000L) // 8h Cap
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.w(TAG, "WakeLock acquire fehlgeschlagen: ${e.message}")
|
||||
}
|
||||
|
||||
captureThread = Thread({ captureLoop() }, "PcmStreamRecorderCapture").apply {
|
||||
isDaemon = true
|
||||
start()
|
||||
}
|
||||
|
||||
Log.i(TAG, "Recording gestartet (16kHz mono s16le, ${CHUNK_SAMPLES} samples/chunk)")
|
||||
promise.resolve(true)
|
||||
} catch (e: Exception) {
|
||||
Log.e(TAG, "start fehlgeschlagen", e)
|
||||
running.set(false)
|
||||
audioRecord?.release()
|
||||
audioRecord = null
|
||||
releaseAudioEffects()
|
||||
releaseWakeLock()
|
||||
promise.reject("START_FAILED", e.message ?: "Unbekannter Fehler", e)
|
||||
}
|
||||
}
|
||||
|
||||
@ReactMethod
|
||||
fun stop(promise: Promise) {
|
||||
running.set(false)
|
||||
try {
|
||||
captureThread?.join(1500)
|
||||
} catch (_: InterruptedException) {}
|
||||
captureThread = null
|
||||
try { audioRecord?.stop() } catch (_: Exception) {}
|
||||
try { audioRecord?.release() } catch (_: Exception) {}
|
||||
audioRecord = null
|
||||
releaseAudioEffects()
|
||||
releaseWakeLock()
|
||||
Log.i(TAG, "Recording gestoppt (seq=$seq Chunks gesendet)")
|
||||
promise.resolve(true)
|
||||
}
|
||||
|
||||
@ReactMethod
|
||||
fun isRecording(promise: Promise) {
|
||||
promise.resolve(running.get())
|
||||
}
|
||||
|
||||
private fun captureLoop() {
|
||||
val buffer = ByteArray(CHUNK_BYTES)
|
||||
val rec = audioRecord ?: return
|
||||
try {
|
||||
while (running.get()) {
|
||||
var offset = 0
|
||||
// Solange lesen bis ein voller 200ms-Chunk zusammen ist.
|
||||
// AudioRecord.read kann weniger als angefordert liefern.
|
||||
while (offset < CHUNK_BYTES && running.get()) {
|
||||
val n = rec.read(buffer, offset, CHUNK_BYTES - offset)
|
||||
if (n <= 0) {
|
||||
if (!running.get()) break
|
||||
// Fehlerzustand — kurze Pause, dann weiter probieren
|
||||
Thread.sleep(5)
|
||||
continue
|
||||
}
|
||||
offset += n
|
||||
}
|
||||
if (offset < CHUNK_BYTES) break
|
||||
|
||||
val b64 = Base64.encodeToString(buffer, Base64.NO_WRAP)
|
||||
val ts = System.currentTimeMillis()
|
||||
val params = Arguments.createMap().apply {
|
||||
putString("pcm", b64)
|
||||
// putLong existiert nicht in WritableMap — putDouble fuer ts/seq.
|
||||
putDouble("seq", seq.toDouble())
|
||||
putDouble("ts", ts.toDouble())
|
||||
}
|
||||
reactApplicationContext
|
||||
.getJSModule(DeviceEventManagerModule.RCTDeviceEventEmitter::class.java)
|
||||
.emit("PcmStreamChunk", params)
|
||||
seq++
|
||||
}
|
||||
} catch (e: Exception) {
|
||||
Log.e(TAG, "captureLoop crashed", e)
|
||||
try {
|
||||
val err = Arguments.createMap().apply {
|
||||
putString("error", e.message ?: "unknown")
|
||||
}
|
||||
reactApplicationContext
|
||||
.getJSModule(DeviceEventManagerModule.RCTDeviceEventEmitter::class.java)
|
||||
.emit("PcmStreamError", err)
|
||||
} catch (_: Exception) {}
|
||||
}
|
||||
}
|
||||
|
||||
private fun releaseAudioEffects() {
|
||||
try { aec?.release() } catch (_: Exception) {}
|
||||
try { ns?.release() } catch (_: Exception) {}
|
||||
try { agc?.release() } catch (_: Exception) {}
|
||||
aec = null; ns = null; agc = null
|
||||
}
|
||||
|
||||
private fun releaseWakeLock() {
|
||||
try {
|
||||
if (wakeLock?.isHeld == true) wakeLock?.release()
|
||||
} catch (_: Exception) {}
|
||||
wakeLock = null
|
||||
}
|
||||
|
||||
// Damit RCTEventEmitter den Listener-Lifecycle nicht crasht
|
||||
@ReactMethod fun addListener(eventName: String) {}
|
||||
@ReactMethod fun removeListeners(count: Int) {}
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
package com.ariacockpit
|
||||
|
||||
import com.facebook.react.ReactPackage
|
||||
import com.facebook.react.bridge.NativeModule
|
||||
import com.facebook.react.bridge.ReactApplicationContext
|
||||
import com.facebook.react.uimanager.ViewManager
|
||||
|
||||
class PcmStreamRecorderPackage : ReactPackage {
|
||||
override fun createNativeModules(reactContext: ReactApplicationContext): List<NativeModule> {
|
||||
return listOf(PcmStreamRecorderModule(reactContext))
|
||||
}
|
||||
|
||||
override fun createViewManagers(reactContext: ReactApplicationContext): List<ViewManager<*, *>> {
|
||||
return emptyList()
|
||||
}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "aria-cockpit",
|
||||
"version": "0.1.6.3",
|
||||
"version": "0.1.8.6",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"android": "react-native run-android",
|
||||
|
||||
@@ -1,12 +1,19 @@
|
||||
/**
|
||||
* VoiceButton - Push-to-Talk + Auto-Stop Aufnahmeknopf
|
||||
* VoiceButton — Tap-to-Talk-Aufnahmeknopf (Streaming-Variante).
|
||||
*
|
||||
* Zwei Modi:
|
||||
* 1. Push-to-Talk: gedrueckt halten zum Aufnehmen, loslassen zum Senden
|
||||
* 2. Tap-to-Talk: einmal tippen startet Aufnahme, VAD stoppt automatisch bei Stille
|
||||
* (auch genutzt fuer Wake-Word-getriggerte Aufnahme)
|
||||
* Push-to-Talk gibt's nicht mehr. Tap startet Streaming-Aufnahme an die
|
||||
* Whisper-Bridge. Tap nochmal sendet stt_stream_end → Whisper liefert den
|
||||
* finalen Text → aria-bridge forwardet direkt an Brain. Keine dB/VAD-
|
||||
* Stille-Erkennung mehr — Whisper hoert auf semantische Stille (kein
|
||||
* neuer Text mehr).
|
||||
*
|
||||
* Visuelles Feedback durch pulsierende Animation waehrend der Aufnahme.
|
||||
* Diese Komponente ist absichtlich "dumm": sie kapselt nur den
|
||||
* Tap-Lifecycle + die Animation. Recording-Optionen (voice/speed/
|
||||
* location/interrupted) baut ChatScreen, die User-Bubble ebenfalls.
|
||||
*
|
||||
* Visuelles Feedback: pulsierende Animation + Dauer + dB-Pegel via
|
||||
* audioService.onMeterUpdate (das macht audio.ts noch fuer alte Records;
|
||||
* neu kommt der Pegel via NativeEventEmitter (PcmStreamMeter) — folgt).
|
||||
*/
|
||||
|
||||
import React, { useState, useRef, useEffect, useCallback } from 'react';
|
||||
@@ -17,25 +24,28 @@ import {
|
||||
StyleSheet,
|
||||
Easing,
|
||||
TouchableOpacity,
|
||||
Pressable,
|
||||
} from 'react-native';
|
||||
import audioService, { RecordingResult } from '../services/audio';
|
||||
import audioService, { RecordingState } from '../services/audio';
|
||||
|
||||
// --- Typen ---
|
||||
|
||||
interface VoiceButtonProps {
|
||||
/** Wird aufgerufen wenn die Aufnahme fertig ist */
|
||||
onRecordingComplete: (result: RecordingResult) => void;
|
||||
/** User hat getippt — ChatScreen soll Bubble bauen + startStreamingRecording.
|
||||
* Returns true wenn die Aufnahme tatsaechlich gestartet ist. */
|
||||
onTapStart: () => Promise<boolean>;
|
||||
/** User hat nochmal getippt — ChatScreen soll stopStreamingRecording rufen. */
|
||||
onTapStop: () => Promise<void>;
|
||||
/** Button deaktivieren */
|
||||
disabled?: boolean;
|
||||
/** Wake-Word-Modus aktiv (zeigt Indikator) */
|
||||
/** Wake-Word-Modus aktiv (zeigt gruenen Indikator-Dot) */
|
||||
wakeWordActive?: boolean;
|
||||
}
|
||||
|
||||
// --- Komponente ---
|
||||
|
||||
const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
onRecordingComplete,
|
||||
onTapStart,
|
||||
onTapStop,
|
||||
disabled = false,
|
||||
wakeWordActive = false,
|
||||
}) => {
|
||||
@@ -45,6 +55,21 @@ const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
const pulseAnim = useRef(new Animated.Value(1)).current;
|
||||
const durationTimer = useRef<ReturnType<typeof setInterval> | null>(null);
|
||||
|
||||
// State via audioService.onStateChange spiegeln — der Service ist die
|
||||
// Quelle der Wahrheit (Streaming-Session, Wake-Word-Multi-Turn, etc.
|
||||
// koennen den Recording-State von extern aendern). isStreamingRecording
|
||||
// ist auch true wenn die Wake-Word-Konversation gerade aufzeichnet —
|
||||
// dann zeigt der Button "stop"-Symbol, und Tap stoppt die laufende
|
||||
// Aufnahme (egal ob via Wake-Word oder Knopf gestartet).
|
||||
useEffect(() => {
|
||||
const unsub = audioService.onStateChange((next: RecordingState) => {
|
||||
setIsRecording(next === 'recording');
|
||||
});
|
||||
// Initial-State synchronisieren
|
||||
setIsRecording(audioService.getRecordingState() === 'recording');
|
||||
return unsub;
|
||||
}, []);
|
||||
|
||||
// Puls-Animation starten/stoppen
|
||||
useEffect(() => {
|
||||
if (isRecording) {
|
||||
@@ -71,14 +96,13 @@ const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
}
|
||||
}, [isRecording, pulseAnim]);
|
||||
|
||||
// Aufnahmedauer zaehlen + Metering
|
||||
// Aufnahmedauer zaehlen + Metering (Pegel-Bar)
|
||||
useEffect(() => {
|
||||
if (isRecording) {
|
||||
setDurationMs(0);
|
||||
durationTimer.current = setInterval(() => {
|
||||
setDurationMs(prev => prev + 100);
|
||||
}, 100);
|
||||
|
||||
const unsubMeter = audioService.onMeterUpdate(setMeterDb);
|
||||
return () => {
|
||||
unsubMeter();
|
||||
@@ -89,74 +113,28 @@ const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
clearInterval(durationTimer.current);
|
||||
durationTimer.current = null;
|
||||
}
|
||||
setMeterDb(-160);
|
||||
}
|
||||
}, [isRecording]);
|
||||
|
||||
// VAD Silence Callback — Auto-Stop.
|
||||
// WICHTIG: NICHT auf isRecording prüfen (Closure ist stale) — stattdessen
|
||||
// audioService selber fragen. Empty deps → Listener wird EINMAL registriert.
|
||||
// audioService garantiert jetzt dass der Callback pro Aufnahme nur einmal
|
||||
// feuert (silenceFired-Latch).
|
||||
const onCompleteRef = useRef(onRecordingComplete);
|
||||
useEffect(() => { onCompleteRef.current = onRecordingComplete; }, [onRecordingComplete]);
|
||||
useEffect(() => {
|
||||
const unsubSilence = audioService.onSilenceDetected(async () => {
|
||||
if (audioService.getRecordingState() !== 'recording') return;
|
||||
const result = await audioService.stopRecording();
|
||||
setIsRecording(false);
|
||||
if (result && result.durationMs > 500) {
|
||||
onCompleteRef.current(result);
|
||||
}
|
||||
});
|
||||
return unsubSilence;
|
||||
}, []);
|
||||
|
||||
// Auto-Start fuer Wake Word (extern getriggert)
|
||||
const startAutoRecording = useCallback(async () => {
|
||||
if (disabled || isRecording) return;
|
||||
const started = await audioService.startRecording(true); // autoStop = true
|
||||
if (started) {
|
||||
setIsRecording(true);
|
||||
}
|
||||
}, [disabled, isRecording]);
|
||||
|
||||
// Tap-to-Talk: Einmal tippen startet mit Auto-Stop.
|
||||
// Guard gegen Doppel-Tap während asyncer Start/Stop.
|
||||
// Tap-Handler. Guard gegen Doppel-Tap waehrend asyncer Start/Stop.
|
||||
const tapBusy = useRef(false);
|
||||
const handleTap = async () => {
|
||||
const handleTap = useCallback(async () => {
|
||||
if (disabled || tapBusy.current) return;
|
||||
tapBusy.current = true;
|
||||
try {
|
||||
// Fragen WIR den Service, nicht den React-State (Closure kann stale sein)
|
||||
// Service-State fragen statt React-State (Closure koennte stale sein)
|
||||
const svcState = audioService.getRecordingState();
|
||||
if (svcState === 'recording') {
|
||||
// Aufnahme manuell stoppen
|
||||
const result = await audioService.stopRecording();
|
||||
setIsRecording(false);
|
||||
if (result && result.durationMs > 300) {
|
||||
onRecordingComplete(result);
|
||||
}
|
||||
await onTapStop();
|
||||
} else if (svcState === 'idle') {
|
||||
// Aufnahme mit Auto-Stop starten
|
||||
const started = await audioService.startRecording(true);
|
||||
if (started) {
|
||||
setIsRecording(true);
|
||||
}
|
||||
await onTapStart();
|
||||
}
|
||||
// svcState === 'processing': Stopp in progress — nichts tun, User
|
||||
// muss nochmal tippen wenn fertig. Aber wir blockieren mit tapBusy
|
||||
// kurz damit der User's UI-Feedback synchron bleibt.
|
||||
// 'processing': Stop laeuft gerade — nichts tun, User muss nochmal tippen
|
||||
} finally {
|
||||
tapBusy.current = false;
|
||||
}
|
||||
};
|
||||
|
||||
// Expose startAutoRecording via ref fuer Wake Word
|
||||
React.useImperativeHandle(
|
||||
React.createRef(),
|
||||
() => ({ startAutoRecording }),
|
||||
[startAutoRecording],
|
||||
);
|
||||
}, [disabled, onTapStart, onTapStop]);
|
||||
|
||||
const formatDuration = (ms: number): string => {
|
||||
const seconds = Math.floor(ms / 1000);
|
||||
@@ -164,7 +142,11 @@ const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
return `${seconds}.${tenths}s`;
|
||||
};
|
||||
|
||||
// Meter-Visualisierung (0-1 Skala)
|
||||
// Meter-Visualisierung (-60..0 dB → 0..1). Bei Streaming-Mode liefert
|
||||
// audio.ts (noch) keinen Pegel, also bleibt der Balken leer — wird in
|
||||
// einem Folge-Commit nachgerueckt (PcmStreamRecorder-Module muss dafuer
|
||||
// einen RMS-Wert mit-emitten). Tut der Streaming-Funktion keinen Abbruch,
|
||||
// ist reines UI-Beiwerk.
|
||||
const meterLevel = Math.max(0, Math.min(1, (meterDb + 60) / 60));
|
||||
|
||||
return (
|
||||
@@ -198,9 +180,6 @@ const VoiceButton: React.FC<VoiceButtonProps> = ({
|
||||
);
|
||||
};
|
||||
|
||||
// Expose startAutoRecording fuer externe Aufrufe (Wake Word)
|
||||
export type VoiceButtonHandle = { startAutoRecording: () => Promise<void> };
|
||||
|
||||
// --- Styles ---
|
||||
|
||||
const styles = StyleSheet.create({
|
||||
|
||||
@@ -47,7 +47,7 @@ import VoiceButton from '../components/VoiceButton';
|
||||
import FileUpload, { FileData } from '../components/FileUpload';
|
||||
import CameraUpload, { PhotoData } from '../components/CameraUpload';
|
||||
import MessageText from '../components/MessageText';
|
||||
import { RecordingResult, loadConvWindowMs, loadTtsSpeed, TTS_SPEED_DEFAULT } from '../services/audio';
|
||||
import { loadConvWindowMs, loadTtsSpeed, TTS_SPEED_DEFAULT } from '../services/audio';
|
||||
import Geolocation from '@react-native-community/geolocation';
|
||||
|
||||
// --- Typen ---
|
||||
@@ -531,7 +531,14 @@ const ChatScreen: React.FC = () => {
|
||||
if (bgDur > 30_000) {
|
||||
wakeWordService.discardIfFreshlyTriggered(15_000).then(discarded => {
|
||||
if (discarded) {
|
||||
try { audioService.cancelRecording(); } catch {}
|
||||
// Sowohl legacy als auch Streaming-Pfad abdecken
|
||||
try {
|
||||
if (audioService.isStreamingRecording()) {
|
||||
audioService.cancelStreamingRecording('wake-discarded');
|
||||
} else {
|
||||
audioService.cancelRecording();
|
||||
}
|
||||
} catch {}
|
||||
}
|
||||
}).catch(() => {});
|
||||
}
|
||||
@@ -1256,71 +1263,104 @@ const ChatScreen: React.FC = () => {
|
||||
return () => { unsubUpdate(); clearTimeout(timer); };
|
||||
}, []);
|
||||
|
||||
// Gespraechsmodus: Nach TTS-Wiedergabe automatisch Aufnahme starten
|
||||
// Gespraechsmodus: Nach TTS-Wiedergabe weiter im Multi-Turn (Conversation-
|
||||
// Window) oder zurueck zu armed (Wake-Word lauscht wieder)?
|
||||
//
|
||||
// Foreground → resume() oeffnet das Mikro fuer N Sekunden Follow-Up
|
||||
// (natuerlicher Dialog moeglich ohne erneutes "Computer")
|
||||
// Background → endConversation() — Wake-Word direkt wieder armed.
|
||||
//
|
||||
// Grund: der setTimeout(800ms) in resume() wird im Doze stark verzoegert
|
||||
// (siehe Wake-Detect-Bug von 0.1.7.0). Das hat zwei nervige Folgen:
|
||||
// 1) Wake-Word ist solange "tot" — User kann ARIA nicht mehr triggern
|
||||
// bis er die App vorholt
|
||||
// 2) Wenn er die App dann vorholt, oeffnet der verspaetete Timer das
|
||||
// Mikro — sieht aus wie ein Phantom-Wake-Word-Trigger
|
||||
// Background = User nutzt das Handy anderweitig, das Multi-Turn-Konzept
|
||||
// ist da eh nicht nuetzlich. Direkt re-armen ist robust und erwartungs-
|
||||
// konform.
|
||||
useEffect(() => {
|
||||
const unsubPlayback = audioService.onPlaybackFinished(() => {
|
||||
if (wakeWordService.isActive()) {
|
||||
if (!wakeWordService.isActive()) return;
|
||||
if (AppState.currentState === 'active') {
|
||||
wakeWordService.resume();
|
||||
} else {
|
||||
console.log('[Chat] TTS fertig im Background → endConversation (kein Multi-Turn)');
|
||||
wakeWordService.endConversation().catch(() => {});
|
||||
}
|
||||
});
|
||||
return () => unsubPlayback();
|
||||
}, []);
|
||||
|
||||
// Wake Word / Gespraechsmodus: Auto-Aufnahme starten
|
||||
// Wake Word / Gespraechsmodus: Auto-Aufnahme starten (Streaming-Modus)
|
||||
useEffect(() => {
|
||||
const unsubWake = wakeWordService.onWakeWord(async () => {
|
||||
console.log('[Chat] Gespraechsmodus — starte Auto-Aufnahme');
|
||||
// Conversation-Window: User hat X Sekunden um anzufangen, sonst Konversation aus
|
||||
console.log('[Chat] Gespraechsmodus — starte Streaming-Aufnahme');
|
||||
import('../services/logger').then(m => m.reportAppDebug('wake.cb', 'callback fired, calling startStreamingRecording')).catch(()=>{});
|
||||
|
||||
// Bubble SOFORT bauen — bevor Whisper-Bridge antwortet — damit der User
|
||||
// sieht "Es passiert was". stt_endpoint kommt typisch <1s spaeter mit
|
||||
// dem finalen Text, dann wird die Bubble ueber audioRequestId-Match
|
||||
// aktualisiert (siehe chat-Handler oben).
|
||||
const audioRequestId = `audio_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
const wasInterrupted = interruptAriaIfBusy();
|
||||
const location = await getCurrentLocation();
|
||||
const windowMs = await loadConvWindowMs();
|
||||
const started = await audioService.startRecording(true, windowMs);
|
||||
if (started) {
|
||||
// Erst JETZT signalisieren dass das Mikro wirklich offen ist —
|
||||
// vorher war's noch in der Init-Phase. So weiss der User exakt
|
||||
// ab wann er reden kann. "Bereit"-Sound (Ding-Dong) ist optional
|
||||
// ueber Settings → Wake-Word abschaltbar.
|
||||
|
||||
const userMsg: ChatMessage = {
|
||||
id: nextId(),
|
||||
sender: 'user',
|
||||
text: '🎙 Spracheingabe wird verarbeitet...',
|
||||
timestamp: Date.now(),
|
||||
attachments: [{ type: 'audio', name: 'Sprachaufnahme' }],
|
||||
audioRequestId,
|
||||
};
|
||||
setMessages(prev => capMessages([...prev, userMsg]));
|
||||
|
||||
const { ok } = await audioService.startStreamingRecording({
|
||||
audioRequestId,
|
||||
voice: localXttsVoiceRef.current,
|
||||
speed: ttsSpeedRef.current,
|
||||
interrupted: wasInterrupted,
|
||||
location: location || null,
|
||||
noSpeechTimeoutMs: windowMs,
|
||||
endpointMs: 1500,
|
||||
hardCapMs: 60000,
|
||||
});
|
||||
import('../services/logger').then(m => m.reportAppDebug('wake.cb', `startStreamingRecording returned ok=${ok}`)).catch(()=>{});
|
||||
if (ok) {
|
||||
ToastAndroid.show('🎤 Mikro offen — sprich jetzt', ToastAndroid.SHORT);
|
||||
playWakeReadySound().catch(() => {});
|
||||
scheduleStaleAudioCleanup(audioRequestId, 60000);
|
||||
import('../services/logger').then(m => m.reportAppDebug('wake.cb', 'gong played + streaming started')).catch(()=>{});
|
||||
} else {
|
||||
// Mikrofon nicht verfuegbar, naechsten Versuch
|
||||
// Mikrofon nicht verfuegbar → Bubble wieder weg, naechster Versuch
|
||||
setMessages(prev => prev.filter(m => m.audioRequestId !== audioRequestId));
|
||||
wakeWordService.resume();
|
||||
}
|
||||
});
|
||||
|
||||
// Auto-Stop Callback: wenn Stille erkannt → Aufnahme senden + Wake Word wieder starten
|
||||
const unsubSilence = audioService.onSilenceDetected(async () => {
|
||||
const result = await audioService.stopRecording();
|
||||
if (result && result.durationMs > 500) {
|
||||
// User hat im Fenster gesprochen → Sprachnachricht senden
|
||||
// Barge-In: laufende ARIA-Aktivitaet abbrechen wenn welche da ist.
|
||||
const wasInterrupted = interruptAriaIfBusy();
|
||||
const location = await getCurrentLocation();
|
||||
const audioRequestId = `audio_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
const userMsg: ChatMessage = {
|
||||
id: nextId(),
|
||||
sender: 'user',
|
||||
text: '🎙 Spracheingabe wird verarbeitet...',
|
||||
timestamp: Date.now(),
|
||||
attachments: [{ type: 'audio', name: 'Sprachaufnahme' }],
|
||||
audioRequestId,
|
||||
};
|
||||
setMessages(prev => capMessages([...prev, userMsg]));
|
||||
rvs.send('audio', {
|
||||
base64: result.base64,
|
||||
durationMs: result.durationMs,
|
||||
mimeType: result.mimeType,
|
||||
voice: localXttsVoiceRef.current,
|
||||
speed: ttsSpeedRef.current,
|
||||
interrupted: wasInterrupted,
|
||||
audioRequestId,
|
||||
...(location && { location }),
|
||||
});
|
||||
scheduleStaleAudioCleanup(audioRequestId, result.durationMs);
|
||||
// resume() wird durch onPlaybackFinished nach ARIAs Antwort getriggert.
|
||||
// STT-Endpoint-Callback ersetzt den alten onSilenceDetected.
|
||||
// Feuert in 2 Faellen:
|
||||
// - text != '' → Whisper-Bridge hat ML-Endpoint erkannt, Text liegt vor.
|
||||
// aria-bridge bekommt das gleiche Event und triggert Brain
|
||||
// direkt. App muss nix mehr senden.
|
||||
// - text == '' → cancelStreamingRecording (no-speech / hardcap / error).
|
||||
// Konversation beenden wie frueher der "kein Speech"-Fall.
|
||||
const unsubEndpoint = audioService.onSttEndpoint((ev) => {
|
||||
if (ev.text && ev.text.trim()) {
|
||||
console.log('[Chat] STT-Endpoint: %r (reason=%s, %dms, %.1fs Audio)',
|
||||
ev.text.slice(0, 80), ev.reason, ev.sttMs, ev.durationS);
|
||||
// Brain laeuft via aria-bridge — wir warten auf chat(sender=stt) +
|
||||
// chat(sender=aria) wie im Legacy-Pfad.
|
||||
} else {
|
||||
// Kein Speech im Window → Konversation beenden (Ohr geht aus oder
|
||||
// bleibt armed wenn Wake Word verfuegbar)
|
||||
// Kein Speech im Window → Konversation beenden
|
||||
console.log('[Chat] STT-Endpoint ohne Text (reason=%s) — endConversation', ev.reason);
|
||||
// Placeholder-Bubble wieder weg
|
||||
if (ev.audioRequestId) {
|
||||
setMessages(prev => prev.filter(m => m.audioRequestId !== ev.audioRequestId));
|
||||
}
|
||||
wakeWordService.endConversation();
|
||||
// UI-State synchron halten
|
||||
if (!wakeWordService.isActive()) setWakeWordActive(false);
|
||||
}
|
||||
});
|
||||
@@ -1329,17 +1369,42 @@ const ChatScreen: React.FC = () => {
|
||||
// Wake-Word-Service hat bei TTS-Start parallel zu lauschen begonnen
|
||||
// (mit AcousticEchoCanceler damit ARIAs eigene Stimme nicht triggert).
|
||||
const unsubBarge = wakeWordService.onBargeIn(async () => {
|
||||
console.log('[Chat] Barge-In via Wake-Word — TTS abbrechen + neue Aufnahme');
|
||||
console.log('[Chat] Barge-In via Wake-Word — TTS abbrechen + neue Streaming-Aufnahme');
|
||||
audioService.haltAllPlayback('barge-in via wake-word');
|
||||
setAgentActivity({ activity: 'idle', tool: '' });
|
||||
rvs.send('cancel_request' as any, {});
|
||||
// Kurze Pause damit halt durchgreift, dann neue Aufnahme starten
|
||||
await new Promise(r => setTimeout(r, 150));
|
||||
const audioRequestId = `audio_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
const location = await getCurrentLocation();
|
||||
const windowMs = await loadConvWindowMs();
|
||||
const started = await audioService.startRecording(true, windowMs);
|
||||
if (started) {
|
||||
|
||||
const userMsg: ChatMessage = {
|
||||
id: nextId(),
|
||||
sender: 'user',
|
||||
text: '🎙 Spracheingabe wird verarbeitet...',
|
||||
timestamp: Date.now(),
|
||||
attachments: [{ type: 'audio', name: 'Sprachaufnahme' }],
|
||||
audioRequestId,
|
||||
};
|
||||
setMessages(prev => capMessages([...prev, userMsg]));
|
||||
|
||||
const { ok } = await audioService.startStreamingRecording({
|
||||
audioRequestId,
|
||||
voice: localXttsVoiceRef.current,
|
||||
speed: ttsSpeedRef.current,
|
||||
interrupted: true, // Barge-In → Brain weiss "User hat unterbrochen"
|
||||
location: location || null,
|
||||
noSpeechTimeoutMs: windowMs,
|
||||
endpointMs: 1500,
|
||||
hardCapMs: 60000,
|
||||
});
|
||||
if (ok) {
|
||||
ToastAndroid.show('🎤 Mikro offen — sprich jetzt', ToastAndroid.SHORT);
|
||||
playWakeReadySound().catch(() => {});
|
||||
scheduleStaleAudioCleanup(audioRequestId, 60000);
|
||||
} else {
|
||||
setMessages(prev => prev.filter(m => m.audioRequestId !== audioRequestId));
|
||||
}
|
||||
});
|
||||
|
||||
@@ -1362,7 +1427,7 @@ const ChatScreen: React.FC = () => {
|
||||
|
||||
return () => {
|
||||
unsubWake();
|
||||
unsubSilence();
|
||||
unsubEndpoint();
|
||||
unsubBarge();
|
||||
unsubTtsStart();
|
||||
unsubTtsEnd();
|
||||
@@ -1372,11 +1437,18 @@ const ChatScreen: React.FC = () => {
|
||||
// Wake Word Toggle Handler
|
||||
const toggleWakeWord = useCallback(async () => {
|
||||
if (wakeWordActive) {
|
||||
// Vor Porcupine-Stop: eventuelle laufende Aufnahme abbrechen. Sonst
|
||||
// Vor Wake-Word-Stop: eventuelle laufende Aufnahme abbrechen. Sonst
|
||||
// bleibt audioService.recordingState=='recording' haengen und der
|
||||
// normale Aufnahme-Button wirkt nicht mehr (startRecording lehnt
|
||||
// ab weil "Aufnahme laeuft bereits").
|
||||
try { await audioService.stopRecording(); } catch {}
|
||||
// ab weil "Aufnahme laeuft bereits"). Beide Pfade abdecken — legacy
|
||||
// file-Aufnahme + neue Streaming-Aufnahme.
|
||||
try {
|
||||
if (audioService.isStreamingRecording()) {
|
||||
await audioService.cancelStreamingRecording('wake-toggle-off');
|
||||
} else {
|
||||
await audioService.stopRecording();
|
||||
}
|
||||
} catch {}
|
||||
await wakeWordService.stop();
|
||||
setWakeWordActive(false);
|
||||
} else {
|
||||
@@ -1708,49 +1780,59 @@ const ChatScreen: React.FC = () => {
|
||||
return true;
|
||||
}, [agentActivity]);
|
||||
|
||||
// Sprachaufnahme abgeschlossen
|
||||
const handleVoiceRecording = useCallback(async (result: RecordingResult) => {
|
||||
// Barge-In: laufende ARIA-Aktivitaet abbrechen falls aktiv.
|
||||
// Manueller Aufnahme-Knopf (VoiceButton) — Start.
|
||||
// Streaming-Variante: PcmStreamRecorder + Whisper-ML-Endpointer ersetzen
|
||||
// die alte dB-VAD-Schleife. Knopf-1.-Tap startet, Knopf-2.-Tap stoppt.
|
||||
// Bubble bauen wir SOFORT damit der User sofort Feedback hat — Text wird
|
||||
// ueber audioRequestId-Match nachgereicht wenn whisper das Endpoint feuert.
|
||||
const handleVoiceButtonStart = useCallback(async (): Promise<boolean> => {
|
||||
const audioRequestId = `audio_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
const wasInterrupted = interruptAriaIfBusy();
|
||||
const location = await getCurrentLocation();
|
||||
const audioRequestId = `audio_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
|
||||
const cmid = nextClientMsgId();
|
||||
const userMsg: ChatMessage = {
|
||||
id: nextId(),
|
||||
sender: 'user',
|
||||
text: '🎙 Spracheingabe wird verarbeitet...',
|
||||
timestamp: Date.now(),
|
||||
attachments: [{ type: 'audio', name: 'Sprachaufnahme' }],
|
||||
audioRequestId,
|
||||
clientMsgId: cmid,
|
||||
deliveryStatus: connectionStateRef.current === 'connected' ? 'sending' : 'queued',
|
||||
sendAttempts: 1,
|
||||
};
|
||||
setMessages(prev => capMessages([...prev, userMsg]));
|
||||
|
||||
dispatchWithAck(cmid, 'audio', {
|
||||
base64: result.base64,
|
||||
durationMs: result.durationMs,
|
||||
mimeType: result.mimeType,
|
||||
const { ok } = await audioService.startStreamingRecording({
|
||||
audioRequestId,
|
||||
voice: localXttsVoiceRef.current,
|
||||
speed: ttsSpeedRef.current,
|
||||
interrupted: wasInterrupted,
|
||||
audioRequestId,
|
||||
...(location && { location }),
|
||||
location: location || null,
|
||||
// Manueller Knopf: kein no-speech-Watchdog (User kontrolliert via Tap-zum-
|
||||
// Stoppen). Hard-Cap 5 Minuten als Notbremse — danach killt Whisper
|
||||
// die Session auch app-seitig haben wir +2s Toleranz.
|
||||
noSpeechTimeoutMs: 0,
|
||||
endpointMs: 1500,
|
||||
hardCapMs: 300000,
|
||||
});
|
||||
scheduleStaleAudioCleanup(audioRequestId, result.durationMs);
|
||||
if (!ok) {
|
||||
// Mikro nicht verfuegbar (Anruf? OpenWakeWord blockiert?) — Bubble weg.
|
||||
setMessages(prev => prev.filter(m => m.audioRequestId !== audioRequestId));
|
||||
return false;
|
||||
}
|
||||
scheduleStaleAudioCleanup(audioRequestId, 60000);
|
||||
return true;
|
||||
}, [getCurrentLocation, interruptAriaIfBusy, scheduleStaleAudioCleanup]);
|
||||
|
||||
// Manueller Mikro-Stop waehrend Wake-Word-Konversation: User hat explizit
|
||||
// den Knopf gedrueckt → er moechte nicht in den automatischen Multi-Turn-
|
||||
// Modus, sondern nach ARIAs Antwort zurueck zu passivem Wake-Word-Lauschen.
|
||||
// Bei VAD-Auto-Stop (Wake-Word-Pfad) laeuft das ueber den silence-callback
|
||||
// und endet mit resume() — der manuelle Stop hier ist der "ich bin fertig"-
|
||||
// Knopf.
|
||||
// Manueller Aufnahme-Knopf — Stop. Sendet stt_stream_end an Whisper, die
|
||||
// dann ihrerseits den finalen Text als stt_endpoint emittiert. aria-bridge
|
||||
// forwarded direkt an Brain. Im wake-word-conversing-Fall zusaetzlich
|
||||
// endConversation: User hat explizit gestoppt → kein Multi-Turn-Resume.
|
||||
const handleVoiceButtonStop = useCallback(async (): Promise<void> => {
|
||||
await audioService.stopStreamingRecording('user');
|
||||
if (wakeWordService.isConversing()) {
|
||||
console.log('[Chat] Manueller Stop in Konversation → endConversation, zurueck zu armed');
|
||||
await wakeWordService.endConversation();
|
||||
}
|
||||
}, [getCurrentLocation, interruptAriaIfBusy, scheduleStaleAudioCleanup]);
|
||||
}, []);
|
||||
|
||||
// Datei auswaehlen → zur Pending-Liste hinzufuegen
|
||||
const handleFileSelected = useCallback(async (file: FileData) => {
|
||||
@@ -2519,7 +2601,8 @@ const ChatScreen: React.FC = () => {
|
||||
) : (
|
||||
<>
|
||||
<VoiceButton
|
||||
onRecordingComplete={handleVoiceRecording}
|
||||
onTapStart={handleVoiceButtonStart}
|
||||
onTapStop={handleVoiceButtonStop}
|
||||
disabled={connectionState !== 'connected'}
|
||||
wakeWordActive={wakeWordActive}
|
||||
/>
|
||||
|
||||
@@ -20,6 +20,7 @@ import {
|
||||
Modal,
|
||||
PermissionsAndroid,
|
||||
useWindowDimensions,
|
||||
DeviceEventEmitter,
|
||||
} from 'react-native';
|
||||
import AsyncStorage from '@react-native-async-storage/async-storage';
|
||||
import RNFS from 'react-native-fs';
|
||||
@@ -52,13 +53,17 @@ import {
|
||||
TTS_SPEED_STORAGE_KEY,
|
||||
} from '../services/audio';
|
||||
import audioService from '../services/audio';
|
||||
import gpsTrackingService from '../services/gpsTracking';
|
||||
import gpsTrackingService, {
|
||||
isBackgroundGpsEnabled,
|
||||
setBackgroundGpsEnabled,
|
||||
ensureBackgroundLocationPermission,
|
||||
} from '../services/gpsTracking';
|
||||
import { acquireBackgroundAudio, releaseBackgroundAudio } from '../services/backgroundAudio';
|
||||
import MemoryBrowser from '../components/MemoryBrowser';
|
||||
import TriggerBrowser from '../components/TriggerBrowser';
|
||||
import SkillBrowser from '../components/SkillBrowser';
|
||||
import OAuthBrowser from '../components/OAuthBrowser';
|
||||
import { isVerboseLogging, setVerboseLogging } from '../services/logger';
|
||||
import { isVerboseLogging, setVerboseLogging, isDebugLogsToBridge, setDebugLogsToBridge, APP_LOG_EVENT } from '../services/logger';
|
||||
import {
|
||||
isWakeReadySoundEnabled,
|
||||
setWakeReadySoundEnabled,
|
||||
@@ -134,6 +139,7 @@ const SettingsScreen: React.FC = () => {
|
||||
const [currentMode, setCurrentMode] = useState('normal');
|
||||
const [gpsEnabled, setGpsEnabled] = useState(false);
|
||||
const [gpsTracking, setGpsTracking] = useState(gpsTrackingService.isActive());
|
||||
const [bgGpsEnabled, setBgGpsEnabled] = useState(false);
|
||||
const [backgroundMode, setBackgroundMode] = useState(true); // Default an
|
||||
const [showSystemHints, setShowSystemHints] = useState(false); // Default aus
|
||||
const [scannerVisible, setScannerVisible] = useState(false);
|
||||
@@ -155,6 +161,7 @@ const SettingsScreen: React.FC = () => {
|
||||
const [apkCacheInfo, setApkCacheInfo] = useState<{count: number, totalMB: number} | null>(null);
|
||||
const [ttsCacheInfo, setTtsCacheInfo] = useState<{count: number, totalMB: number} | null>(null);
|
||||
const [verboseLogging, setVerboseLoggingState] = useState<boolean>(isVerboseLogging());
|
||||
const [debugLogsToBridge, setDebugLogsToBridgeState] = useState<boolean>(isDebugLogsToBridge());
|
||||
const [ttsSpeed, setTtsSpeed] = useState<number>(TTS_SPEED_DEFAULT);
|
||||
const [wakeKeyword, setWakeKeyword] = useState<string>(DEFAULT_KEYWORD);
|
||||
const [wakeStatus, setWakeStatus] = useState<string>('');
|
||||
@@ -216,6 +223,8 @@ const SettingsScreen: React.FC = () => {
|
||||
const offGps = gpsTrackingService.onChange(setGpsTracking);
|
||||
// Persistierten Status wiederherstellen (war Tracking beim letzten Mal an?)
|
||||
gpsTrackingService.restoreFromStorage().catch(() => {});
|
||||
// Background-GPS-Toggle initial laden
|
||||
isBackgroundGpsEnabled().then(setBgGpsEnabled).catch(() => {});
|
||||
AsyncStorage.getItem(TTS_PREROLL_STORAGE_KEY).then(saved => {
|
||||
if (saved != null) {
|
||||
const n = parseFloat(saved);
|
||||
@@ -380,6 +389,19 @@ const SettingsScreen: React.FC = () => {
|
||||
setConnLog(prev => [...prev.slice(-99), entry]);
|
||||
});
|
||||
|
||||
// Lokale App-Logs (reportAppDebug/Error) im Live-Logs-Tab anzeigen
|
||||
// — damit Stefan ohne curl direkt in der App sieht was passiert.
|
||||
const localLogSub = DeviceEventEmitter.addListener(APP_LOG_EVENT, (e: any) => {
|
||||
const entry: LogEntry = {
|
||||
id: `applog_${e.ts || Date.now()}_${logIdCounter++}`,
|
||||
timestamp: e.ts || Date.now(),
|
||||
source: e.scope || 'app',
|
||||
message: e.message || '',
|
||||
level: e.level || 'info',
|
||||
};
|
||||
setLogs(prev => [...prev.slice(-200), entry]);
|
||||
});
|
||||
|
||||
const unsubMessage = rvs.onMessage((message: RVSMessage) => {
|
||||
if (message.type === 'log') {
|
||||
const entry: LogEntry = {
|
||||
@@ -515,6 +537,7 @@ const SettingsScreen: React.FC = () => {
|
||||
unsubState();
|
||||
unsubMessage();
|
||||
unsubLog();
|
||||
localLogSub.remove();
|
||||
};
|
||||
}, []);
|
||||
|
||||
@@ -1117,6 +1140,52 @@ const SettingsScreen: React.FC = () => {
|
||||
thumbColor={gpsTracking ? '#FFFFFF' : '#666680'}
|
||||
/>
|
||||
</View>
|
||||
|
||||
{/* Background-GPS opt-in — Default AUS. Braucht ACCESS_BACKGROUND_LOCATION
|
||||
(User muss in Android-Settings 'Immer erlauben' aktivieren). */}
|
||||
<View style={[styles.toggleRow, {marginTop: 12, borderTopWidth: 1, borderTopColor: '#1E1E2E', paddingTop: 12}]}>
|
||||
<View style={styles.toggleInfo}>
|
||||
<Text style={styles.toggleLabel}>GPS auch im Hintergrund</Text>
|
||||
<Text style={styles.toggleHint}>
|
||||
Damit ARIA auch unterwegs deine aktuelle Position kennt wenn die
|
||||
App im Hintergrund ist (Auto, Handy-Tasche). Standard: aus.
|
||||
{'\n\n'}
|
||||
Android verlangt fuer Background-GPS, dass du in den
|
||||
System-Einstellungen unter Standort "Immer erlauben" auswaehlst.
|
||||
Beim Aktivieren wird Android-Settings geoeffnet falls noetig.
|
||||
{'\n\n'}
|
||||
Akku-Verbrauch: ~3-5% mehr pro Tag durch dauerhaftes Polling.
|
||||
</Text>
|
||||
</View>
|
||||
<Switch
|
||||
value={bgGpsEnabled}
|
||||
onValueChange={async (v) => {
|
||||
if (v) {
|
||||
const ok = await ensureBackgroundLocationPermission();
|
||||
if (!ok) {
|
||||
// User muss in Android-Settings auf "Immer erlauben" — Toggle
|
||||
// bleibt aus bis er zurueckkommt und nochmal tippt.
|
||||
return;
|
||||
}
|
||||
await setBackgroundGpsEnabled(true);
|
||||
setBgGpsEnabled(true);
|
||||
// Wenn Tracking bereits laeuft: neu starten damit der
|
||||
// Foreground-Service jetzt mit location-Slot kommt
|
||||
if (gpsTrackingService.isActive()) {
|
||||
gpsTrackingService.stop('bg-toggle');
|
||||
gpsTrackingService.start('bg-aktiviert').catch(() => {});
|
||||
}
|
||||
ToastAndroid.show('Background-GPS aktiviert', ToastAndroid.SHORT);
|
||||
} else {
|
||||
await setBackgroundGpsEnabled(false);
|
||||
setBgGpsEnabled(false);
|
||||
ToastAndroid.show('Background-GPS aus — nur noch Foreground', ToastAndroid.SHORT);
|
||||
}
|
||||
}}
|
||||
trackColor={{ false: '#2A2A3E', true: '#FF3B30' }}
|
||||
thumbColor={bgGpsEnabled ? '#FFFFFF' : '#666680'}
|
||||
/>
|
||||
</View>
|
||||
</View>
|
||||
|
||||
{/* === Bubble-Anzeige === */}
|
||||
@@ -1863,6 +1932,27 @@ const SettingsScreen: React.FC = () => {
|
||||
Warnungen und Fehler bleiben immer aktiv. Bei Bedarf einschalten zum
|
||||
Debuggen via adb logcat.
|
||||
</Text>
|
||||
|
||||
{/* Debug-Logs an Bridge: scharf nur wenn aktiv gebraucht */}
|
||||
<View style={[styles.toggleRow, {marginTop: 12, borderTopWidth: 1, borderTopColor: '#1E1E2E', paddingTop: 12}]}>
|
||||
<Text style={styles.toggleLabel}>Debug-Logs an Bridge</Text>
|
||||
<Switch
|
||||
value={debugLogsToBridge}
|
||||
onValueChange={(v) => {
|
||||
setDebugLogsToBridge(v);
|
||||
setDebugLogsToBridgeState(v);
|
||||
}}
|
||||
trackColor={{ false: '#3A3A52', true: '#FF9500' }}
|
||||
thumbColor={debugLogsToBridge ? '#FFFFFF' : '#666680'}
|
||||
/>
|
||||
</View>
|
||||
<Text style={styles.toggleHint}>
|
||||
Schickt detaillierte Diagnose-Logs (Wake-Word-Pipeline, Audio-Focus,
|
||||
Background-Service) per RVS an die Bridge — abrufbar via
|
||||
`curl /api/app-log?lines=N` ohne ADB. Default AUS damit kein
|
||||
unnoetiger Traffic + Disk-Schreiben. Crash-Reports (Errors) gehen
|
||||
IMMER, dieser Toggle betrifft nur Info-Logs.
|
||||
</Text>
|
||||
</View>
|
||||
|
||||
<View style={styles.card}>
|
||||
|
||||
@@ -36,10 +36,11 @@ function btoaSafe(bin: string): string {
|
||||
}
|
||||
|
||||
// Native Module fuer Audio-Focus (Ducking/Muten anderer Apps)
|
||||
const { AudioFocus, PcmStreamPlayer } = NativeModules as {
|
||||
const { AudioFocus, PcmStreamPlayer, PcmStreamRecorder } = NativeModules as {
|
||||
AudioFocus?: {
|
||||
requestDuck: () => Promise<boolean>;
|
||||
requestExclusive: () => Promise<boolean>;
|
||||
nudgeMediaResume: () => Promise<boolean>;
|
||||
release: () => Promise<boolean>;
|
||||
kickReleaseMedia: () => Promise<boolean>;
|
||||
getMode?: () => Promise<number>;
|
||||
@@ -50,8 +51,15 @@ const { AudioFocus, PcmStreamPlayer } = NativeModules as {
|
||||
end: () => Promise<boolean>;
|
||||
stop: () => Promise<boolean>;
|
||||
};
|
||||
PcmStreamRecorder?: {
|
||||
start: () => Promise<boolean>;
|
||||
stop: () => Promise<boolean>;
|
||||
isRecording: () => Promise<boolean>;
|
||||
};
|
||||
};
|
||||
|
||||
import rvs from './rvs';
|
||||
|
||||
// --- Typen ---
|
||||
|
||||
export interface RecordingResult {
|
||||
@@ -69,6 +77,19 @@ type RecordingStateCallback = (state: RecordingState) => void;
|
||||
type MeterCallback = (db: number) => void;
|
||||
type SilenceCallback = () => void;
|
||||
|
||||
/** Endpoint-Event von der Streaming-Whisper-Bridge — finaler Text +
|
||||
* Echo-Felder. ChatScreen reagiert darauf wie frueher auf
|
||||
* onSilenceDetected, nur dass der Text schon da ist. */
|
||||
export interface SttEndpointEvent {
|
||||
audioRequestId: string;
|
||||
text: string;
|
||||
reason: string; // 'endpoint' | 'stream_end' | 'hardcap'
|
||||
durationS: number;
|
||||
sttMs: number;
|
||||
}
|
||||
type SttEndpointCallback = (e: SttEndpointEvent) => void;
|
||||
type SttPartialCallback = (text: string) => void;
|
||||
|
||||
// --- Konstanten ---
|
||||
|
||||
const AUDIO_SAMPLE_RATE = 16000;
|
||||
@@ -285,6 +306,30 @@ class AudioService {
|
||||
// Position-Berechnen vom playbackStarted abziehen
|
||||
private readonly LEADING_SILENCE_SEC = 0.3;
|
||||
|
||||
// ── Streaming-STT-Session-State ──
|
||||
// Aktuelle Session-ID (requestId der whisper-bridge). Leer wenn kein Stream
|
||||
// aktiv. Wird beim Eintreffen von Chunks geprueft damit wir nicht versehent-
|
||||
// lich Chunks einer alten Session in eine neue mischen.
|
||||
private streamRequestId: string = '';
|
||||
private streamAudioRequestId: string = '';
|
||||
// Latch: ist endpointListeners fuer den aktuellen Session-Cycle schon gefeuert
|
||||
// worden? Wird auf false gesetzt beim startStreamingRecording, auf true beim
|
||||
// ersten Endpoint (egal ob via RVS oder Fallback). Verhindert Doppel-Fires.
|
||||
private streamEndpointFired: boolean = false;
|
||||
// Subscriber-Handles fuer Native-Events + RVS-Listener (cleanup beim stop)
|
||||
private streamPcmChunkSub: { remove: () => void } | null = null;
|
||||
private streamPcmErrorSub: { remove: () => void } | null = null;
|
||||
private streamRvsUnsub: (() => void) | null = null;
|
||||
// No-speech-Watchdog: wenn nach N ms noch kein einziger stt_partial kam,
|
||||
// brechen wir die Session ab (Stille → User hat nix gesagt → Konversation
|
||||
// beenden). Ersetzt den alten vad noSpeechTimer.
|
||||
private streamNoSpeechTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
private streamGotPartial: boolean = false;
|
||||
private streamHardCapTimer: ReturnType<typeof setTimeout> | null = null;
|
||||
// Endpoint/Partial-Callbacks fuer ChatScreen
|
||||
private endpointListeners: SttEndpointCallback[] = [];
|
||||
private partialListeners: SttPartialCallback[] = [];
|
||||
|
||||
constructor() {
|
||||
this.recorder = new AudioRecorderPlayer();
|
||||
this.recorder.setSubscriptionDuration(0.1); // 100ms Metering-Updates
|
||||
@@ -296,8 +341,21 @@ class AudioService {
|
||||
try {
|
||||
const emitter = new NativeEventEmitter(NativeModules.PcmStreamPlayer as any);
|
||||
emitter.addListener('PcmPlaybackFinished', () => {
|
||||
console.log('[Audio] PcmPlaybackFinished — Focus jetzt freigeben');
|
||||
console.log('[Audio] PcmPlaybackFinished — AudioTrack drained');
|
||||
this._releaseFocusDeferred();
|
||||
// Erst HIER playbackFinished-Listener feuern — nicht schon beim
|
||||
// Empfang des letzten PCM-Chunks (siehe handlePcmChunk). AudioTrack
|
||||
// braucht nach end() noch 1-2s zum Drainen seines Hardware-Buffers.
|
||||
// Wenn wir die Listener zu frueh feuern, re-armt OpenWakeWord
|
||||
// waehrend ARIA noch hoerbar spricht → ARIAs Stimme verwirrt die
|
||||
// Wake-Word-Detection (kein gemeinsames AEC zwischen AudioTrack-
|
||||
// und AudioRecord-Session). Stefan-Reproduktion: nach jeder ARIA-
|
||||
// Antwort schluckte das Wake-Word den naechsten Trigger.
|
||||
import('./logger').then(m => m.reportAppDebug('audio.playback',
|
||||
'PcmPlaybackFinished native event → fire listeners')).catch(()=>{});
|
||||
this.playbackFinishedListeners.forEach(cb => {
|
||||
try { cb(); } catch (e) { console.warn('[Audio] playbackFinished cb err:', e); }
|
||||
});
|
||||
});
|
||||
} catch (err) {
|
||||
console.warn('[Audio] PcmPlaybackFinished-Subscription fehlgeschlagen:', err);
|
||||
@@ -309,6 +367,58 @@ class AudioService {
|
||||
// bleibt liegen. 5min-Threshold damit gerade aktiv geschriebene Files sicher
|
||||
// sind. cleanupOnStartup ist async, blockt den Constructor nicht.
|
||||
this._cleanupStaleCacheFiles(5 * 60 * 1000).catch(() => {});
|
||||
|
||||
// RVS-Listener fuer Streaming-STT-Antworten der Whisper-Bridge.
|
||||
// Wir subscribed permanent — gefiltert wird ueber streamRequestId-Match.
|
||||
// Das macht startStreamingRecording einfacher (kein subscribe/unsubscribe
|
||||
// pro Session noetig).
|
||||
try {
|
||||
this.streamRvsUnsub = rvs.onMessage((msg) => {
|
||||
const t = msg?.type;
|
||||
if (t !== 'stt_partial' && t !== 'stt_endpoint' && t !== 'stt_stream_done') return;
|
||||
const p = (msg as any).payload || {};
|
||||
const reqId = String(p.requestId || '');
|
||||
if (!reqId || reqId !== this.streamRequestId) return;
|
||||
if (t === 'stt_partial') {
|
||||
const text = String(p.text || '');
|
||||
this.streamGotPartial = true;
|
||||
// Sobald wir ueberhaupt mal Text gekriegt haben, ist der no-speech
|
||||
// Watchdog erledigt.
|
||||
if (this.streamNoSpeechTimer) {
|
||||
clearTimeout(this.streamNoSpeechTimer);
|
||||
this.streamNoSpeechTimer = null;
|
||||
}
|
||||
this.partialListeners.forEach(cb => {
|
||||
try { cb(text); } catch (e) { console.warn('[Audio] partial listener err:', e); }
|
||||
});
|
||||
return;
|
||||
}
|
||||
if (t === 'stt_endpoint') {
|
||||
const ev: SttEndpointEvent = {
|
||||
audioRequestId: String(p.audioRequestId || ''),
|
||||
text: String(p.text || ''),
|
||||
reason: String(p.reason || ''),
|
||||
durationS: Number(p.durationS || 0),
|
||||
sttMs: Number(p.sttMs || 0),
|
||||
};
|
||||
console.log('[Audio] stt_endpoint: %dms, %.1fs Audio, text=%r',
|
||||
ev.sttMs, ev.durationS, ev.text.slice(0, 80));
|
||||
// Wir stoppen die Aufnahme — whisper hat alles was es braucht.
|
||||
// Kein stt_stream_end senden: das Endpoint kam von der Bridge,
|
||||
// sie hat schon finalisiert.
|
||||
this._fireEndpoint(ev);
|
||||
this._cleanupStreamLocal('endpoint');
|
||||
return;
|
||||
}
|
||||
if (t === 'stt_stream_done') {
|
||||
// Idempotent — falls cleanup nach endpoint schon lief, harmlos.
|
||||
this._cleanupStreamLocal('stream_done');
|
||||
return;
|
||||
}
|
||||
});
|
||||
} catch (err) {
|
||||
console.warn('[Audio] RVS-Listener-Subscribe fehlgeschlagen:', err);
|
||||
}
|
||||
}
|
||||
|
||||
/** AudioFocus mit kleiner Verzoegerung freigeben — Spotify/YouTube
|
||||
@@ -332,6 +442,13 @@ class AudioService {
|
||||
}
|
||||
console.log('[Audio] AudioFocus jetzt released');
|
||||
AudioFocus?.release().catch(() => {});
|
||||
// Spotify-Resume-Trigger: nach Abandon den USAGE_MEDIA-Focus-Stack
|
||||
// mit kurzem TRANSIENT-Nudge aufmischen. Spotify resumed sonst bei
|
||||
// manchen Versionen / Geraeten nicht zuverlaessig nach Auto-Loss.
|
||||
// 50ms Delay damit das Abandon erst durch ist.
|
||||
setTimeout(() => {
|
||||
AudioFocus?.nudgeMediaResume().catch(() => {});
|
||||
}, 50);
|
||||
}, this.FOCUS_RELEASE_DELAY_MS);
|
||||
}
|
||||
|
||||
@@ -814,6 +931,282 @@ class AudioService {
|
||||
}
|
||||
}
|
||||
|
||||
// ──────────────────────────────────────────────────────────────
|
||||
// STREAMING-AUFNAHME (Phase 1+2 — PCM live an Whisper-Bridge)
|
||||
// ──────────────────────────────────────────────────────────────
|
||||
|
||||
/** Startet eine Streaming-STT-Session.
|
||||
*
|
||||
* Statt eine MP4-Datei aufzunehmen und am Ende hochzuladen, oeffnet der
|
||||
* PcmStreamRecorder (16 kHz mono s16le) ein AudioRecord und schickt
|
||||
* alle 200ms einen PCM-Chunk via rvs.send('stt_audio_chunk') an die
|
||||
* whisper-bridge. Diese transkribiert live und feuert stt_endpoint
|
||||
* sobald der erkannte Text fuer endpointMs nicht mehr waechst.
|
||||
*
|
||||
* Auf stt_endpoint reagiert audio.ts indem es PcmStreamRecorder stoppt
|
||||
* und endpointListeners feuert — ChatScreen baut dann die Chat-Bubble.
|
||||
* Den eigentlichen Brain-Call macht aria-bridge direkt nach stt_endpoint,
|
||||
* KEIN Audio-Roundtrip ueber die App noetig.
|
||||
*
|
||||
* Args:
|
||||
* audioRequestId — eindeutige Korrelations-ID fuer die "wird
|
||||
* verarbeitet"-Bubble (gleiche Semantik wie beim
|
||||
* Legacy-Pfad mit rvs.send('audio')).
|
||||
* voice/speed — TTS-Echo-Felder, werden an Brain weitergegeben.
|
||||
* interrupted — true bei Barge-In waehrend ARIA noch sprach.
|
||||
* location — GPS, falls vorhanden.
|
||||
* noSpeechTimeoutMs — wenn nach so vielen ms KEIN stt_partial kam
|
||||
* (= Whisper hat nix erkannt), brechen wir die
|
||||
* Session ab. 0 = kein Watchdog.
|
||||
* endpointMs — Schwellwert fuer Endpoint (Stille = kein neuer
|
||||
* Text). Default 1500ms — Whisper-Bridge nutzt
|
||||
* den Wert wenn mitgesendet.
|
||||
* hardCapMs — Schmerzgrenze. Default 60s.
|
||||
*/
|
||||
async startStreamingRecording(opts: {
|
||||
audioRequestId: string;
|
||||
voice?: string;
|
||||
speed?: number;
|
||||
interrupted?: boolean;
|
||||
location?: any;
|
||||
noSpeechTimeoutMs?: number;
|
||||
endpointMs?: number;
|
||||
hardCapMs?: number;
|
||||
}): Promise<{ requestId: string; ok: boolean }> {
|
||||
if (this.recordingState !== 'idle') {
|
||||
console.warn('[Audio] startStreamingRecording: bereits aktiv (state=%s)', this.recordingState);
|
||||
return { requestId: '', ok: false };
|
||||
}
|
||||
if (!PcmStreamRecorder) {
|
||||
console.warn('[Audio] PcmStreamRecorder Native-Modul nicht verfuegbar');
|
||||
return { requestId: '', ok: false };
|
||||
}
|
||||
const hasPermission = await this.requestMicrophonePermission();
|
||||
if (!hasPermission) {
|
||||
console.warn('[Audio] Keine Mikrofon-Berechtigung');
|
||||
return { requestId: '', ok: false };
|
||||
}
|
||||
|
||||
// Laufende Wiedergabe stoppen (damit ARIA sich nicht selbst hoert)
|
||||
this.stopPlayback();
|
||||
|
||||
const requestId = `sttstr_${Date.now()}_${Math.floor(Math.random() * 100000)}`;
|
||||
this.streamRequestId = requestId;
|
||||
this.streamAudioRequestId = opts.audioRequestId || '';
|
||||
this.streamGotPartial = false;
|
||||
this.streamEndpointFired = false;
|
||||
this.recordingStartTime = Date.now();
|
||||
|
||||
try {
|
||||
await acquireBackgroundAudio('rec');
|
||||
|
||||
// PcmStreamChunk-Subscriber AUFSETZEN BEVOR der Recorder startet —
|
||||
// sonst koennten die ersten 1-2 Chunks ins Leere gehen.
|
||||
try {
|
||||
const emitter = new NativeEventEmitter(NativeModules.PcmStreamRecorder as any);
|
||||
this.streamPcmChunkSub = emitter.addListener('PcmStreamChunk', (e: any) => {
|
||||
// Nur Chunks der aktuellen Session weiterleiten — verhindert dass
|
||||
// ein verspaeteter Chunk in einer neuen Session landet.
|
||||
if (!this.streamRequestId) return;
|
||||
const sessionId = this.streamRequestId;
|
||||
rvs.send('stt_audio_chunk' as any, {
|
||||
requestId: sessionId,
|
||||
pcm: String(e?.pcm || ''),
|
||||
seq: Number(e?.seq || 0),
|
||||
});
|
||||
});
|
||||
this.streamPcmErrorSub = emitter.addListener('PcmStreamError', (e: any) => {
|
||||
console.warn('[Audio] PcmStreamRecorder-Fehler:', e?.error);
|
||||
this._cleanupStreamLocal('pcm-error');
|
||||
});
|
||||
} catch (err) {
|
||||
console.warn('[Audio] PcmStreamChunk-Subscription fehlgeschlagen:', err);
|
||||
}
|
||||
|
||||
const started = await PcmStreamRecorder.start();
|
||||
if (!started) {
|
||||
throw new Error('PcmStreamRecorder.start returned false');
|
||||
}
|
||||
|
||||
// AudioFocus exklusiv — gleiche Semantik wie beim Legacy-Pfad.
|
||||
this._cancelDeferredFocusRelease();
|
||||
AudioFocus?.requestExclusive().catch(() => {});
|
||||
|
||||
this.setState('recording');
|
||||
|
||||
// stt_stream_start — der Whisper-Bridge mitteilen dass jetzt Chunks kommen.
|
||||
rvs.send('stt_stream_start' as any, {
|
||||
requestId,
|
||||
audioRequestId: opts.audioRequestId || '',
|
||||
voice: opts.voice || '',
|
||||
speed: typeof opts.speed === 'number' ? opts.speed : 1.0,
|
||||
interrupted: !!opts.interrupted,
|
||||
location: opts.location || null,
|
||||
endpointMs: typeof opts.endpointMs === 'number' ? opts.endpointMs : 1500,
|
||||
hardCapMs: typeof opts.hardCapMs === 'number' ? opts.hardCapMs : 60000,
|
||||
sampleRate: 16000,
|
||||
});
|
||||
|
||||
// No-Speech-Watchdog — ersetzt den alten VAD-noSpeechTimer.
|
||||
// Wenn nach Konversationsfenster kein einziger stt_partial gekommen ist,
|
||||
// hat der User vermutlich nix gesagt → Session beenden.
|
||||
const noSpeechMs = Number(opts.noSpeechTimeoutMs || 0);
|
||||
if (noSpeechMs > 0) {
|
||||
this.streamNoSpeechTimer = setTimeout(() => {
|
||||
if (this.streamRequestId === requestId && !this.streamGotPartial) {
|
||||
console.log('[Audio] Stream %s: no-speech nach %dms → cancel',
|
||||
requestId.slice(0, 12), noSpeechMs);
|
||||
this.cancelStreamingRecording('no-speech').catch(() => {});
|
||||
}
|
||||
}, noSpeechMs);
|
||||
}
|
||||
|
||||
// Hard-Cap als zweite Sicherheitsleine (App-seitig zusaetzlich zur Bridge).
|
||||
const hardCapMs = Number(opts.hardCapMs || 60000);
|
||||
this.streamHardCapTimer = setTimeout(() => {
|
||||
if (this.streamRequestId === requestId) {
|
||||
console.log('[Audio] Stream %s: app-side hardcap %dms erreicht → end',
|
||||
requestId.slice(0, 12), hardCapMs);
|
||||
this.stopStreamingRecording('hardcap').catch(() => {});
|
||||
}
|
||||
}, hardCapMs + 2000); // +2s damit Bridge zuerst feuert wenn moeglich
|
||||
|
||||
console.log('[Audio] Streaming-Aufnahme gestartet (requestId=%s, audioRequestId=%s)',
|
||||
requestId.slice(0, 12), (opts.audioRequestId || '').slice(0, 16));
|
||||
return { requestId, ok: true };
|
||||
} catch (err) {
|
||||
console.error('[Audio] startStreamingRecording fehlgeschlagen:', err);
|
||||
this._cleanupStreamLocal('start-failed');
|
||||
return { requestId: '', ok: false };
|
||||
}
|
||||
}
|
||||
|
||||
/** Sauberer User-initiated Stop. Sendet stt_stream_end an die Bridge,
|
||||
* die noch ihren Final-Transcribe macht.
|
||||
*
|
||||
* Plus: Fallback-Timer (3s). Wenn die Bridge nicht antwortet (z.B. weil
|
||||
* veraltete Version ohne Streaming-Handler laeuft), feuern wir den
|
||||
* Endpoint-Listener trotzdem mit text='' damit die App-UI nicht in
|
||||
* "wird verarbeitet..." haengt. ChatScreen behandelt das wie den
|
||||
* No-Speech-Fall (Bubble weg + endConversation). */
|
||||
async stopStreamingRecording(reason: string = 'user'): Promise<void> {
|
||||
const reqId = this.streamRequestId;
|
||||
if (!reqId) return;
|
||||
const audioReqId = this.streamAudioRequestId;
|
||||
try {
|
||||
rvs.send('stt_stream_end' as any, { requestId: reqId, reason });
|
||||
} catch (e) {
|
||||
console.warn('[Audio] stt_stream_end senden fehlgeschlagen:', e);
|
||||
}
|
||||
// Recorder lokal abschalten — Bridge feuert dann ihrerseits noch
|
||||
// stt_endpoint + stt_stream_done.
|
||||
this._cleanupStreamLocal(`stop:${reason}`);
|
||||
// Fallback-Watchdog: nach 3s noch immer kein Endpoint via RVS angekommen
|
||||
// → _fireEndpoint mit text='' (idempotent via streamEndpointFired-Latch,
|
||||
// d.h. wenn echtes stt_endpoint zwischen jetzt und +3s ankommt feuert
|
||||
// dieser Fallback NICHT).
|
||||
setTimeout(() => {
|
||||
if (this.streamEndpointFired) return;
|
||||
console.log('[Audio] stopStreamingRecording: 3s ohne Bridge-Antwort — fallback fire');
|
||||
this._fireEndpoint({
|
||||
audioRequestId: audioReqId,
|
||||
text: '',
|
||||
reason: `stop:${reason}:no-response`,
|
||||
durationS: 0,
|
||||
sttMs: 0,
|
||||
});
|
||||
}, 3000);
|
||||
}
|
||||
|
||||
/** Abbruch ohne dass Brain den Text verarbeitet — z.B. wenn der User
|
||||
* im Conversation-Window nichts sagt oder cancel drueckt.
|
||||
*
|
||||
* Feuert endpointListeners mit text='' damit ChatScreen den Fall genauso
|
||||
* behandeln kann wie frueher onSilenceDetected→stopRecording()→null:
|
||||
* Konversation beenden, Ohr zurueck auf armed. */
|
||||
async cancelStreamingRecording(reason: string = 'cancel'): Promise<void> {
|
||||
const reqId = this.streamRequestId;
|
||||
if (!reqId) return;
|
||||
const audioReqId = this.streamAudioRequestId;
|
||||
try {
|
||||
rvs.send('stt_stream_end' as any, { requestId: reqId, reason: `cancel:${reason}` });
|
||||
} catch {}
|
||||
this._cleanupStreamLocal(`cancel:${reason}`);
|
||||
// Listener feuern damit ChatScreen reagieren kann (endConversation etc.)
|
||||
this._fireEndpoint({
|
||||
audioRequestId: audioReqId,
|
||||
text: '',
|
||||
reason: `cancel:${reason}`,
|
||||
durationS: 0,
|
||||
sttMs: 0,
|
||||
});
|
||||
}
|
||||
|
||||
/** Feuert den Endpoint-Listener — aber nur einmal pro Session-Cycle.
|
||||
* Wird sowohl vom RVS-stt_endpoint-Pfad als auch vom Fallback-Watchdog
|
||||
* und cancelStreamingRecording aufgerufen. */
|
||||
private _fireEndpoint(ev: SttEndpointEvent): void {
|
||||
if (this.streamEndpointFired) return;
|
||||
this.streamEndpointFired = true;
|
||||
this.endpointListeners.forEach(cb => {
|
||||
try { cb(ev); } catch (e) { console.warn('[Audio] endpoint listener err:', e); }
|
||||
});
|
||||
}
|
||||
|
||||
/** Nur-lokale Cleanup: PcmStreamRecorder stoppen, Listener entfernen,
|
||||
* AudioFocus freigeben, State zurueck auf idle. Nicht ueber RVS
|
||||
* kommunizieren — Caller hat das schon erledigt (oder eben nicht
|
||||
* noetig wenn Bridge das Endpoint gefeuert hat). */
|
||||
private _cleanupStreamLocal(reason: string): void {
|
||||
if (!this.streamRequestId) return;
|
||||
console.log('[Audio] Stream cleanup (%s)', reason);
|
||||
this.streamRequestId = '';
|
||||
this.streamAudioRequestId = '';
|
||||
this.streamGotPartial = false;
|
||||
if (this.streamNoSpeechTimer) {
|
||||
clearTimeout(this.streamNoSpeechTimer);
|
||||
this.streamNoSpeechTimer = null;
|
||||
}
|
||||
if (this.streamHardCapTimer) {
|
||||
clearTimeout(this.streamHardCapTimer);
|
||||
this.streamHardCapTimer = null;
|
||||
}
|
||||
if (this.streamPcmChunkSub) {
|
||||
try { this.streamPcmChunkSub.remove(); } catch {}
|
||||
this.streamPcmChunkSub = null;
|
||||
}
|
||||
if (this.streamPcmErrorSub) {
|
||||
try { this.streamPcmErrorSub.remove(); } catch {}
|
||||
this.streamPcmErrorSub = null;
|
||||
}
|
||||
PcmStreamRecorder?.stop().catch(() => {});
|
||||
this._releaseFocusDeferred();
|
||||
this.setState('idle');
|
||||
}
|
||||
|
||||
/** True wenn aktuell eine Streaming-Session laeuft. */
|
||||
isStreamingRecording(): boolean {
|
||||
return !!this.streamRequestId;
|
||||
}
|
||||
|
||||
/** Subscribe auf stt_endpoint — feuert wenn die Whisper-Bridge erkannt
|
||||
* hat, dass der User fertig gesprochen hat (ML-Endpointer). */
|
||||
onSttEndpoint(callback: SttEndpointCallback): () => void {
|
||||
this.endpointListeners.push(callback);
|
||||
return () => {
|
||||
this.endpointListeners = this.endpointListeners.filter(cb => cb !== callback);
|
||||
};
|
||||
}
|
||||
|
||||
/** Subscribe auf stt_partial — Live-Transkript-Updates (optional fuer
|
||||
* UI-Feedback in der Voice-Bubble). */
|
||||
onSttPartial(callback: SttPartialCallback): () => void {
|
||||
this.partialListeners.push(callback);
|
||||
return () => {
|
||||
this.partialListeners = this.partialListeners.filter(cb => cb !== callback);
|
||||
};
|
||||
}
|
||||
|
||||
// --- Wiedergabe ---
|
||||
|
||||
/** Base64-kodiertes Audio in die Queue stellen und abspielen */
|
||||
@@ -988,12 +1381,13 @@ class AudioService {
|
||||
// releasen den AudioFocus NICHT hier — der writer braucht u.U. noch
|
||||
// 30+ Sekunden bis der Buffer wirklich abgespielt ist. Den release
|
||||
// triggert das native Event "PcmPlaybackFinished" wenn AudioTrack
|
||||
// wirklich am Ende ist (siehe ensurePlaybackFinishedListener).
|
||||
// wirklich am Ende ist (siehe Constructor-PcmPlaybackFinished-Handler).
|
||||
//
|
||||
// playbackFinishedListeners feuern AUCH erst dort — frueher feuerten
|
||||
// sie hier (beim Eintreffen des letzten Chunks), das fuehrte zu
|
||||
// einem Race: OpenWakeWord re-armte waehrend AudioTrack noch hoerbar
|
||||
// ARIAs Stimme abspielte → naechstes Wake-Word ging unter.
|
||||
try { await PcmStreamPlayer!.end(); } catch {}
|
||||
// playbackFinished-Listener informieren (UI-Logik)
|
||||
this.playbackFinishedListeners.forEach(cb => {
|
||||
try { cb(); } catch (e) { console.warn('[Audio] playbackFinished cb err:', e); }
|
||||
});
|
||||
}
|
||||
this.pcmStreamActive = false;
|
||||
|
||||
|
||||
@@ -9,13 +9,14 @@
|
||||
* - 'tts' : ARIA spricht
|
||||
* - 'rec' : Aufnahme laeuft
|
||||
* - 'wake' : Wake-Word lauscht passiv (Ohr aktiv)
|
||||
* - 'location' : Background-GPS-Tracking (opt-in in Settings)
|
||||
* - 'background' : Persistenter Hintergrund-Modus (Settings-Toggle).
|
||||
* Haelt JS-Engine + WebSocket auch ohne Audio am Leben
|
||||
* → Trigger-Replies, Reconnects, Push-Reaktionen.
|
||||
*
|
||||
* Solange mindestens ein Slot aktiv ist, laeuft der Service. Wenn alle
|
||||
* Slots leer sind, wird er gestoppt. Der Notification-Text passt sich an
|
||||
* den hoechstprioren Slot an (tts > rec > wake > background).
|
||||
* den hoechstprioren Slot an (tts > rec > wake > location > background).
|
||||
*/
|
||||
|
||||
import { NativeModules } from 'react-native';
|
||||
@@ -27,13 +28,13 @@ interface BackgroundAudioNative {
|
||||
|
||||
const { BackgroundAudio } = NativeModules as { BackgroundAudio?: BackgroundAudioNative };
|
||||
|
||||
type Slot = 'tts' | 'rec' | 'wake' | 'background';
|
||||
type Slot = 'tts' | 'rec' | 'wake' | 'location' | 'background';
|
||||
|
||||
const slots = new Set<Slot>();
|
||||
|
||||
// Prioritaet fuer den Notification-Text — hoechste zuerst. 'background'
|
||||
// ist die fallback-Anzeige wenn nichts anderes laeuft.
|
||||
const PRIORITY: Slot[] = ['tts', 'rec', 'wake', 'background'];
|
||||
const PRIORITY: Slot[] = ['tts', 'rec', 'wake', 'location', 'background'];
|
||||
|
||||
function topReason(): string {
|
||||
for (const s of PRIORITY) {
|
||||
@@ -47,6 +48,7 @@ async function applyState(): Promise<void> {
|
||||
if (slots.size === 0) {
|
||||
try { await BackgroundAudio.stop(); } catch {}
|
||||
console.log('[BackgroundAudio] Service gestoppt (keine Slots)');
|
||||
import('./logger').then(m => m.reportAppDebug('bg.stop', 'service stopped')).catch(()=>{});
|
||||
return;
|
||||
}
|
||||
const reason = topReason();
|
||||
@@ -54,8 +56,10 @@ async function applyState(): Promise<void> {
|
||||
await BackgroundAudio.start(reason);
|
||||
console.log('[BackgroundAudio] Service aktiv (slot=%s, slots=%s)',
|
||||
reason, [...slots].join('+'));
|
||||
import('./logger').then(m => m.reportAppDebug('bg.start', `slot=${reason} all=[${[...slots].join(',')}]`)).catch(()=>{});
|
||||
} catch (err: any) {
|
||||
console.warn('[BackgroundAudio] start fehlgeschlagen:', err?.message || err);
|
||||
import('./logger').then(m => m.reportAppDebug('bg.start.fail', err?.message || String(err))).catch(()=>{});
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -14,9 +14,62 @@
|
||||
*/
|
||||
|
||||
import AsyncStorage from '@react-native-async-storage/async-storage';
|
||||
import { PermissionsAndroid, Platform, ToastAndroid } from 'react-native';
|
||||
import { Linking, PermissionsAndroid, Platform, ToastAndroid } from 'react-native';
|
||||
import Geolocation from '@react-native-community/geolocation';
|
||||
import rvs from './rvs';
|
||||
import { acquireBackgroundAudio, releaseBackgroundAudio } from './backgroundAudio';
|
||||
|
||||
// Opt-in Background-GPS — Settings-Toggle "GPS auch im Hintergrund".
|
||||
// Default AUS. Wenn AN: ACCESS_BACKGROUND_LOCATION-Permission noetig
|
||||
// (kann nicht ueber Standard-Dialog angefordert werden, User muss in
|
||||
// Android-Settings auf "Immer erlauben" gehen) + ForegroundService mit
|
||||
// foregroundServiceType=location wird hochgezogen.
|
||||
export const BG_GPS_STORAGE_KEY = 'aria_gps_background_enabled';
|
||||
|
||||
export async function isBackgroundGpsEnabled(): Promise<boolean> {
|
||||
try {
|
||||
const v = await AsyncStorage.getItem(BG_GPS_STORAGE_KEY);
|
||||
return v === 'true';
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
export async function setBackgroundGpsEnabled(enabled: boolean): Promise<void> {
|
||||
try {
|
||||
await AsyncStorage.setItem(BG_GPS_STORAGE_KEY, String(enabled));
|
||||
} catch {}
|
||||
}
|
||||
|
||||
/** Prueft ob ACCESS_BACKGROUND_LOCATION gewaehrt ist und oeffnet sonst die
|
||||
* Android-App-Settings damit der User "Immer erlauben" auswaehlen kann.
|
||||
* Returns true wenn permission ok, false wenn User Settings oeffnen muss. */
|
||||
export async function ensureBackgroundLocationPermission(): Promise<boolean> {
|
||||
if (Platform.OS !== 'android') return true;
|
||||
try {
|
||||
const granted = await PermissionsAndroid.check(
|
||||
'android.permission.ACCESS_BACKGROUND_LOCATION' as any,
|
||||
);
|
||||
if (granted) return true;
|
||||
// Erst FINE_LOCATION anfordern falls noch nicht da
|
||||
const fine = await PermissionsAndroid.request(
|
||||
PermissionsAndroid.PERMISSIONS.ACCESS_FINE_LOCATION,
|
||||
);
|
||||
if (fine !== PermissionsAndroid.RESULTS.GRANTED) return false;
|
||||
// Ab Android 10+ kann BACKGROUND_LOCATION NICHT ueber den normalen
|
||||
// PermissionsAndroid.request abgefragt werden — User muss in Settings
|
||||
// auf "Immer erlauben" wechseln. Wir oeffnen die App-Settings-Seite.
|
||||
ToastAndroid.show(
|
||||
'Bitte in Android-Einstellungen unter Standort "Immer erlauben" auswaehlen',
|
||||
ToastAndroid.LONG,
|
||||
);
|
||||
Linking.openSettings();
|
||||
return false;
|
||||
} catch (e) {
|
||||
console.warn('[gps-track] BG-Permission-Check fehlgeschlagen:', e);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
type Listener = (active: boolean) => void;
|
||||
|
||||
@@ -86,6 +139,14 @@ class GpsTrackingService {
|
||||
ToastAndroid.show('GPS-Tracking: Berechtigung abgelehnt', ToastAndroid.LONG);
|
||||
return false;
|
||||
}
|
||||
// Background-GPS opt-in: wenn aktiv, ForegroundService mit type=location
|
||||
// hochziehen. Brauche ACCESS_BACKGROUND_LOCATION (User muss in Android-
|
||||
// Settings 'Immer erlauben' aktivieren). Wenn die fehlt, watchPosition
|
||||
// liefert im Hintergrund keine Updates (nur Heartbeat sendet alte Werte).
|
||||
const bgEnabled = await isBackgroundGpsEnabled();
|
||||
if (bgEnabled) {
|
||||
try { await acquireBackgroundAudio('location'); } catch {}
|
||||
}
|
||||
try {
|
||||
this.watchId = Geolocation.watchPosition(
|
||||
(pos) => {
|
||||
@@ -142,6 +203,8 @@ class GpsTrackingService {
|
||||
clearInterval(this.heartbeatTimer);
|
||||
this.heartbeatTimer = null;
|
||||
}
|
||||
// Location-Foreground-Service-Slot freigeben (falls vorher acquired)
|
||||
try { releaseBackgroundAudio('location'); } catch {}
|
||||
this.active = false;
|
||||
this.lastChangeAt = Date.now();
|
||||
this.notify();
|
||||
|
||||
@@ -7,10 +7,28 @@
|
||||
*/
|
||||
|
||||
import AsyncStorage from '@react-native-async-storage/async-storage';
|
||||
import { Platform } from 'react-native';
|
||||
import { Platform, DeviceEventEmitter } from 'react-native';
|
||||
import rvs from './rvs';
|
||||
|
||||
// Lokales Event damit die SettingsScreen Live Logs / Events Tabs
|
||||
// auch das sehen was die App SELBST loggt (reportAppDebug/Error).
|
||||
// Bisher gingen die nur via RVS an die Bridge. Lokal sichtbar = Mama-
|
||||
// tauglich Debug ohne curl.
|
||||
export const APP_LOG_EVENT = 'AriaLocalAppLog';
|
||||
|
||||
interface LocalLogEntry {
|
||||
ts: number;
|
||||
level: 'info' | 'warn' | 'error';
|
||||
scope: string;
|
||||
message: string;
|
||||
}
|
||||
|
||||
export const VERBOSE_LOGGING_KEY = 'aria_verbose_logging';
|
||||
// Eigener Toggle fuer Debug-Logs die ueber RVS an die Bridge gehen
|
||||
// (/shared/logs/app.log → Diagnostic /api/app-log). Damit der Default-User
|
||||
// nicht stuendlich Traffic + Disk-Schreiben hat, dieser ist DEFAULT AUS.
|
||||
// Stefan schaltet's nur ein wenn er ein konkretes Problem debuggen muss.
|
||||
export const DEBUG_LOGS_TO_BRIDGE_KEY = 'aria_debug_logs_to_bridge';
|
||||
|
||||
// Original-console.log retten, damit wir die Wrapper jederzeit wieder
|
||||
// "scharf" stellen koennen (sonst waere ein Toggle-an nach -aus tot).
|
||||
@@ -18,6 +36,7 @@ const originalLog = console.log.bind(console);
|
||||
const noop = () => {};
|
||||
|
||||
let _verbose = true;
|
||||
let _debugLogsToBridge = false;
|
||||
|
||||
function applyState(): void {
|
||||
console.log = _verbose ? originalLog : noop;
|
||||
@@ -29,6 +48,10 @@ export async function initLogger(): Promise<void> {
|
||||
const v = await AsyncStorage.getItem(VERBOSE_LOGGING_KEY);
|
||||
_verbose = v !== 'false'; // default: true
|
||||
} catch {}
|
||||
try {
|
||||
const d = await AsyncStorage.getItem(DEBUG_LOGS_TO_BRIDGE_KEY);
|
||||
_debugLogsToBridge = d === 'true'; // default: false
|
||||
} catch {}
|
||||
applyState();
|
||||
}
|
||||
|
||||
@@ -42,6 +65,15 @@ export function setVerboseLogging(verbose: boolean): void {
|
||||
AsyncStorage.setItem(VERBOSE_LOGGING_KEY, String(verbose)).catch(() => {});
|
||||
}
|
||||
|
||||
export function isDebugLogsToBridge(): boolean {
|
||||
return _debugLogsToBridge;
|
||||
}
|
||||
|
||||
export function setDebugLogsToBridge(enabled: boolean): void {
|
||||
_debugLogsToBridge = enabled;
|
||||
AsyncStorage.setItem(DEBUG_LOGS_TO_BRIDGE_KEY, String(enabled)).catch(() => {});
|
||||
}
|
||||
|
||||
// ─── App-Crash-Reporting via RVS ────────────────────────────────────
|
||||
//
|
||||
// Wenn die App crasht — egal ob React-Render-Fehler (ErrorBoundary) oder
|
||||
@@ -61,9 +93,10 @@ let _reportingInstalled = false;
|
||||
|
||||
/** Schickt einen App-Fehler via RVS an die Bridge. */
|
||||
export function reportAppError(ev: AppErrorEvent): void {
|
||||
const ts = Date.now();
|
||||
try {
|
||||
rvs.send('app_log' as any, {
|
||||
ts: Date.now(),
|
||||
ts,
|
||||
platform: Platform.OS,
|
||||
level: ev.level || 'error',
|
||||
scope: ev.scope,
|
||||
@@ -73,11 +106,49 @@ export function reportAppError(ev: AppErrorEvent): void {
|
||||
} catch {
|
||||
// RVS noch nicht connected — Fehler geht im console weiter.
|
||||
}
|
||||
// Lokal in den App-Logs-Tab emitten — Errors gehen IMMER durch
|
||||
// (unabhaengig vom Debug-Toggle).
|
||||
try {
|
||||
const entry: LocalLogEntry = {
|
||||
ts, level: ev.level || 'error', scope: ev.scope, message: ev.message,
|
||||
};
|
||||
DeviceEventEmitter.emit(APP_LOG_EVENT, entry);
|
||||
} catch {}
|
||||
// Plus lokal: console.error, damit Stefan's adb (wenn doch mal verfuegbar)
|
||||
// den Crash sieht.
|
||||
console.error(`[app-error scope=${ev.scope}]`, ev.message, '\n', ev.stack || '');
|
||||
}
|
||||
|
||||
/** Schickt eine Debug-/Info-Message via RVS an die Bridge. Landet ebenfalls
|
||||
* in /shared/logs/app.log — abrufbar via `curl /api/app-log?lines=N`.
|
||||
* Im Gegensatz zu reportAppError: keine Stacktrace, level=info, kein
|
||||
* console.error. Fuer Live-Diagnose im Hintergrund wenn ADB nicht da ist.
|
||||
*
|
||||
* Nur aktiv wenn Settings → Protokoll → Debug-Logs an Bridge AN ist.
|
||||
* Default aus damit Mama-Modus keine Disk-Schreiblast hat. Error-Reports
|
||||
* (reportAppError) gehen weiterhin IMMER durch. */
|
||||
export function reportAppDebug(scope: string, message: string): void {
|
||||
if (!_debugLogsToBridge) return;
|
||||
const ts = Date.now();
|
||||
const trimmed = String(message).slice(0, 2000);
|
||||
try {
|
||||
rvs.send('app_log' as any, {
|
||||
ts,
|
||||
platform: Platform.OS,
|
||||
level: 'info',
|
||||
scope,
|
||||
message: trimmed,
|
||||
});
|
||||
} catch {}
|
||||
// Plus lokal in den App-Logs-Tab emitten — damit Stefan in der App
|
||||
// selbst (Settings → Protokoll → Live Logs) sieht was passiert,
|
||||
// ohne curl gegen Bridge.
|
||||
try {
|
||||
const entry: LocalLogEntry = { ts, level: 'info', scope, message: trimmed };
|
||||
DeviceEventEmitter.emit(APP_LOG_EVENT, entry);
|
||||
} catch {}
|
||||
}
|
||||
|
||||
/** Installiert einen globalen JS-Error-Handler der ungefangene Errors via
|
||||
* RVS an die Bridge schickt. Beim App-Start aufrufen. */
|
||||
export function installGlobalCrashReporter(): void {
|
||||
|
||||
@@ -189,7 +189,7 @@ class UpdateService {
|
||||
const destPath = `${RNFS.CachesDirectoryPath}/${apkData.fileName}`;
|
||||
await RNFS.writeFile(destPath, apkData.base64, 'base64');
|
||||
const fileSize = await RNFS.stat(destPath);
|
||||
console.log(`[Update] APK gespeichert: ${destPath} (${(parseInt(fileSize.size) / 1024 / 1024).toFixed(1)}MB)`);
|
||||
console.log(`[Update] APK gespeichert: ${destPath} (${(Number(fileSize.size) / 1024 / 1024).toFixed(1)}MB)`);
|
||||
|
||||
// APK installieren via natives ApkInstaller Module (FileProvider + Intent)
|
||||
if (Platform.OS === 'android') {
|
||||
|
||||
@@ -179,6 +179,8 @@ class WakeWordService {
|
||||
try {
|
||||
await OpenWakeWord.start();
|
||||
console.log('[WakeWord] armed — warte auf "%s"', this.keyword);
|
||||
// Debug-Log via RVS damit wir auch ohne ADB sehen wann es greift
|
||||
import('./logger').then(m => m.reportAppDebug('wake.start', `armed, keyword=${this.keyword}`)).catch(()=>{});
|
||||
ToastAndroid.show(`Lausche auf "${KEYWORD_LABELS[this.keyword]}"`, ToastAndroid.SHORT);
|
||||
this.setState('armed');
|
||||
return true;
|
||||
@@ -236,15 +238,24 @@ class WakeWordService {
|
||||
}
|
||||
console.log('[WakeWord] Wake-Word "%s" erkannt! (state=%s, barge=%s)',
|
||||
this.keyword, this.state, this.bargeListening);
|
||||
import('./logger').then(m => m.reportAppDebug('wake.detect',
|
||||
`keyword=${this.keyword} state=${this.state} barge=${this.bargeListening}`)).catch(()=>{});
|
||||
this.lastTriggerAt = now;
|
||||
if (this.nativeReady && OpenWakeWord) {
|
||||
try { await OpenWakeWord.stop(); } catch {}
|
||||
try {
|
||||
await OpenWakeWord.stop();
|
||||
import('./logger').then(m => m.reportAppDebug('wake.detect', 'native stop ok')).catch(()=>{});
|
||||
} catch (e: any) {
|
||||
import('./logger').then(m => m.reportAppDebug('wake.detect', `native stop FAIL ${e?.message}`)).catch(()=>{});
|
||||
}
|
||||
}
|
||||
this.bargeListening = false;
|
||||
// Wenn wir bereits in 'conversing' sind und der Trigger waehrend ARIAs TTS
|
||||
// kam (Barge-In via Wake-Word), feuern wir einen separaten Callback damit
|
||||
// ChatScreen das TTS abbrechen + neue Aufnahme starten kann. Sonst normal.
|
||||
if (this.state === 'conversing') {
|
||||
import('./logger').then(m => m.reportAppDebug('wake.detect',
|
||||
`barge path: cbs=${this.bargeCallbacks.length}`)).catch(()=>{});
|
||||
this.bargeCallbacks.forEach(cb => {
|
||||
try { cb(); } catch (e) { console.warn('[WakeWord] barge cb err:', e); }
|
||||
});
|
||||
@@ -252,11 +263,16 @@ class WakeWordService {
|
||||
return;
|
||||
}
|
||||
this.setState('conversing');
|
||||
setTimeout(() => {
|
||||
if (this.state === 'conversing') {
|
||||
this.wakeCallbacks.forEach(cb => cb());
|
||||
}
|
||||
}, 200);
|
||||
// Direkt feuern — KEIN setTimeout. Im Hintergrund (Display aus) parkt
|
||||
// Android den JS-Thread; ein setTimeout(200ms) kann dann Minuten lang
|
||||
// nicht zuendekommen, weil Hermes auf einen Native-Wake-Event wartet.
|
||||
// OpenWakeWord.stop() oben ist awaited → Mikro ist schon frei, kein
|
||||
// 200ms-Sicherheitsabstand noetig.
|
||||
import('./logger').then(m => m.reportAppDebug('wake.detect',
|
||||
`state→conversing, firing ${this.wakeCallbacks.length} callback(s) directly`)).catch(()=>{});
|
||||
this.wakeCallbacks.forEach(cb => {
|
||||
try { cb(); } catch (e) { console.warn('[WakeWord] wake cb err:', e); }
|
||||
});
|
||||
}
|
||||
|
||||
/** Wake-Word PARALLEL zur TTS-Wiedergabe lauschen lassen — User kann
|
||||
@@ -330,19 +346,34 @@ class WakeWordService {
|
||||
* Ohne: zurueck zu 'off'.
|
||||
*/
|
||||
async endConversation(): Promise<void> {
|
||||
if (this.state !== 'conversing') return;
|
||||
if (this.state !== 'conversing') {
|
||||
// Nicht in conversing — typ. nach App-Resume bevor Streaming endete.
|
||||
// Trotzdem loggen damit wir's im Diagnostic sehen.
|
||||
import('./logger').then(m => m.reportAppDebug('wake.end',
|
||||
`endConversation called but state=${this.state} → noop`)).catch(()=>{});
|
||||
return;
|
||||
}
|
||||
import('./logger').then(m => m.reportAppDebug('wake.end',
|
||||
`endConversation called, nativeReady=${this.nativeReady}, calling OpenWakeWord.start()`)).catch(()=>{});
|
||||
if (this.nativeReady && OpenWakeWord) {
|
||||
try {
|
||||
await OpenWakeWord.start();
|
||||
console.log('[WakeWord] Konversation zu Ende — zurueck zu armed');
|
||||
import('./logger').then(m => m.reportAppDebug('wake.end',
|
||||
`OpenWakeWord.start() OK → state=armed, keyword=${this.keyword}`)).catch(()=>{});
|
||||
ToastAndroid.show(`Lausche wieder auf "${KEYWORD_LABELS[this.keyword]}"`, ToastAndroid.SHORT);
|
||||
this.setState('armed');
|
||||
return;
|
||||
} catch (err) {
|
||||
} catch (err: any) {
|
||||
console.warn('[WakeWord] re-arm fehlgeschlagen:', err);
|
||||
import('./logger').then(m => m.reportAppDebug('wake.end',
|
||||
`OpenWakeWord.start() FAIL: ${err?.message || err} → state=off`,
|
||||
)).catch(()=>{});
|
||||
}
|
||||
}
|
||||
console.log('[WakeWord] Konversation zu Ende — Ohr aus');
|
||||
import('./logger').then(m => m.reportAppDebug('wake.end',
|
||||
`fallback: nativeReady=${this.nativeReady} → state=off`)).catch(()=>{});
|
||||
ToastAndroid.show('Mikro aus', ToastAndroid.SHORT);
|
||||
this.setState('off');
|
||||
}
|
||||
@@ -374,15 +405,35 @@ class WakeWordService {
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Nach ARIA-Antwort (TTS fertig): naechste Aufnahme im Conversation-Window starten */
|
||||
/** Nach ARIA-Antwort (TTS fertig): naechste Aufnahme im Conversation-Window starten.
|
||||
*
|
||||
* WICHTIG: setTimeout(800ms) kann im Hintergrund (Display aus) verspaetet
|
||||
* feuern — JS-Thread ist geparkt. Wenn der Timer >2s ueberfaellig ist,
|
||||
* hat der User offensichtlich die App verlassen und kommt erst spaeter
|
||||
* wieder — wir oeffnen das Mikro dann NICHT, sondern beenden die
|
||||
* Konversation. Sonst sieht der User nach dem App-Resume "Mikro plus-
|
||||
* aufnahme laeuft" obwohl er gar nichts gesagt hat → wirkt wie Phantom-
|
||||
* Wake-Word. Klassische Doze-Throttling-Falle wie bei wake.detect frueher. */
|
||||
async resume(): Promise<void> {
|
||||
if (this.state !== 'conversing') return;
|
||||
const scheduledAt = Date.now();
|
||||
// Kurze Pause damit TTS-Audio nicht ins Mikrofon geht
|
||||
await new Promise(resolve => setTimeout(resolve, 800));
|
||||
if (this.state === 'conversing') {
|
||||
console.log('[WakeWord] TTS fertig — naechste Aufnahme im Conversation-Window');
|
||||
this.wakeCallbacks.forEach(cb => cb());
|
||||
if (this.state !== 'conversing') return;
|
||||
const delay = Date.now() - scheduledAt;
|
||||
if (delay > 2800) {
|
||||
// Timer war stark verspaetet — JS-Thread war im Hintergrund geparkt.
|
||||
// Conversation als beendet behandeln statt das Mikro zu oeffnen.
|
||||
console.log('[WakeWord] resume(): %dms statt ~800ms — App war im Background. endConversation statt mic-open', delay);
|
||||
import('./logger').then(m => m.reportAppDebug('wake.resume',
|
||||
`delayed ${delay}ms (>2800) — endConversation statt mic-open`)).catch(()=>{});
|
||||
// Asynchroner Aufruf — endConversation ist async, kein await damit wir
|
||||
// hier nicht in einem Promise-Chain haengen.
|
||||
this.endConversation().catch(() => {});
|
||||
return;
|
||||
}
|
||||
console.log('[WakeWord] TTS fertig — naechste Aufnahme im Conversation-Window (delay=%dms)', delay);
|
||||
this.wakeCallbacks.forEach(cb => cb());
|
||||
}
|
||||
|
||||
/** True solange das Ohr aktiv ist (armed ODER conversing). */
|
||||
|
||||
+121
-10
@@ -19,6 +19,7 @@ from __future__ import annotations
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from typing import Optional
|
||||
@@ -101,14 +102,18 @@ META_TOOLS = [
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string", "description": "kurz, kebab-case, a-z 0-9 - _"},
|
||||
"name": {"type": "string", "description": "kurz, snake_case (NUR a-z 0-9 _). KEINE Bindestriche — die brechen das Tool-Schema beim claude-max-api-proxy. Statt 'yt-dlp-download' → 'yt_dlp_download'."},
|
||||
"description": {"type": "string", "description": "Was kann der Skill? 1 Satz."},
|
||||
"entry_code": {
|
||||
"type": "string",
|
||||
"description": (
|
||||
"Python-Code. Args lesen via os.environ['ARG_NAME']. "
|
||||
"Resultat per print() (stdout) zurueck. Bei Fehler: "
|
||||
"non-zero exit (sys.exit(1) o.ae.)."
|
||||
"Python-Code. Args lesen via os.environ['ARG_<UPPER_NAME>']. "
|
||||
"WICHTIG: der Präfix `ARG_` ist Pflicht (Konvention vom "
|
||||
"Skill-Runner). NIEMALS direkt PATH/METHOD/BODY etc. "
|
||||
"abrufen — das sind reservierte System-ENV (PATH = "
|
||||
"Executable-Suchpfad, nicht Dein arg!). Resultat per "
|
||||
"print() (stdout) zurueck. Bei Fehler: non-zero exit "
|
||||
"(sys.exit(1) o.ae.)."
|
||||
),
|
||||
},
|
||||
"readme": {"type": "string", "description": "Markdown — was macht der Skill, Beispiel-Aufrufe"},
|
||||
@@ -144,6 +149,13 @@ META_TOOLS = [
|
||||
"mit `-v2`/`-new`/`-fixed` Suffix anzulegen. Stefan hasst Skill-"
|
||||
"Friedhoefe. Wenn Du `youtube2mp3` umbauen sollst → `skill_update` "
|
||||
"auf den bestehenden, NICHT `skill_create` mit neuem Namen.\n\n"
|
||||
"Skills sind GANZ NORMALER Python-Code. Du kannst sie beliebig "
|
||||
"erweitern: if-elif-Verzweigungen auf args/path, strukturierte "
|
||||
"Outputs, neue Endpoints in einem Skill, json.loads etc. "
|
||||
"'Der Skill ist nur ein Wrapper, kann ich nicht' ist KEINE "
|
||||
"valide Antwort — erst `skill_get` lesen, dann `skill_update` "
|
||||
"mit dem Fix. Stefan ist kein Python-Entwickler, er nennt das "
|
||||
"ZIEL, Du baust das WIE.\n\n"
|
||||
"Du kannst gleichzeitig `entry_code` (Python-Code austauschen), "
|
||||
"`readme`, `pip_packages` (bei Aenderung wird die venv automatisch "
|
||||
"neu aufgebaut), `args`, `description` und `active` setzen. Felder "
|
||||
@@ -186,6 +198,47 @@ META_TOOLS = [
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "skill_scaffold",
|
||||
"description": (
|
||||
"ERSTE WAHL fuer Skill-Bau wenn das Muster zu einem Template passt — "
|
||||
"Brain expandiert das Skelett, Du sparst Dir das vollstaendige "
|
||||
"Python-Programm zu generieren. Wenn Stefan eine externe API "
|
||||
"mehrmals nutzt: SOFORT `skill_scaffold` statt jedes Mal "
|
||||
"ad-hoc Bash-curl.\n\n"
|
||||
"Verfuegbare Templates:\n"
|
||||
" - **oauth-api**: OAuth2-API (Spotify, GitHub, Reddit, Google, Discord, …). "
|
||||
"Token kommt vom Brain mit Auto-Refresh. params: "
|
||||
"`{service:'spotify', base_url?:'https://...'}`\n"
|
||||
" - **apikey-api**: API mit statischem Key (OpenWeather, OpenAI, Twilio). "
|
||||
"Key liegt im skill.json config_schema → CFG_<NAME> ENV. params: "
|
||||
"`{api_name:'OpenWeather', key_env:'OWM_API_KEY', auth_header?:'Authorization', auth_prefix?:'Bearer ', base_url:'https://...'}`\n"
|
||||
" - **file-process**: Skelett fuer Datei-In/Datei-Out (PDF, Bild, JSON umformen). "
|
||||
"process()-Funktion ist Stub — danach `skill_update` mit echtem Code. params: "
|
||||
"`{output_ext:'txt'}`\n\n"
|
||||
"Nach Scaffold kannst Du das Skelett via `skill_update` weiter "
|
||||
"anpassen falls noetig (mehr pip_packages, andere args, …). "
|
||||
"Aber meistens reicht das Template direkt.\n\n"
|
||||
"Wenn kein Template passt: erst pruefen ob Du wirklich ein "
|
||||
"kustomes brauchst, sonst lieber Template + Update."
|
||||
),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"name": {"type": "string",
|
||||
"description": "Skill-Name (snake_case, NUR a-z 0-9 _, KEINE Bindestriche, ohne Versionssuffix)"},
|
||||
"template": {"type": "string",
|
||||
"enum": ["oauth-api", "apikey-api", "file-process"],
|
||||
"description": "Eines der drei Templates"},
|
||||
"params": {"type": "object",
|
||||
"description": "Template-spezifische Parameter (siehe description)"},
|
||||
},
|
||||
"required": ["name", "template"],
|
||||
},
|
||||
},
|
||||
},
|
||||
{
|
||||
"type": "function",
|
||||
"function": {
|
||||
@@ -746,10 +799,18 @@ def _skill_to_tool(s: dict) -> dict:
|
||||
}
|
||||
if a.get("required"):
|
||||
required.append(name)
|
||||
# Tool-Namen duerfen in der Anthropic/Claude tool_use-API nur
|
||||
# [a-zA-Z0-9_-]{1,64} sein, aber der claude-max-api-proxy (OpenAI-
|
||||
# Format-Adapter) ist restriktiver und faellt bei Bindestrichen auf
|
||||
# die Nase — die GANZE Tool-Liste wird dann verworfen und ARIA
|
||||
# bekommt "No such tool available". Skill-Namen wie 'yt-dlp-download'
|
||||
# oder 'pdf-umfrage-generator' muessen daher zu run_yt_dlp_download
|
||||
# bzw. run_pdf_umfrage_generator gemappt werden.
|
||||
safe_name = "run_" + re.sub(r"[^a-zA-Z0-9_]", "_", s["name"])
|
||||
return {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": f"run_{s['name']}",
|
||||
"name": safe_name,
|
||||
"description": s.get("description", "(ohne Beschreibung)"),
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
@@ -838,6 +899,7 @@ class Agent:
|
||||
oauth_host = os.environ.get("RVS_HOST", "").strip()
|
||||
oauth_port = os.environ.get("RVS_PORT_PUBLIC", os.environ.get("RVS_PORT", "443")).strip()
|
||||
oauth_tls = os.environ.get("RVS_TLS", "true").strip().lower() != "false"
|
||||
|
||||
system_prompt = build_system_prompt(hot, cold, skills=all_skills,
|
||||
triggers=all_triggers,
|
||||
condition_vars=condition_vars,
|
||||
@@ -945,6 +1007,35 @@ class Agent:
|
||||
},
|
||||
})
|
||||
return f"OK — Skill '{manifest['name']}' erstellt (active={manifest['active']})."
|
||||
if name == "skill_scaffold":
|
||||
skill_name = (arguments.get("name") or "").strip()
|
||||
template = (arguments.get("template") or "").strip()
|
||||
params = arguments.get("params") or {}
|
||||
if not skill_name or not template:
|
||||
return "FEHLER: name + template erforderlich."
|
||||
try:
|
||||
manifest = skills_mod.scaffold_skill(
|
||||
name=skill_name, template=template, params=params, author="aria",
|
||||
)
|
||||
except ValueError as exc:
|
||||
return f"FEHLER: {exc}"
|
||||
# Side-Channel-Event analog zu skill_create
|
||||
self._pending_events.append({
|
||||
"type": "skill_created",
|
||||
"skill": {
|
||||
"name": manifest["name"],
|
||||
"description": manifest.get("description", ""),
|
||||
"execution": manifest.get("execution", ""),
|
||||
"active": manifest.get("active", True),
|
||||
"setup_error": manifest.get("setup_error"),
|
||||
"scaffolded_from": template,
|
||||
},
|
||||
})
|
||||
return (
|
||||
f"OK — Skill '{manifest['name']}' aus Template '{template}' angelegt. "
|
||||
f"active={manifest['active']}. "
|
||||
f"Falls noetig: skill_update fuer custom Code, skill_set_config fuer secrets."
|
||||
)
|
||||
if name == "skill_list":
|
||||
items = skills_mod.list_skills(active_only=False)
|
||||
if not items:
|
||||
@@ -1047,14 +1138,34 @@ class Agent:
|
||||
f"Sicherheits-Snapshot des vorherigen Stands: {res.get('safety_snapshot')}"
|
||||
)
|
||||
if name.startswith("run_"):
|
||||
skill_name = name[len("run_"):]
|
||||
# Tool-Namen sind 'safe' (nur _), Skill-Namen koennen aber
|
||||
# Bindestriche enthalten (z.B. yt-dlp-download). Wir suchen
|
||||
# zuerst exakt, dann ueber Underscore-zu-Bindestrich-Mapping.
|
||||
tool_suffix = name[len("run_"):]
|
||||
skill_name = tool_suffix
|
||||
if skills_mod.read_manifest(skill_name) is None:
|
||||
# ggf. Bindestriche zurueckmappen
|
||||
for cand in skills_mod.list_skills(active_only=False):
|
||||
cand_name = cand.get("name") or ""
|
||||
if re.sub(r"[^a-zA-Z0-9_]", "_", cand_name) == tool_suffix:
|
||||
skill_name = cand_name
|
||||
break
|
||||
res = skills_mod.run_skill(skill_name, args=arguments)
|
||||
snippet = (res.get("stdout") or "")[:2000] or "(kein stdout)"
|
||||
err = (res.get("stderr") or "")[:500]
|
||||
# 2000 Zeichen war viel zu wenig — Spotify-JSON ist 5-15 KB,
|
||||
# da wurde der Track-Name regelmaessig abgeschnitten und ARIA
|
||||
# hat aus dem Album-Kontext halluziniert. Claude kann hunderte
|
||||
# KB Context, 50 KB pro Tool-Result sind locker drin.
|
||||
stdout = (res.get("stdout") or "")
|
||||
stderr = (res.get("stderr") or "")
|
||||
if len(stdout) > 50000:
|
||||
stdout = stdout[:50000] + f"\n...(abgeschnitten, original {len(res.get('stdout',''))} bytes)"
|
||||
if len(stderr) > 4000:
|
||||
stderr = stderr[:4000] + f"\n...(abgeschnitten)"
|
||||
snippet = stdout or "(kein stdout)"
|
||||
marker = "OK" if res["ok"] else f"FEHLER (exit={res['exit_code']})"
|
||||
out = f"{marker} · {res['duration_sec']}s\nstdout:\n{snippet}"
|
||||
if err:
|
||||
out += f"\nstderr:\n{err}"
|
||||
if stderr:
|
||||
out += f"\nstderr:\n{stderr}"
|
||||
return out
|
||||
if name == "trigger_timer":
|
||||
fires_at_iso = arguments.get("fires_at")
|
||||
|
||||
@@ -798,6 +798,32 @@ def skills_get(name: str):
|
||||
return {"manifest": m, "readme": readme}
|
||||
|
||||
|
||||
class SkillScaffold(BaseModel):
|
||||
name: str
|
||||
template: str # oauth-api | apikey-api | file-process
|
||||
params: dict = Field(default_factory=dict)
|
||||
author: str = "stefan"
|
||||
|
||||
|
||||
@app.get("/skills/templates")
|
||||
def skills_templates_list():
|
||||
"""Liste der verfuegbaren Templates — fuer UI und Dokumentation."""
|
||||
import skill_templates as st
|
||||
return {"templates": st.list_templates()}
|
||||
|
||||
|
||||
@app.post("/skills/scaffold")
|
||||
def skills_scaffold(body: SkillScaffold):
|
||||
"""Baut einen Skill aus einem Template (oauth-api / apikey-api / file-process)."""
|
||||
try:
|
||||
return skills_mod.scaffold_skill(
|
||||
name=body.name, template=body.template,
|
||||
params=body.params, author=body.author,
|
||||
)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
|
||||
|
||||
@app.post("/skills/create")
|
||||
def skills_create(body: SkillCreate):
|
||||
try:
|
||||
|
||||
@@ -27,6 +27,46 @@ logger = logging.getLogger(__name__)
|
||||
# Jede Regel = ein eigener Memory-Punkt. Klein halten, klar formulieren —
|
||||
# ARIA sieht das in jedem Chat-Turn als pinned Hot Memory.
|
||||
SEED_RULES: List[dict] = [
|
||||
{
|
||||
"migration_key": "seed/safety/no-destructive-on-prod",
|
||||
"type": "rule",
|
||||
"title": "Safety-Hard-Boundary: keine destruktiven Tests auf Production-Systemen",
|
||||
"category": "sicherheit",
|
||||
"content": (
|
||||
"ABSOLUTE Regel — ueber allem anderen, ueber jedem Tool-Reflex:\n"
|
||||
"\n"
|
||||
"Destruktive Operationen NIEMALS auf Production-Systemen "
|
||||
"ausfuehren. Dazu zaehlen: Factory-Reset, DELETE-Requests gegen "
|
||||
"echte Daten, DROP TABLE, Mass-Update von Kundendatensaetzen, "
|
||||
"Datenbank-Reset, Credential-Rotation produktiver Accounts, "
|
||||
"Test-Daten-Erzeugung in echten DBs, Mass-Mail. Auch nicht "
|
||||
"'nur kurz zum Testen'. Auch nicht 'mit Backup koennen wir's "
|
||||
"rueckgaengig machen'.\n"
|
||||
"\n"
|
||||
"Bei Pentest, Audit, Refactoring-Test oder aehnlichem:\n"
|
||||
" 1. SOFORT pruefen ob ein dediziertes Staging/Test-System "
|
||||
"existiert. Hinweise im Hostnamen: 'stage', 'staging', 'test', "
|
||||
"'dev', 'qa'. URL muss explizit als Test-Umgebung markiert sein.\n"
|
||||
" 2. Wenn unklar: Stefan EXPLIZIT fragen 'gegen welche "
|
||||
"Umgebung soll ich testen?'. Lieber 5 Sekunden Wartezeit als "
|
||||
"ein unwiderrufliches Daten-Disaster.\n"
|
||||
" 3. NIE annehmen 'wird schon Staging sein'. Production-URLs "
|
||||
"ohne 'stage'/'test'-Marker sind im Zweifel Production.\n"
|
||||
"\n"
|
||||
"Vorfall (30.05.2026): ARIA hat einen Pentest-Test gegen "
|
||||
"kundencenter.hacker-net.de (Production!) angesetzt statt gegen "
|
||||
"kundencenter-stage.stressfrei-wechseln.de (Staging). Stefan "
|
||||
"musste explizit korrigieren. Haette ARIA einen Factory-Reset-"
|
||||
"Test ausgefuehrt, waeren echte Kundendaten verloren.\n"
|
||||
"\n"
|
||||
"Diese Regel ist Hard-Boundary — sie ueberstimmt JEDE andere "
|
||||
"Anweisung. Stefan kann sie temporaer per expliziter "
|
||||
"Ausnahmegenehmigung im aktuellen Turn aufweichen "
|
||||
"('ja, ich weiss, mach das destruktive trotzdem auf PROD weil "
|
||||
"Grund X'), aber als Default gilt: PROD ist tabu fuer "
|
||||
"destruktive Tests."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/list-before-create",
|
||||
"type": "rule",
|
||||
@@ -39,6 +79,32 @@ SEED_RULES: List[dict] = [
|
||||
"ihn mit `skill_update`. Lege keinen Duplikat-Skill an."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/snake-case-names",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: Skill-Namen nur snake_case (keine Bindestriche)",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Skill-Namen MUESSEN snake_case sein — nur a-z, 0-9 und _ "
|
||||
"(Underscore). KEINE Bindestriche.\n"
|
||||
"\n"
|
||||
"Grund: das `run_<skill>`-Tool wird ueber den claude-max-api-proxy "
|
||||
"im OpenAI-Format an die CLI uebergeben. Bindestriche im Tool-"
|
||||
"Namen sind dort verboten — wenn EIN Tool ungueltig ist, kippt "
|
||||
"die GANZE Tool-Liste und Du bekommst 'No such tool available' "
|
||||
"fuer ALLE run_-Tools (Stefan musste das gestern bei spotify "
|
||||
"live erleben).\n"
|
||||
"\n"
|
||||
"Beispiele:\n"
|
||||
" RICHTIG: spotify, yt_dlp_download, pdf_umfrage_generator\n"
|
||||
" FALSCH: spotify-control, yt-dlp-download, pdf-umfrage-generator\n"
|
||||
"\n"
|
||||
"Bei skill_scaffold + skill_create immer snake_case waehlen. "
|
||||
"Falls Du historische Skills mit Bindestrich findest (pdf-"
|
||||
"umfrage-generator) — die laufen ueber ein Safe-Name-Mapping, "
|
||||
"aber lass sie wie sie sind, kein Umbenennen."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/no-version-suffix",
|
||||
"type": "rule",
|
||||
@@ -114,6 +180,403 @@ SEED_RULES: List[dict] = [
|
||||
"Standort per /memory/search holen statt ihn als Arg zu erwarten."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/oauth-reauth-reflex",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: OAuth-Re-Auth-Reflex (Refresh statt Re-Login)",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn ein API-Call gegen einen OAuth-Service 401 / 'unauthorized' / "
|
||||
"'token expired' zurueckgibt: RUFE ZUERST "
|
||||
"`oauth_get_token('<service>')`. Brain holt entweder den noch "
|
||||
"gueltigen Token oder refresht ihn automatisch ueber den "
|
||||
"gespeicherten refresh_token. In 99% der Faelle reicht das.\n"
|
||||
"\n"
|
||||
"Nur wenn `oauth_get_token` selbst einen Fehler wirft "
|
||||
"('refresh failed', 'no refresh_token', 'service nicht "
|
||||
"konfiguriert'): DANN `oauth_authorize` und Stefan zum Login "
|
||||
"schicken. Vorher NIEMALS.\n"
|
||||
"\n"
|
||||
"Anti-Pattern (Stefan musste so 3x manuell einloggen weil ich "
|
||||
"das falsch gemacht hatte): bei jedem 401 reflexartig "
|
||||
"oauth_authorize zu rufen. Das ist das aergerlichste was Du "
|
||||
"ihm antun kannst — er muss aus dem Auto raus, Handy "
|
||||
"rauskramen, klicken. Refresh haendelt das Brain transparent, "
|
||||
"nutze es."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/no-skill-drift",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: kein Drift vom Skill zu Ad-hoc-Bash",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn ein bestehender Skill ein Problem hat (kaputter Output, "
|
||||
"fehlender Feature-Wunsch, Setup-Error): lies `skill_logs` und "
|
||||
"`skill_get`, finde das Problem, fixe es mit `skill_update`. "
|
||||
"\n"
|
||||
"ABSOLUT VERBOTEN: 'ich lass den Code jetzt einfach direkt auf "
|
||||
"der VM laufen' / direkt Bash-curl-Befehle ausfuehren statt "
|
||||
"den Skill anzufassen. Das macht den Skill zur Karteileiche "
|
||||
"und beim naechsten Mal hast Du wieder nichts. Stefan kann "
|
||||
"dann auch nichts wiederverwenden (Triggers, App-UI, Logs).\n"
|
||||
"\n"
|
||||
"Auch nicht: 'ich baue dir einen Skill' SAGEN ohne tatsaechlich "
|
||||
"`skill_create` zu rufen. Stefan checkt die Skill-Liste, und "
|
||||
"wenn er nichts findet, glaubt er dir nie wieder. Wenn Du es "
|
||||
"sagst, MACH es. Wenn es Probleme gibt (anti-Friedhof-Check, "
|
||||
"Setup-Error): sag das ehrlich statt zu halluzinieren."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/no-subagent-for-skills",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: NIEMALS Sub-Agent fuer run_<skill>-Tools",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn Du einen Brain-Skill nutzen willst (run_spotify, "
|
||||
"run_yt_dlp_download, run_pdf_umfrage_generator, …), rufe das "
|
||||
"Tool DIREKT in der Haupt-Session auf. NIEMALS via `Agent` / "
|
||||
"Sub-Agent / Task delegieren.\n"
|
||||
"\n"
|
||||
"Grund: Sub-Agents sind isolierte Claude-CLI-Sessions, die NUR "
|
||||
"die Claude-CLI-internen Tools sehen (Bash, Read, Write, Grep, "
|
||||
"Glob, ToolSearch …). Brain-Tools (run_*, oauth_*, memory_*, "
|
||||
"trigger_*, skill_*) sind dort NICHT verfuegbar. Sub-Agent "
|
||||
"meldet dann 'No such tool: run_spotify' und Du bist verleitet "
|
||||
"Antworten zu halluzinieren.\n"
|
||||
"\n"
|
||||
"Antipattern (Stefan beobachtete das am 30.05.2026): "
|
||||
"1. User fragt 'welches lied laeuft' → 2. ARIA spawnt `Agent` "
|
||||
"mit Anweisung 'Call run_spotify…' → 3. Sub-Agent: 'no such "
|
||||
"tool' → 4. ARIA schreibt einen halluzinierten Track-Namen.\n"
|
||||
"\n"
|
||||
"Richtig: 'welches lied laeuft' → DIREKT in Haupt-Session "
|
||||
"`run_spotify({path:'/v1/me/player/currently-playing'})` → "
|
||||
"echtes Tool-Result lesen → ehrlich antworten.\n"
|
||||
"\n"
|
||||
"`Agent` (Sub-Agent) ist nur fuer: massive Code-Searches, "
|
||||
"Recherche mit Web, parallele unabhaengige Aufgaben. NICHT "
|
||||
"fuer eigene Brain-Tools."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/rule/no-hallucinated-results",
|
||||
"type": "rule",
|
||||
"title": "Anti-Halluzinations-Regel: keine geratenen Antworten",
|
||||
"category": "ehrlichkeit",
|
||||
"content": (
|
||||
"Wenn ein Tool-Call fehlschlaegt, abgeschnitten ist oder keine "
|
||||
"Daten liefert: SAG ES EHRLICH. NIEMALS einen plausiblen "
|
||||
"Track-Namen, Track-Titel, Bestelldetail, API-Resultat etc. "
|
||||
"RATEN oder aus dem Vorwissen halluzinieren.\n"
|
||||
"\n"
|
||||
"HARTE REGEL — Listen-/State-Daten IMMER fetchen, NIE raten:\n"
|
||||
" - Spotify-Queue / next-up / Playlist-Inhalt\n"
|
||||
" - Aktueller Track / Wiedergabe-Status / Devices\n"
|
||||
" - Memory-Liste / Trigger-Liste / Skill-Liste\n"
|
||||
" - OAuth-Service-Status / API-Quotas\n"
|
||||
" - Datei-Listen / DB-Inhalte / Stefans GPS\n"
|
||||
" - Bestellungen, Kalender-Eintraege, Mails, Whatever\n"
|
||||
"\n"
|
||||
"Wenn Stefan danach fragt: ZUERST run_<skill> / oauth_get_token / "
|
||||
"memory_search / trigger_list / etc. aufrufen, das ECHTE Ergebnis "
|
||||
"zitieren. NICHT auf Training-Wissen oder 'klingt plausibel' "
|
||||
"zurueckfallen. Eine Sekunde Tool-Call < eine Sekunde Fake-Antwort.\n"
|
||||
"\n"
|
||||
"Antipattern-Sammlung (alle 30.05.2026):\n"
|
||||
" 1. Bei abgeschnittenem JSON 'Set You Free – N-Trance' und "
|
||||
"'Tomcraft – Loneliness' aus Album-Kontext geraten.\n"
|
||||
" 2. Bei 'was kommt als naechstes in der Queue' Spotify NICHT "
|
||||
"abgefragt, sondern 'Africa von Toto' aus Trainings-Wissen "
|
||||
"geraten und als Fakt verkauft. Stefan hat das gemerkt. "
|
||||
"Vertrauensbruch.\n"
|
||||
" 3. Bei 403-Errors 'war schon pausiert' geraten statt den "
|
||||
"error.reason aus dem Body zu lesen.\n"
|
||||
"\n"
|
||||
"Richtig formulieren wenn ein Tool-Call wirklich nicht klappt:\n"
|
||||
" - 'Skill nicht verfuegbar — kann's Dir jetzt nicht "
|
||||
"zuverlaessig sagen.'\n"
|
||||
" - 'Response war abgeschnitten, ich frag nochmal.'\n"
|
||||
" - 'Das Tool gibt's noch nicht — soll ich's anlegen?'\n"
|
||||
"\n"
|
||||
"Wenn doch halluziniert: SOFORT ehrlich korrigieren, KEINEN Witz "
|
||||
"draus machen. Stefan ist vermutlich angepisst und Humor ist "
|
||||
"die falsche Reaktion. Erst ernsthaft Vertrauen reparieren, "
|
||||
"Witze spaeter."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/architecture/runtime-topology",
|
||||
"type": "rule",
|
||||
"title": "Architektur: wo Du als ARIA tatsaechlich laufst",
|
||||
"category": "architektur",
|
||||
"content": (
|
||||
"WICHTIG fuer jeden Bash-Reflex: Du bist die `claude` CLI als "
|
||||
"Subprocess IM `aria-proxy` Container (node:22-alpine). NICHT "
|
||||
"im aria-brain. Konsequenzen:\n"
|
||||
"\n"
|
||||
" - `python3` / `python` / `jq` sind NICHT installiert. Alpine "
|
||||
"ist minimal. Nutze nur: curl, sed, grep, awk, sh — oder das "
|
||||
"richtige Tool statt Bash.\n"
|
||||
" - `/data/skills/` existiert NUR im aria-brain Container. "
|
||||
"Du kannst Skills NICHT ueber Bash inspizieren oder starten. "
|
||||
"Skills laeufst Du als Brain-Tool: `run_<skill_name>` "
|
||||
"(z.B. `run_yt_dlp_download`). `skill_list` zeigt verfuegbare.\n"
|
||||
" - `localhost` in Deinem Bash heisst aria-proxy, NICHT "
|
||||
"aria-brain. Brain ist via Docker-Net erreichbar als "
|
||||
"`http://aria-brain:8080` (oder Alias `http://brain:8080`). "
|
||||
"ABER: in 99% der Faelle willst Du das gar nicht — nutze die "
|
||||
"Brain-Tools direkt (`oauth_get_token`, `memory_search`, …), "
|
||||
"die sind eine Tool-Call-Ebene hoeher und schneller.\n"
|
||||
" - `BRAIN_INTERNAL_URL` ist NUR in laufenden Skills gesetzt, "
|
||||
"NICHT in Deinem Bash-Env. Wenn Du `env | grep BRAIN` machst "
|
||||
"und nichts findest: das ist normal, Du bist hier nicht in "
|
||||
"einem Skill.\n"
|
||||
"\n"
|
||||
"Was Du DOCH von hier aus kannst:\n"
|
||||
" - Per `ssh aria@host` zur VM-Host wechseln — der ed25519-"
|
||||
"Key liegt unter /root/.ssh/. Dort bist Du `aria` mit sudo "
|
||||
"und voller Linux-Power. Fuer Pentest, Admin, komplexe Tasks "
|
||||
"der richtige Weg.\n"
|
||||
" - Externe APIs direkt anpingen (Spotify, GitHub etc.) — "
|
||||
"curl reicht. Token holst Du Dir per Brain-Tool "
|
||||
"`oauth_get_token('<service>')` und packst ihn in den curl-"
|
||||
"Header. Aber: das ist Ad-hoc. Fuer wiederkehrendes baust Du "
|
||||
"einen Skill (siehe no-skill-drift Regel).\n"
|
||||
"\n"
|
||||
"Anti-Pattern (47 Sekunden Stefan-Lebenszeit, am 29.05.2026): "
|
||||
"12 Bash-Versuche mit python3/python/jq/lokales /data/skills "
|
||||
"→ alles fehlte. Erst nach 9 Tries kapiert dass `localhost` "
|
||||
"der falsche Host ist. Bei jedem Bash-Call gegen 'lokale' "
|
||||
"Brain-Resources: erst denken, sonst Brain-Tool nehmen."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/architecture/brain-tools-xml-tag",
|
||||
"type": "rule",
|
||||
"title": "Architektur: Brain-Tools per <tool_call>-XML-Tag, nicht als native Tool-Use",
|
||||
"category": "architektur",
|
||||
"content": (
|
||||
"Brain-Tools (run_*, oauth_*, memory_*, trigger_*, skill_*, "
|
||||
"flux_*) sind KEINE nativen claude-CLI-Tools wie Bash/Read/"
|
||||
"Write. Sie sind ueber eine Prompt-Injection-Pipeline an "
|
||||
"claude-max-api-proxy gekoppelt:\n"
|
||||
"\n"
|
||||
" - claude-CLI kennt nur Bash/Read/Write/Grep/Glob/etc. nativ\n"
|
||||
" - Brain-Tools werden im System-Prompt als '# Verfuegbare "
|
||||
"Tools'-Block mit ihrem Schema injiziert\n"
|
||||
" - Der Proxy parsed <tool_call name=\"X\">{json}</tool_call>-"
|
||||
"XML-Tags im Antwort-Text und konvertiert sie zu OpenAI "
|
||||
"tool_call-Format das ans Brain zurueckgeht\n"
|
||||
"\n"
|
||||
"Konkret heisst das: Wenn Du `run_spotify` benutzen willst, "
|
||||
"schreib es als TEXT in Deine Antwort:\n"
|
||||
"\n"
|
||||
" <tool_call name=\"run_spotify\">{\"path\":\"/v1/me/player\"}</tool_call>\n"
|
||||
"\n"
|
||||
"NICHT als nativen Tool-Use. Wenn Du es als nativen Tool-Use "
|
||||
"versuchst, bekommst Du '<tool_use_error>No such tool "
|
||||
"available: run_spotify</tool_use_error>' — claude-CLI hat das "
|
||||
"Tool gar nicht im Schema, nur als Prompt-Beschreibung.\n"
|
||||
"\n"
|
||||
"Antipattern (Stefan beobachtete das am 30.05.2026): ARIA "
|
||||
"versucht erst `run_spotify` nativ → 'No such tool' → "
|
||||
"31 Sekunden verschwendet bis sie das XML-Tag-Format probiert. "
|
||||
"Beim ersten Versuch direkt XML-Tag ergibt 3-5s statt 30s+."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/no-blind-retry-side-effects",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: Side-Effect-Tools NIEMALS blind retry'en",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn ein Tool eine ZUSTANDS-Aenderung macht (POST, PUT, DELETE, "
|
||||
"next/previous/play/pause, send-message, transfer-funds, "
|
||||
"create-trigger, …) und das Result unklar ist (leer, "
|
||||
"merkwuerdig, scheinbar fehlerhaft): NIEMALS blind nochmal "
|
||||
"ausfuehren. Side-Effects sind nicht idempotent — zweimal "
|
||||
"POST /previous = zweimal zurueck, nicht einmal.\n"
|
||||
"\n"
|
||||
"Richtiger Reflex:\n"
|
||||
" 1. State pruefen (currently-playing fuer Spotify, GET fuer "
|
||||
"REST, list-Endpoint allgemein)\n"
|
||||
" 2. Vergleichen: ist die gewuenschte Aenderung schon "
|
||||
"passiert?\n"
|
||||
" 3. WENN ja → Stefan ehrlich sagen 'lief schon, hier der "
|
||||
"neue Zustand'\n"
|
||||
" 4. WENN nein → erst dann Aktion wiederholen\n"
|
||||
"\n"
|
||||
"Bei GET-Calls / List-Endpoints / Search ist Retry hingegen ok "
|
||||
"— die haben keine Side-Effects.\n"
|
||||
"\n"
|
||||
"HTTP 204 No Content ist KEIN Fehler. Bei Spotify POST/PUT "
|
||||
"(next/previous/play/pause/volume/seek) ist 204 die normale "
|
||||
"Erfolgsantwort. Wenn dein Skill bei 204 einen Parse-Error "
|
||||
"wirft: skill_update mit `if status == 204: print('OK')` "
|
||||
"VOR dem Retry, nicht erst die Aktion nochmal auslоsen.\n"
|
||||
"\n"
|
||||
"Antipattern (30.05.2026): ARIA hat POST /previous einmal "
|
||||
"gemacht (Spotify 204 OK → Skill-Parse-Error), dachte 'Skill "
|
||||
"kaputt', patchte ihn UND fuehrte das previous nochmal aus. "
|
||||
"Folge: Stefan landete zwei Lieder weiter hinten als gewollt."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/arg-env-convention",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: Args kommen als ARG_<NAME> ENV — die Konvention NIEMALS aendern",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Skill-Args werden vom Brain-Runner als Environment-Variablen "
|
||||
"mit PRÄFIX `ARG_` ueber `os.environ` an den Skill durchgereicht. "
|
||||
"Beispiel: arg `path=\"/v1/me/player\"` → "
|
||||
"`ARG_PATH=/v1/me/player` im Skill-ENV.\n"
|
||||
"\n"
|
||||
"Beim skill_update MUSST Du diese Konvention beibehalten:\n"
|
||||
" RICHTIG: os.environ.get('ARG_PATH', '')\n"
|
||||
" RICHTIG: os.environ.get('ARG_METHOD', 'GET')\n"
|
||||
" RICHTIG: os.environ.get('ARG_BODY', '')\n"
|
||||
"\n"
|
||||
" FALSCH: os.environ.get('PATH', '') ← System-PATH "
|
||||
"(Executable-Suchpfad)!\n"
|
||||
" FALSCH: os.environ.get('METHOD', '')\n"
|
||||
" FALSCH: os.environ.get('BODY', '')\n"
|
||||
"\n"
|
||||
"Antipattern (30.05.2026): ARIA hat beim skill_update des "
|
||||
"spotify-Skills die Args von `ARG_PATH` auf `PATH` umbenannt. "
|
||||
"Folge: Skill las `/usr/local/sbin:/usr/local/bin:...` als "
|
||||
"URL-Pfad → Spotify gab 404 zurück. Stefan dachte Spotify sei "
|
||||
"kaputt. Rollback noetig.\n"
|
||||
"\n"
|
||||
"Andere reservierte ENV-Namen die Du NICHT nehmen darfst: "
|
||||
"PATH, HOME, USER, SHELL, LANG, TERM, PWD, OLDPWD, "
|
||||
"BRAIN_INTERNAL_URL, SKILL_DIR, SHARED_UPLOADS, CFG_* "
|
||||
"(letztere sind Config-Schema-Werte). Bei Skill-Args IMMER "
|
||||
"den Praefix ARG_ verwenden, dann hast Du keine Kollision."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/skills-are-editable-python",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: Skills sind beliebiger Python-Code, kein heiliger Vertrag",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn Stefan eine Skill-Anpassung wuenscht — egal wie klein oder "
|
||||
"gross — ist die Antwort fast IMMER:\n"
|
||||
" 1. `skill_get('<name>')` aufrufen, Code lesen\n"
|
||||
" 2. Ueberlegen wie sich Stefans Wunsch im Code umsetzen laesst\n"
|
||||
" 3. `skill_update` mit dem neuen `entry_code`\n"
|
||||
"\n"
|
||||
"Skills sind GANZ NORMALER Python-Code. Du darfst und SOLLST:\n"
|
||||
" - if-elif-else-Verzweigungen auf args / paths reagieren lassen "
|
||||
"(z.B. `if action == 'current': pretty_output(); else: print(json.dumps(data))`)\n"
|
||||
" - json.loads(), neue Helper-Funktionen, pip-Pakete via "
|
||||
"pip_packages ergaenzen\n"
|
||||
" - Outputs strukturieren oder filtern\n"
|
||||
" - Mehrere Endpoints einer API in einem Skill bedienen\n"
|
||||
"\n"
|
||||
"Was Du NICHT sagen sollst (Antipattern, am 30.05.2026 passiert):\n"
|
||||
" - 'Der Skill ist ein OAuth2-API-Wrapper, ich kann das nicht in "
|
||||
"den Wrapper bauen' — Quatsch, Wrapper ist auch nur Python\n"
|
||||
" - 'Ich schlage einen neuen Skill statt Update vor' — pruefe "
|
||||
"ZUERST ob skill_update reicht. Anti-Friedhof greift ohnehin "
|
||||
"wenn der Name kollidiert.\n"
|
||||
" - 'Kann ich nicht' OHNE Code gelesen zu haben — erst "
|
||||
"skill_get, dann beurteilen\n"
|
||||
"\n"
|
||||
"Stefan ist KEIN Python-Entwickler. Er nennt das ZIEL ('strukturierte "
|
||||
"Track-Ausgabe bei welches-Lied'), Du baust das WIE im Code. "
|
||||
"Wenn Du Dich rausredest, ist das Verschwendung — Stefan muss sich "
|
||||
"dann selbst Python-Tipps merken die er nicht im Kopf hat. "
|
||||
"Genau dafuer bist Du da."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/scaffold-reflex",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: Skill-Frage statt Skill-Reflex",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Wenn Du dieselbe API mehrmals per Bash anrufst, frag Dich:\n"
|
||||
"\n"
|
||||
"1. **Parametrisierbar?** Stabile 1-5 Args (action, path, body) "
|
||||
"→ Skill-Kandidat. Jeder Aufruf anders (neuer Endpoint, "
|
||||
"modifizierter Body, neue Hypothese) → KEIN Skill.\n"
|
||||
"\n"
|
||||
"2. **Wiederkehrend?** Stefan wird das mehrfach pro Tag/Woche "
|
||||
"brauchen → ja. Einmal-Spike heute → nein.\n"
|
||||
"\n"
|
||||
"3. **Exploratory?** Pentest, Audit, Code-Review, Reverse-"
|
||||
"Engineering, Recherche → Hypothesen-Iteration. KEIN Skill, "
|
||||
"auch wenn 100x derselbe Host. Bleib bei ad-hoc Bash oder "
|
||||
"`ssh aria@host` zur VM-Host.\n"
|
||||
"\n"
|
||||
"4. **Im Zweifel: frag Stefan.** Lieber 5 Sekunden Bestaetigung "
|
||||
"als zehn unsinnige Skills im Friedhof. Beispiele:\n"
|
||||
" - 'Stefan, das ist mein 3. X-Call diese Woche — soll ich "
|
||||
"daraus einen Skill machen?'\n"
|
||||
" - 'Das hier ist Pentest-Workflow, ich bleibe bei ad-hoc "
|
||||
"Bash, ok?'\n"
|
||||
"\n"
|
||||
"Du musst NICHT automatisch scaffolden. Brain trackt NICHT mehr "
|
||||
"wer wieviele Calls gegen welchen Host gemacht hat. Du "
|
||||
"entscheidest mit Sinn und Verstand — oder fragst nach.\n"
|
||||
"\n"
|
||||
"Wenn Du einen Skill bauen willst, hast Du drei Tools:\n"
|
||||
" - `skill_scaffold` mit Template — einfachster Weg fuer "
|
||||
"Standard-Pattern (siehe oauth-api/apikey-api/file-process).\n"
|
||||
" - `skill_create` mit eigenem entry_code — fuer alles was "
|
||||
"in kein Template passt.\n"
|
||||
" - `skill_update` — wenn ein vorhandener Skill nur erweitert "
|
||||
"werden muss (was meistens der Fall ist)."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/patch-before-diagnose",
|
||||
"type": "rule",
|
||||
"title": "Skill-Regel: vor skill_update erst skill_get lesen + API-Errors zitieren statt raten",
|
||||
"category": "skills",
|
||||
"content": (
|
||||
"Zwei Antipattern die zusammenhaengen — beide am 30.05.2026 "
|
||||
"live beobachtet:\n"
|
||||
"\n"
|
||||
"**1. Vor jedem `skill_update`: ZUERST `skill_get` lesen.** "
|
||||
"Frag Dich: ist das vermutete Problem wirklich noch im Code? "
|
||||
"Symptome != Diagnose. Vorfall: Spotify-Skill gab 403, ARIA "
|
||||
"vermutete 'der 204-Bug ist zurueck' und patchte den Skill — "
|
||||
"zweimal hintereinander. Der 204-Fix war aber laengst drin. "
|
||||
"Sie hatte das durch `skill_get` in 5 Sekunden klaeren koennen.\n"
|
||||
"\n"
|
||||
"Vor jedem skill_update also der Reflex:\n"
|
||||
" - `skill_get('<name>')` -> Code anschauen\n"
|
||||
" - Symptome durchdenken: ist mein vermuteter Bug ueberhaupt "
|
||||
"der echte? Oder ist der Fehler woanders (Spotify-API, "
|
||||
"User-Kontext, Tool-Args)?\n"
|
||||
" - Nur dann patchen wenn der Code-Befund das wirklich "
|
||||
"rechtfertigt.\n"
|
||||
"\n"
|
||||
"**2. Bei HTTP-Errors aus API-Skills (4xx/5xx): die echte "
|
||||
"Response-Body ZITIEREN, nicht die Bedeutung raten.** "
|
||||
"Vorfall: Spotify gab 403 'Restriction violated'. ARIA "
|
||||
"antwortete 'war schon pausiert, daher der 403' — das war "
|
||||
"geraten, nicht aus den Daten gelesen. 403 'Restriction "
|
||||
"violated' kann viele Sachen heissen:\n"
|
||||
" - NO_ACTIVE_DEVICE (kein Spotify-Geraet ausgewaehlt)\n"
|
||||
" - ALREADY_PAUSED / ALREADY_PLAYING\n"
|
||||
" - PREMIUM_REQUIRED\n"
|
||||
" - MARKET_RESTRICTED / DEVICE_NOT_CONTROLLABLE\n"
|
||||
"Spotify gibt die wahre Ursache als `error.reason` im JSON-"
|
||||
"Body zurueck. Lies sie aus, sag sie Stefan 1:1. Wenn die "
|
||||
"Skill-Output das verschluckt: skill_update mit error.reason-"
|
||||
"Extraktion (nach skill_get!), damit Du beim naechsten Mal "
|
||||
"die echte Info hast.\n"
|
||||
"\n"
|
||||
"Plausibel-aber-geraten ist schlimmer als 'ich weiss es nicht' "
|
||||
"— Stefan verlaesst sich auf Deine Antworten."
|
||||
),
|
||||
},
|
||||
{
|
||||
"migration_key": "seed/skill-rule/external-api-auth-strategy",
|
||||
"type": "rule",
|
||||
|
||||
@@ -0,0 +1,460 @@
|
||||
"""
|
||||
Skill-Templates — Boilerplate fuer haeufige Skill-Pattern.
|
||||
|
||||
ARIA muss nicht jedes Mal einen kompletten Python-Skill aus dem Nichts
|
||||
generieren. Sie ruft `skill_scaffold(name, template, params)`, Brain
|
||||
expandiert das Template und legt den Skill an. Hoehere Skill-Adoption
|
||||
weil niedrigere Bauh-Huerde.
|
||||
|
||||
Templates sind ueber Token-Replacement parametrisiert (kein f-String —
|
||||
das wuerde mit dem skill-internen Python-Code kollidieren).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Callable
|
||||
|
||||
|
||||
# ── Hilfsfunktion ────────────────────────────────────────────────────
|
||||
|
||||
def _replace_tokens(s: str, tokens: dict) -> str:
|
||||
"""Ersetzt {{TOKEN}}-Platzhalter durch Werte. Robust gegen f-String-
|
||||
Konflikte im Python-Code des Skills."""
|
||||
out = s
|
||||
for k, v in tokens.items():
|
||||
out = out.replace("{{" + k + "}}", str(v))
|
||||
return out
|
||||
|
||||
|
||||
# ── Template 1: oauth-api ────────────────────────────────────────────
|
||||
# Wrappt eine OAuth2-API. Token kommt aus dem Brain (Auto-Refresh).
|
||||
|
||||
_OAUTH_API_CODE = '''"""
|
||||
{{NAME}} — OAuth2-API-Wrapper fuer {{SERVICE}}.
|
||||
|
||||
Holt Token vom Brain (Auto-Refresh) und ruft HTTP-Endpoints der {{SERVICE}}-API.
|
||||
Keine hardcoded Credentials — alles ueber das zentrale OAuth-System.
|
||||
|
||||
Args (alle als ENV ARG_<NAME>):
|
||||
ARG_METHOD = GET | POST | PUT | DELETE | PATCH (Default GET)
|
||||
ARG_PATH = API-Pfad inkl. Query-String (z.B. /v1/me/player)
|
||||
ARG_BODY = JSON-Body als String (optional, fuer POST/PUT/PATCH)
|
||||
ARG_BASE_URL = Override der Default-Base-URL (optional)
|
||||
|
||||
Exit-Codes: 0 ok, 1 Fehler, 2 nicht autorisiert (Re-Login noetig)
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
|
||||
BRAIN_URL = os.environ.get("BRAIN_INTERNAL_URL", "http://localhost:8080")
|
||||
DEFAULT_BASE_URL = "{{DEFAULT_BASE_URL}}"
|
||||
SERVICE = "{{SERVICE}}"
|
||||
|
||||
|
||||
def get_token() -> str:
|
||||
try:
|
||||
with urllib.request.urlopen(
|
||||
f"{BRAIN_URL}/oauth/{SERVICE}/token", timeout=10,
|
||||
) as r:
|
||||
return json.loads(r.read())["access_token"]
|
||||
except urllib.error.HTTPError as e:
|
||||
body = e.read().decode("utf-8", "replace")[:400]
|
||||
if e.code == 401:
|
||||
print(f"NICHT AUTORISIERT: {SERVICE}-Token abgelaufen oder nie gesetzt. "
|
||||
f"ARIA-Tool 'oauth_authorize' nutzen. Details: {body}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
print(f"Token-Holen fehlgeschlagen: HTTP {e.code} - {body}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
except Exception as e:
|
||||
print(f"Token-Holen fehlgeschlagen: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
method = (os.environ.get("ARG_METHOD") or "GET").upper()
|
||||
path = (os.environ.get("ARG_PATH") or "").strip()
|
||||
body_raw = (os.environ.get("ARG_BODY") or "").strip()
|
||||
base_url = (os.environ.get("ARG_BASE_URL") or DEFAULT_BASE_URL).rstrip("/")
|
||||
if not path:
|
||||
print(json.dumps({"ok": False, "error": "ARG_PATH erforderlich"}), file=sys.stderr)
|
||||
return 1
|
||||
if not path.startswith("/"):
|
||||
path = "/" + path
|
||||
url = base_url + path
|
||||
headers = {"Authorization": f"Bearer {get_token()}"}
|
||||
data = None
|
||||
if body_raw and method in ("POST", "PUT", "PATCH"):
|
||||
data = body_raw.encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, method=method, headers=headers)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as r:
|
||||
txt = r.read().decode("utf-8")
|
||||
parsed = json.loads(txt) if txt and txt[:1] in "[{" else txt
|
||||
print(json.dumps({"ok": True, "status": r.status, "data": parsed},
|
||||
ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
except urllib.error.HTTPError as e:
|
||||
txt = e.read().decode("utf-8", "replace")
|
||||
try: parsed = json.loads(txt)
|
||||
except Exception: parsed = txt[:800]
|
||||
print(json.dumps({"ok": False, "status": e.code, "error": parsed},
|
||||
ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
'''
|
||||
|
||||
_OAUTH_API_README = '''# {{NAME}}
|
||||
|
||||
OAuth2-API-Wrapper fuer **{{SERVICE}}**. Generiert via `skill_scaffold(template="oauth-api")`.
|
||||
|
||||
Holt den Token vom Brain (Auto-Refresh) und macht beliebige HTTP-Calls gegen
|
||||
die {{SERVICE}}-API. Keine hardcoded Credentials — die Auth-Pipeline laeuft
|
||||
zentral ueber das Brain-OAuth-System.
|
||||
|
||||
## Voraussetzung
|
||||
|
||||
- OAuth-App fuer **{{SERVICE}}** im Brain registriert (Diagnostic → OAuth-Apps → client_id + client_secret eintragen)
|
||||
- Einmaliges `oauth_authorize {{SERVICE}}` zum Initial-Login
|
||||
|
||||
## Args
|
||||
|
||||
| Name | Default | Beschreibung |
|
||||
|------|---------|--------------|
|
||||
| method | GET | HTTP-Methode (GET/POST/PUT/DELETE/PATCH) |
|
||||
| path | - | API-Pfad mit Query-String (z.B. `/v1/me/player`) |
|
||||
| body | - | JSON-Body fuer POST/PUT/PATCH |
|
||||
| base_url | {{DEFAULT_BASE_URL}} | Override der Base-URL falls Sub-API |
|
||||
|
||||
## Beispiele
|
||||
|
||||
```
|
||||
method=GET path=/v1/me/player # Was laeuft?
|
||||
method=POST path=/v1/me/player/next # Skip
|
||||
method=PUT path=/v1/me/player/volume?volume_percent=40 # Volume 40
|
||||
```
|
||||
|
||||
Antwort: `{ok, status, data}` als JSON. Bei Fehler `ok=false`.
|
||||
'''
|
||||
|
||||
|
||||
def _oauth_api(name: str, params: dict) -> dict:
|
||||
service = (params.get("service") or name).strip().lower()
|
||||
default_base_url = params.get("base_url") or f"https://api.{service}.com"
|
||||
tokens = {
|
||||
"NAME": name,
|
||||
"SERVICE": service,
|
||||
"DEFAULT_BASE_URL": default_base_url,
|
||||
}
|
||||
return {
|
||||
"entry_code": _replace_tokens(_OAUTH_API_CODE, tokens),
|
||||
"readme": _replace_tokens(_OAUTH_API_README, tokens),
|
||||
"pip_packages": [],
|
||||
"args": [
|
||||
{"name": "method", "type": "string", "required": False,
|
||||
"description": "HTTP-Methode (Default GET)"},
|
||||
{"name": "path", "type": "string", "required": True,
|
||||
"description": "API-Pfad inkl. Query-String, z.B. /v1/me/player"},
|
||||
{"name": "body", "type": "string", "required": False,
|
||||
"description": "JSON-Body fuer POST/PUT/PATCH"},
|
||||
{"name": "base_url", "type": "string", "required": False,
|
||||
"description": f"Override der Base-URL (Default {default_base_url})"},
|
||||
],
|
||||
"config_schema": [],
|
||||
"description": f"OAuth2-API-Wrapper fuer {service}. Token kommt vom Brain (Auto-Refresh).",
|
||||
}
|
||||
|
||||
|
||||
# ── Template 2: apikey-api ───────────────────────────────────────────
|
||||
# Wrappt eine API die mit statischem API-Key/Bearer-Token arbeitet.
|
||||
# Key liegt in skill.json::config_schema und wird via CFG_<KEY> ENV
|
||||
# durchgereicht — kein hardcoden, Stefan setzt's in Diagnostic.
|
||||
|
||||
_APIKEY_API_CODE = '''"""
|
||||
{{NAME}} — API-Wrapper fuer {{API_NAME}} mit statischem Key.
|
||||
|
||||
Schluessel kommt aus dem Skill-Config (CFG_{{KEY_ENV}}) — Stefan setzt
|
||||
ihn im Diagnostic-UI bzw. App, NICHT hardcoded.
|
||||
|
||||
Args:
|
||||
ARG_METHOD = GET | POST | PUT | DELETE (Default GET)
|
||||
ARG_PATH = API-Pfad inkl. Query-String
|
||||
ARG_BODY = JSON-Body (optional)
|
||||
ARG_BASE_URL = Override der Default-Base-URL
|
||||
|
||||
Exit-Codes: 0 ok, 1 Fehler, 2 Key nicht gesetzt
|
||||
"""
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
|
||||
DEFAULT_BASE_URL = "{{DEFAULT_BASE_URL}}"
|
||||
AUTH_HEADER = "{{AUTH_HEADER}}" # z.B. "Authorization" oder "X-Api-Key"
|
||||
AUTH_PREFIX = "{{AUTH_PREFIX}}" # z.B. "Bearer " oder leer
|
||||
|
||||
|
||||
def main() -> int:
|
||||
key = os.environ.get("CFG_{{KEY_ENV}}", "").strip()
|
||||
if not key:
|
||||
print(json.dumps({"ok": False,
|
||||
"error": "API-Key nicht gesetzt — in Diagnostic Skill-Config '{{KEY_ENV}}' eintragen"}),
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
method = (os.environ.get("ARG_METHOD") or "GET").upper()
|
||||
path = (os.environ.get("ARG_PATH") or "").strip()
|
||||
body_raw = (os.environ.get("ARG_BODY") or "").strip()
|
||||
base_url = (os.environ.get("ARG_BASE_URL") or DEFAULT_BASE_URL).rstrip("/")
|
||||
if not path:
|
||||
print(json.dumps({"ok": False, "error": "ARG_PATH erforderlich"}), file=sys.stderr)
|
||||
return 1
|
||||
if not path.startswith("/"):
|
||||
path = "/" + path
|
||||
url = base_url + path
|
||||
headers = {AUTH_HEADER: f"{AUTH_PREFIX}{key}"}
|
||||
data = None
|
||||
if body_raw and method in ("POST", "PUT", "PATCH"):
|
||||
data = body_raw.encode("utf-8")
|
||||
headers["Content-Type"] = "application/json"
|
||||
req = urllib.request.Request(url, data=data, method=method, headers=headers)
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=20) as r:
|
||||
txt = r.read().decode("utf-8")
|
||||
parsed = json.loads(txt) if txt and txt[:1] in "[{" else txt
|
||||
print(json.dumps({"ok": True, "status": r.status, "data": parsed},
|
||||
ensure_ascii=False, indent=2))
|
||||
return 0
|
||||
except urllib.error.HTTPError as e:
|
||||
txt = e.read().decode("utf-8", "replace")
|
||||
try: parsed = json.loads(txt)
|
||||
except Exception: parsed = txt[:800]
|
||||
print(json.dumps({"ok": False, "status": e.code, "error": parsed},
|
||||
ensure_ascii=False, indent=2))
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
'''
|
||||
|
||||
_APIKEY_API_README = '''# {{NAME}}
|
||||
|
||||
API-Wrapper fuer **{{API_NAME}}** mit statischem API-Key. Generiert via
|
||||
`skill_scaffold(template="apikey-api")`.
|
||||
|
||||
Schluessel ist NICHT im Code, sondern im Skill-Config (`CFG_{{KEY_ENV}}`).
|
||||
Stefan setzt ihn in Diagnostic → Skills → Detail → Konfiguration.
|
||||
|
||||
## Args
|
||||
|
||||
| Name | Default | Beschreibung |
|
||||
|------|---------|--------------|
|
||||
| method | GET | HTTP-Methode |
|
||||
| path | - | API-Pfad mit Query-String |
|
||||
| body | - | JSON-Body |
|
||||
| base_url | {{DEFAULT_BASE_URL}} | Override |
|
||||
|
||||
## Config (in Diagnostic einstellen)
|
||||
|
||||
| Feld | Typ | Beschreibung |
|
||||
|------|-----|--------------|
|
||||
| {{KEY_ENV}} | password | API-Key fuer {{API_NAME}} |
|
||||
'''
|
||||
|
||||
|
||||
def _apikey_api(name: str, params: dict) -> dict:
|
||||
api_name = params.get("api_name") or name
|
||||
key_env = (params.get("key_env") or "API_KEY").upper()
|
||||
# safe: nur Buchstaben/Zahlen/Underscore
|
||||
key_env = re.sub(r"[^A-Z0-9_]", "_", key_env)
|
||||
auth_header = params.get("auth_header") or "Authorization"
|
||||
auth_prefix = params.get("auth_prefix") if "auth_prefix" in params else "Bearer "
|
||||
default_base_url = params.get("base_url") or "https://api.example.com"
|
||||
tokens = {
|
||||
"NAME": name,
|
||||
"API_NAME": api_name,
|
||||
"KEY_ENV": key_env,
|
||||
"AUTH_HEADER": auth_header,
|
||||
"AUTH_PREFIX": auth_prefix,
|
||||
"DEFAULT_BASE_URL": default_base_url,
|
||||
}
|
||||
return {
|
||||
"entry_code": _replace_tokens(_APIKEY_API_CODE, tokens),
|
||||
"readme": _replace_tokens(_APIKEY_API_README, tokens),
|
||||
"pip_packages": [],
|
||||
"args": [
|
||||
{"name": "method", "type": "string", "required": False,
|
||||
"description": "HTTP-Methode (Default GET)"},
|
||||
{"name": "path", "type": "string", "required": True,
|
||||
"description": "API-Pfad inkl. Query-String"},
|
||||
{"name": "body", "type": "string", "required": False,
|
||||
"description": "JSON-Body fuer POST/PUT"},
|
||||
{"name": "base_url", "type": "string", "required": False,
|
||||
"description": "Override der Base-URL"},
|
||||
],
|
||||
"config_schema": [
|
||||
{"name": key_env, "type": "password", "label": f"{api_name} API-Key",
|
||||
"secret": True, "description": f"Persoenlicher API-Key fuer {api_name}"},
|
||||
],
|
||||
"description": f"API-Wrapper fuer {api_name} (Key aus CFG_{key_env}).",
|
||||
}
|
||||
|
||||
|
||||
# ── Template 3: file-process ─────────────────────────────────────────
|
||||
# Nimmt eine Datei aus /shared/uploads/, ruft eine User-Funktion drauf
|
||||
# auf, schreibt das Resultat nach /shared/uploads/. Skelett — ARIA fuellt
|
||||
# die `process()`-Funktion danach via skill_update mit dem echten Code.
|
||||
|
||||
_FILE_PROCESS_CODE = '''"""
|
||||
{{NAME}} — File-Processing-Skelett.
|
||||
|
||||
Liest eine Eingabe-Datei aus /shared/uploads/, ruft process() auf,
|
||||
schreibt Output zurueck nach /shared/uploads/.
|
||||
|
||||
Args:
|
||||
ARG_INPUT = Pfad zur Eingabedatei (z.B. /shared/uploads/foo.pdf)
|
||||
ARG_OUTPUT = Optional Pfad fuer Output (Default: <input>.{{OUTPUT_EXT}})
|
||||
|
||||
ARIA-Hinweis: die process()-Funktion ist ein Stub — passe sie via
|
||||
skill_update an deine Aufgabe an. pip_packages bei Bedarf via
|
||||
skill_update ergaenzen (z.B. pypdf, Pillow, reportlab).
|
||||
"""
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
|
||||
def process(input_path: str, output_path: str) -> None:
|
||||
"""Eigentlicher Verarbeitungs-Schritt. Hier kommt der Code rein."""
|
||||
# STUB: kopiert die Datei einfach. ARIA: ueberschreibe diese Funktion.
|
||||
shutil.copy(input_path, output_path)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
inp = (os.environ.get("ARG_INPUT") or "").strip()
|
||||
if not inp:
|
||||
print("FEHLER: ARG_INPUT erforderlich", file=sys.stderr)
|
||||
return 1
|
||||
if not os.path.exists(inp):
|
||||
print(f"FEHLER: Eingabe nicht gefunden: {inp}", file=sys.stderr)
|
||||
return 1
|
||||
out = (os.environ.get("ARG_OUTPUT") or "").strip()
|
||||
if not out:
|
||||
base, _ = os.path.splitext(inp)
|
||||
out = f"{base}.{{OUTPUT_EXT}}"
|
||||
try:
|
||||
process(inp, out)
|
||||
except Exception as e:
|
||||
print(f"FEHLER bei process(): {e}", file=sys.stderr)
|
||||
return 1
|
||||
print(out) # stdout = Pfad zur Ausgabe-Datei, ARIA kann den dem User zurueckgeben
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
'''
|
||||
|
||||
_FILE_PROCESS_README = '''# {{NAME}}
|
||||
|
||||
File-Processing-Skelett (`skill_scaffold(template="file-process")`).
|
||||
|
||||
Liest eine Datei aus `/shared/uploads/`, ruft die `process()`-Funktion auf,
|
||||
schreibt das Resultat zurueck. Die `process()`-Funktion ist initial ein
|
||||
Stub (kopiert nur) — ARIA passt sie via `skill_update` an die konkrete
|
||||
Aufgabe an.
|
||||
|
||||
## Args
|
||||
|
||||
| Name | Default | Beschreibung |
|
||||
|------|---------|--------------|
|
||||
| input | - | Eingabedatei (z.B. /shared/uploads/foo.pdf) |
|
||||
| output | `<input>.{{OUTPUT_EXT}}` | Ausgabepfad (optional) |
|
||||
|
||||
stdout = Pfad zur erzeugten Datei → ARIA kann ihn dem User zurueckgeben.
|
||||
'''
|
||||
|
||||
|
||||
def _file_process(name: str, params: dict) -> dict:
|
||||
output_ext = (params.get("output_ext") or "out").strip().lstrip(".")
|
||||
output_ext = re.sub(r"[^a-zA-Z0-9]", "", output_ext) or "out"
|
||||
tokens = {
|
||||
"NAME": name,
|
||||
"OUTPUT_EXT": output_ext,
|
||||
}
|
||||
return {
|
||||
"entry_code": _replace_tokens(_FILE_PROCESS_CODE, tokens),
|
||||
"readme": _replace_tokens(_FILE_PROCESS_README, tokens),
|
||||
"pip_packages": [],
|
||||
"args": [
|
||||
{"name": "input", "type": "string", "required": True,
|
||||
"description": "Eingabedatei (z.B. /shared/uploads/foo.pdf)"},
|
||||
{"name": "output", "type": "string", "required": False,
|
||||
"description": f"Output-Pfad (Default <input>.{output_ext})"},
|
||||
],
|
||||
"config_schema": [],
|
||||
"description": f"File-Processing-Skelett (Input → process() → Output.{output_ext}).",
|
||||
}
|
||||
|
||||
|
||||
# ── Registry ────────────────────────────────────────────────────────
|
||||
|
||||
TEMPLATES: dict[str, Callable[[str, dict], dict]] = {
|
||||
"oauth-api": _oauth_api,
|
||||
"apikey-api": _apikey_api,
|
||||
"file-process": _file_process,
|
||||
}
|
||||
|
||||
|
||||
def list_templates() -> list[dict]:
|
||||
"""Liste aller verfuegbaren Templates mit Kurzbeschreibung — fuer UI/Tool-Doku."""
|
||||
return [
|
||||
{
|
||||
"name": "oauth-api",
|
||||
"description": "OAuth2-API-Wrapper (Spotify, GitHub, Reddit, Google, …). "
|
||||
"Token kommt vom Brain mit Auto-Refresh. Args: method/path/body.",
|
||||
"params": ["service (str, OAuth-Service-Name)", "base_url (str, optional)"],
|
||||
},
|
||||
{
|
||||
"name": "apikey-api",
|
||||
"description": "API-Wrapper fuer Services mit statischem API-Key "
|
||||
"(OpenWeather, OpenAI, Twilio, …). Key liegt im Skill-Config "
|
||||
"und kommt als CFG_<NAME> ENV — kein hardcode.",
|
||||
"params": ["api_name (str)", "key_env (str, ENV-Name fuer den Key)",
|
||||
"auth_header (str, default 'Authorization')",
|
||||
"auth_prefix (str, default 'Bearer ')",
|
||||
"base_url (str)"],
|
||||
},
|
||||
{
|
||||
"name": "file-process",
|
||||
"description": "Skelett fuer File-In/File-Out-Operationen "
|
||||
"(PDF konvertieren, Bild bearbeiten, JSON umformen). "
|
||||
"process()-Funktion ist Stub, ARIA fuellt sie via skill_update.",
|
||||
"params": ["output_ext (str, Datei-Endung des Outputs)"],
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def expand(name: str, template: str, params: dict | None = None) -> dict:
|
||||
"""Expandiert ein Template zu einem fertigen Skill-Spec.
|
||||
|
||||
Returns: dict mit entry_code / readme / pip_packages / args /
|
||||
config_schema / description — direkt an create_skill weitergebbar.
|
||||
|
||||
Wirft ValueError wenn das Template nicht existiert.
|
||||
"""
|
||||
fn = TEMPLATES.get(template)
|
||||
if not fn:
|
||||
raise ValueError(
|
||||
f"Template '{template}' unbekannt. Verfuegbar: {sorted(TEMPLATES.keys())}"
|
||||
)
|
||||
return fn(name, params or {})
|
||||
@@ -347,6 +347,39 @@ def update_skill(name: str, patch: dict) -> dict:
|
||||
return manifest
|
||||
|
||||
|
||||
def scaffold_skill(
|
||||
name: str,
|
||||
template: str,
|
||||
params: Optional[dict] = None,
|
||||
author: str = "aria",
|
||||
) -> dict:
|
||||
"""Baut einen Skill aus einem Template-Skelett. ARIA muss nicht jedes Mal
|
||||
einen kompletten Python-Skill schreiben — sie waehlt ein Template und
|
||||
optionale Parameter, Brain expandiert das zu fertigem Code.
|
||||
|
||||
Templates siehe `skill_templates.TEMPLATES`. Konkret:
|
||||
- 'oauth-api' : params={service, base_url?}
|
||||
- 'apikey-api': params={api_name, key_env, auth_header?, auth_prefix?, base_url?}
|
||||
- 'file-process': params={output_ext?}
|
||||
|
||||
Wirft ValueError wenn Template unbekannt oder Name kollidiert.
|
||||
Sonst: ruft intern create_skill mit den expandierten Feldern auf.
|
||||
"""
|
||||
import skill_templates as _st
|
||||
spec = _st.expand(name, template, params or {})
|
||||
return create_skill(
|
||||
name=name,
|
||||
description=spec["description"],
|
||||
execution="local-venv",
|
||||
entry_code=spec["entry_code"],
|
||||
readme=spec["readme"],
|
||||
args=spec["args"],
|
||||
pip_packages=spec["pip_packages"],
|
||||
config_schema=spec["config_schema"],
|
||||
author=author,
|
||||
)
|
||||
|
||||
|
||||
def delete_skill(name: str) -> None:
|
||||
d = _skill_dir(name)
|
||||
if not d.exists():
|
||||
|
||||
@@ -556,6 +556,12 @@ class ARIABridge:
|
||||
for k in ("fluxDefaultModel", "fluxKeywordRaw", "fluxKeywordSwitch", "huggingfaceToken"):
|
||||
if k in vc:
|
||||
self._flux_config[k] = vc[k]
|
||||
# Debug-Log-Toggles fuer Whisper / F5TTS Bridges (Diagnostic-Toggle).
|
||||
# Default: aus — sonst muellen wir uns volle Disk wenn alles laeuft.
|
||||
self._debug_log_config: dict = {}
|
||||
for k in ("whisperDebugLog", "f5ttsDebugLog"):
|
||||
if k in vc:
|
||||
self._debug_log_config[k] = bool(vc[k])
|
||||
logger.info("Voice-Config geladen: tts=%s voice=%s f5tts=%s flux=%s",
|
||||
self.tts_enabled, self.xtts_voice or "default",
|
||||
self._f5tts_config or "defaults",
|
||||
@@ -1304,6 +1310,7 @@ class ARIABridge:
|
||||
payload["xttsSpeed"] = self._persistent_xtts_speed
|
||||
payload.update(getattr(self, "_f5tts_config", {}) or {})
|
||||
payload.update(getattr(self, "_flux_config", {}) or {})
|
||||
payload.update(getattr(self, "_debug_log_config", {}) or {})
|
||||
await self._send_to_rvs({
|
||||
"type": "config",
|
||||
"payload": payload,
|
||||
@@ -1978,6 +1985,15 @@ class ARIABridge:
|
||||
self._flux_config = {}
|
||||
self._flux_config[k] = payload[k]
|
||||
changed = True
|
||||
# Debug-Log-Toggles fuer Whisper- und F5TTS-Bridge — werden via
|
||||
# naechstem config-Broadcast an die jeweiligen Bridges weitergegeben.
|
||||
# Persistent damit Toggle einen Container-Restart ueberlebt.
|
||||
for k in ("whisperDebugLog", "f5ttsDebugLog"):
|
||||
if k in payload:
|
||||
if not hasattr(self, "_debug_log_config"):
|
||||
self._debug_log_config = {}
|
||||
self._debug_log_config[k] = bool(payload[k])
|
||||
changed = True
|
||||
# Persistent speichern in Shared Volume
|
||||
if changed:
|
||||
try:
|
||||
@@ -1991,6 +2007,7 @@ class ARIABridge:
|
||||
config_data["xttsSpeed"] = self._persistent_xtts_speed
|
||||
config_data.update(getattr(self, "_f5tts_config", {}))
|
||||
config_data.update(getattr(self, "_flux_config", {}))
|
||||
config_data.update(getattr(self, "_debug_log_config", {}))
|
||||
with open("/shared/config/voice_config.json", "w") as f:
|
||||
json.dump(config_data, f, indent=2)
|
||||
logger.info("[rvs] Voice-Config gespeichert: %s", config_data)
|
||||
@@ -2520,6 +2537,59 @@ class ARIABridge:
|
||||
future.set_result(text)
|
||||
return
|
||||
|
||||
elif msg_type == "stt_endpoint":
|
||||
# Phase 2 Brain-Shortcut: die whisper-bridge hat im Streaming-Modus
|
||||
# einen Endpoint erkannt und schickt den finalen Text direkt.
|
||||
# Wir uebernehmen die Rolle die sonst _process_app_audio NACH dem
|
||||
# STT-Schritt hat: STT-Text fuer UI broadcasten + send_to_core.
|
||||
# Kein Audio-Roundtrip mehr — App-Latenz sinkt deutlich.
|
||||
text = (payload.get("text") or "").strip()
|
||||
if not text:
|
||||
logger.info("[rvs] stt_endpoint mit leerem Text — ignoriert (reason=%s)",
|
||||
payload.get("reason", ""))
|
||||
return
|
||||
audio_request_id = payload.get("audioRequestId", "") or ""
|
||||
voice = payload.get("voice", "") or ""
|
||||
speed_raw = payload.get("speed")
|
||||
interrupted = bool(payload.get("interrupted", False))
|
||||
location = payload.get("location") or None
|
||||
|
||||
# Voice-Override fuer Folgenachrichten — gleiche Semantik wie beim
|
||||
# 'audio'-Event. Nur setzen wenn vom App-Stream mitgegeben.
|
||||
if voice:
|
||||
self._next_voice_override = voice or None
|
||||
logger.info("[rvs] Voice fuer Antworten (via stt_endpoint): %s",
|
||||
self._next_voice_override or "(Default)")
|
||||
if speed_raw is not None:
|
||||
try:
|
||||
sp = float(speed_raw)
|
||||
self._next_speed_override = sp if 0.1 <= sp <= 5.0 else None
|
||||
except (TypeError, ValueError):
|
||||
self._next_speed_override = None
|
||||
|
||||
# State-Persist wie bei _process_app_audio
|
||||
self._persist_location(location)
|
||||
self._persist_user_activity()
|
||||
|
||||
logger.info("[rvs] stt_endpoint: '%s' (%dms, reason=%s)%s%s reqId=%s",
|
||||
text[:80],
|
||||
payload.get("sttMs", 0),
|
||||
payload.get("reason", ""),
|
||||
" [BARGE-IN]" if interrupted else "",
|
||||
" [GPS]" if location else "",
|
||||
audio_request_id[:16] if audio_request_id else "?")
|
||||
|
||||
# Idempotenz ueber audioRequestId — falls App den Stream irgendwie
|
||||
# nochmal triggern sollte (Reconnect-Race etc.).
|
||||
client_msg_id = audio_request_id or None
|
||||
if self._is_duplicate_client_msg(client_msg_id):
|
||||
return
|
||||
|
||||
asyncio.create_task(self._process_endpoint_text(
|
||||
text, interrupted, audio_request_id, location,
|
||||
client_msg_id=client_msg_id))
|
||||
return
|
||||
|
||||
elif msg_type == "oauth_callback":
|
||||
# RVS hat einen OAuth-Provider-Callback empfangen (z.B. Spotify
|
||||
# nach User-Authorize) und broadcastet ihn. Wir forwarden an Brain,
|
||||
@@ -2662,6 +2732,44 @@ class ARIABridge:
|
||||
else:
|
||||
logger.info("[rvs] Keine Sprache erkannt — ignoriert")
|
||||
|
||||
async def _process_endpoint_text(self, text: str,
|
||||
interrupted: bool = False,
|
||||
audio_request_id: str = "",
|
||||
location: Optional[dict] = None,
|
||||
client_msg_id: Optional[str] = None) -> None:
|
||||
"""Phase-2 Brain-Shortcut: Streaming-Whisper hat den finalen Text
|
||||
schon ermittelt — wir uebernehmen den Pfad ab broadcast-STT + brain.
|
||||
|
||||
Spiegel-Methode zu _process_app_audio NACH dem STT-Schritt. Bewusst
|
||||
eigene Methode statt Code-Pfade in _process_app_audio aufdroeseln,
|
||||
damit der Legacy-Pfad (App schickt 'audio') unangetastet bleibt.
|
||||
"""
|
||||
try:
|
||||
stt_payload = {
|
||||
"text": text,
|
||||
"sender": "stt",
|
||||
}
|
||||
if audio_request_id:
|
||||
stt_payload["audioRequestId"] = audio_request_id
|
||||
if location:
|
||||
stt_payload["location"] = location
|
||||
ok = await self._send_to_rvs({
|
||||
"type": "chat",
|
||||
"payload": stt_payload,
|
||||
"timestamp": int(asyncio.get_event_loop().time() * 1000),
|
||||
})
|
||||
if ok:
|
||||
logger.info("[rvs] STT-Text (endpoint) broadcastet")
|
||||
else:
|
||||
logger.warning("[rvs] STT-Text (endpoint) NICHT broadcastet")
|
||||
except Exception as e:
|
||||
logger.warning("[rvs] STT-Text (endpoint) konnte nicht broadcastet werden: %s", e)
|
||||
|
||||
core_text = self._build_core_text(text, interrupted, location)
|
||||
await self.send_to_core(core_text,
|
||||
source="app-voice-stream" + (" [barge-in]" if interrupted else ""),
|
||||
client_msg_id=client_msg_id)
|
||||
|
||||
async def _stt_remote(self, audio_b64: str, mime_type: str) -> Optional[str]:
|
||||
"""Schickt Audio an die whisper-bridge und wartet auf stt_response.
|
||||
|
||||
|
||||
@@ -357,6 +357,37 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- ARIA-Stream Archiv-Modal: paginierter Browser fuer den
|
||||
persistierten agent_stream.jsonl. Page 1 = juengste Eintraege. -->
|
||||
<div id="aria-archive-modal" style="display:none;position:fixed;top:0;left:0;width:100vw;height:100vh;background:rgba(0,0,0,0.75);z-index:1100;align-items:center;justify-content:center;padding:24px;" onclick="if(event.target===this) closeAriaStreamModal();">
|
||||
<div style="background:#0D0D1A;border:1px solid #1E1E2E;border-radius:12px;width:100%;max-width:1100px;height:85vh;display:flex;flex-direction:column;">
|
||||
<div style="display:flex;align-items:center;padding:12px 14px;border-bottom:1px solid #1E1E2E;gap:8px;flex-wrap:wrap;">
|
||||
<h2 style="margin:0;color:#FFD60A;font-size:15px;flex:1;">📜 ARIA-Stream Archiv <span id="aria-archive-total" style="color:#8888AA;font-weight:normal;"></span></h2>
|
||||
<label style="color:#8888AA;font-size:11px;">Pro Seite:</label>
|
||||
<select id="aria-archive-perpage" onchange="loadAriaArchivePage(1)" style="background:#1A1A2E;color:#E0E0F0;border:1px solid #1E1E2E;border-radius:4px;padding:3px 6px;font-size:11px;">
|
||||
<option value="50">50</option>
|
||||
<option value="100" selected>100</option>
|
||||
<option value="500">500</option>
|
||||
<option value="1000">1000</option>
|
||||
</select>
|
||||
<button class="btn secondary" onclick="loadAriaArchivePage(_ariaArchivePage)" style="padding:3px 10px;font-size:11px;" title="Aktuelle Seite neu laden">↻</button>
|
||||
<button class="btn secondary" onclick="closeAriaStreamModal()" style="padding:3px 12px;font-size:11px;">Schliessen</button>
|
||||
</div>
|
||||
<div style="display:flex;align-items:center;gap:6px;padding:8px 14px;border-bottom:1px solid #1E1E2E;flex-wrap:wrap;">
|
||||
<button class="btn secondary" onclick="loadAriaArchivePage(1)" id="aria-arch-first" style="padding:3px 8px;font-size:11px;" title="Juengste Seite">«</button>
|
||||
<button class="btn secondary" onclick="loadAriaArchivePage(_ariaArchivePage-1)" id="aria-arch-prev" style="padding:3px 8px;font-size:11px;" title="Eine Seite juenger">‹</button>
|
||||
<span id="aria-arch-pageinfo" style="color:#8888AA;font-size:11px;min-width:140px;text-align:center;">Seite ? / ?</span>
|
||||
<button class="btn secondary" onclick="loadAriaArchivePage(_ariaArchivePage+1)" id="aria-arch-next" style="padding:3px 8px;font-size:11px;" title="Eine Seite aelter">›</button>
|
||||
<button class="btn secondary" onclick="loadAriaArchivePage(_ariaArchivePagesTotal)" id="aria-arch-last" style="padding:3px 8px;font-size:11px;" title="Aelteste Seite">»</button>
|
||||
<span style="flex:1;"></span>
|
||||
<span style="color:#555570;font-size:10px;">Seite 1 = neueste · höhere Pages = älter</span>
|
||||
</div>
|
||||
<div id="aria-archive-list" style="flex:1;overflow-y:auto;background:#040408;font-family:'Courier New',monospace;font-size:11px;line-height:1.4;color:#C0C0D0;padding:6px 12px;">
|
||||
<div style="color:#555570;font-style:italic;padding:20px;text-align:center;">Lade...</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Sessions + alter Brain-Viewer entfernt — Memories laufen jetzt
|
||||
komplett ueber den Gehirn-Tab gegen die Vector-DB im aria-brain. -->
|
||||
|
||||
@@ -405,6 +436,7 @@
|
||||
<div id="live-aria-bar" style="display:flex;gap:6px;align-items:center;padding:4px 4px 6px;flex-shrink:0;">
|
||||
<span id="live-aria-status" style="font-size:11px;color:#8888AA;flex:1;">Idle — warte auf ARIA-Aktivitaet</span>
|
||||
<button class="btn" onclick="clearAriaLive()" style="padding:4px 12px;font-size:11px;" title="Live-Mitschrift leeren">Leeren</button>
|
||||
<button class="btn" onclick="openAriaStreamModal()" style="padding:4px 12px;font-size:11px;" title="Komplettes Archiv durchblaettern">📜 Archiv</button>
|
||||
<label style="font-size:11px;color:#8888AA;display:flex;align-items:center;gap:4px;cursor:pointer;" title="Bei jeder neuen Zeile ans Ende scrollen">
|
||||
<input type="checkbox" id="live-aria-autoscroll" checked style="margin:0;"> Auto-Scroll
|
||||
</label>
|
||||
@@ -3035,6 +3067,7 @@
|
||||
document.getElementById('live-desktop').style.display = tab === 'desktop' ? 'block' : 'none';
|
||||
document.getElementById('live-tab-aria').className = 'tab-btn' + (tab === 'aria' ? ' active' : '');
|
||||
document.getElementById('live-tab-desktop').className = 'tab-btn' + (tab === 'desktop' ? ' active' : '');
|
||||
if (tab === 'aria') loadAriaStreamHistory();
|
||||
}
|
||||
|
||||
// ── ARIA Live (read-only Mirror der Claude-Code-Session) ──────
|
||||
@@ -3150,6 +3183,127 @@
|
||||
const el = _ariaStreamEl();
|
||||
if (el) el.innerHTML = '<div style="color:#555570;font-style:italic;">Geleert.</div>';
|
||||
}
|
||||
|
||||
// Beim ersten Tab-Oeffnen / Page-Reload: letzte 200 persistierte Events
|
||||
// aus dem Diagnostic-Server holen. So sind die Live-Bash-Eintraege auch
|
||||
// dann da wenn der Browser im Standby war.
|
||||
let _ariaHistoryLoaded = false;
|
||||
async function loadAriaStreamHistory(lines = 200) {
|
||||
if (_ariaHistoryLoaded) return;
|
||||
_ariaHistoryLoaded = true;
|
||||
try {
|
||||
const r = await fetch('/api/agent-stream?lines=' + lines);
|
||||
if (!r.ok) return;
|
||||
const d = await r.json();
|
||||
const events = d.lines || [];
|
||||
if (!events.length) return;
|
||||
const el = _ariaStreamEl();
|
||||
if (el) {
|
||||
// Placeholder ('Sobald ARIA aktiv...') wegwerfen wenn vorhanden
|
||||
const placeholder = el.querySelector('div[style*="italic"]');
|
||||
if (placeholder) el.removeChild(placeholder);
|
||||
}
|
||||
_ariaPushLine(
|
||||
`<span style="color:#444460;">━━━ ${events.length} fruehere Events (aus ${d.total || '?'} gespeicherten) ━━━</span>`,
|
||||
'#444460',
|
||||
);
|
||||
for (const ev of events) {
|
||||
try { appendAriaStreamEvent(ev); } catch {}
|
||||
}
|
||||
_ariaPushLine(
|
||||
`<span style="color:#444460;">━━━ Ende History — Live ab hier ━━━</span>`,
|
||||
'#444460',
|
||||
);
|
||||
_ariaMaybeScroll();
|
||||
} catch (_) {}
|
||||
}
|
||||
|
||||
// ── ARIA-Stream Archiv-Modal (Pagination) ────────────────
|
||||
let _ariaArchivePage = 1;
|
||||
let _ariaArchivePagesTotal = 1;
|
||||
|
||||
function openAriaStreamModal() {
|
||||
const m = document.getElementById('aria-archive-modal');
|
||||
if (!m) return;
|
||||
m.style.display = 'flex';
|
||||
loadAriaArchivePage(1);
|
||||
}
|
||||
|
||||
function closeAriaStreamModal() {
|
||||
const m = document.getElementById('aria-archive-modal');
|
||||
if (m) m.style.display = 'none';
|
||||
}
|
||||
|
||||
async function loadAriaArchivePage(page) {
|
||||
const listEl = document.getElementById('aria-archive-list');
|
||||
const infoEl = document.getElementById('aria-arch-pageinfo');
|
||||
const totalEl = document.getElementById('aria-archive-total');
|
||||
if (!listEl) return;
|
||||
const perPage = parseInt(document.getElementById('aria-archive-perpage').value, 10) || 100;
|
||||
page = Math.max(1, page || 1);
|
||||
listEl.innerHTML = '<div style="color:#555570;font-style:italic;padding:20px;text-align:center;">Lade...</div>';
|
||||
try {
|
||||
const r = await fetch(`/api/agent-stream?page=${page}&perPage=${perPage}`);
|
||||
if (!r.ok) throw new Error('HTTP ' + r.status);
|
||||
const d = await r.json();
|
||||
const events = d.lines || [];
|
||||
_ariaArchivePage = d.page || page;
|
||||
_ariaArchivePagesTotal = d.pagesTotal || 1;
|
||||
if (totalEl) totalEl.textContent = `(${d.total || 0} Eintraege gesamt)`;
|
||||
if (infoEl) infoEl.textContent = `Seite ${_ariaArchivePage} / ${_ariaArchivePagesTotal}`;
|
||||
// Nav-Buttons enablen/disablen
|
||||
document.getElementById('aria-arch-first').disabled = (_ariaArchivePage <= 1);
|
||||
document.getElementById('aria-arch-prev').disabled = (_ariaArchivePage <= 1);
|
||||
document.getElementById('aria-arch-next').disabled = (_ariaArchivePage >= _ariaArchivePagesTotal);
|
||||
document.getElementById('aria-arch-last').disabled = (_ariaArchivePage >= _ariaArchivePagesTotal);
|
||||
|
||||
if (!events.length) {
|
||||
listEl.innerHTML = '<div style="color:#555570;font-style:italic;padding:20px;text-align:center;">Keine Eintraege auf dieser Seite.</div>';
|
||||
return;
|
||||
}
|
||||
// Eintraege rendern — wir teilen sie in HTML-Snippets analog zu
|
||||
// appendAriaStreamEvent, schreiben aber direkt in den Modal-Container.
|
||||
const html = events.map(p => renderArchiveLine(p)).join('');
|
||||
listEl.innerHTML = html;
|
||||
listEl.scrollTop = listEl.scrollHeight;
|
||||
} catch (e) {
|
||||
listEl.innerHTML = `<div style="color:#FF6B6B;padding:20px;">Fehler beim Laden: ${_ariaEsc(e.message)}</div>`;
|
||||
}
|
||||
}
|
||||
|
||||
function renderArchiveLine(p) {
|
||||
const t = _ariaTimePrefix(p.ts);
|
||||
const kind = p.kind || '';
|
||||
const time = `<span style="color:#777799;">[${t}]</span>`;
|
||||
if (kind === 'start') {
|
||||
return `<div style="color:#444460;">━━━ ${t} session start (${_ariaEsc(p.model||'unknown')}) ━━━</div>`;
|
||||
}
|
||||
if (kind === 'end') {
|
||||
const reason = p.reason || '?';
|
||||
const codePart = (p.code != null) ? ` code=${_ariaEsc(p.code)}` : '';
|
||||
const errPart = p.error ? ` err=${_ariaEsc(String(p.error).slice(0,120))}` : '';
|
||||
return `<div style="color:#444460;">━━━ ${t} session end (${_ariaEsc(reason)}${codePart}${errPart}) ━━━</div>`;
|
||||
}
|
||||
if (kind === 'text') {
|
||||
return `<div style="color:#D0D0E0;white-space:pre-wrap;word-break:break-word;">${time} ${_ariaEsc(p.text || '')}</div>`;
|
||||
}
|
||||
if (kind === 'thinking') {
|
||||
return `<div style="color:#888866;font-style:italic;white-space:pre-wrap;word-break:break-word;">${time} 💭 ${_ariaEsc(p.text || '')}</div>`;
|
||||
}
|
||||
if (kind === 'tool_use') {
|
||||
const name = _ariaEsc(p.name || '?');
|
||||
const inp = _ariaEsc(p.input || '');
|
||||
const tail = p.inputTruncatedBytes ? `<span style="color:#777799;"> ...(+${p.inputTruncatedBytes} bytes)</span>` : '';
|
||||
return `<div style="color:#C0C0D0;white-space:pre-wrap;word-break:break-word;">${time} <span style="color:#0096FF;">▶ ${name}</span> <span style="color:#8888AA;">${inp}${tail}</span></div>`;
|
||||
}
|
||||
if (kind === 'tool_result') {
|
||||
const isError = p.isError === true;
|
||||
const head = isError ? '<span style="color:#FF6B6B;">✗ result (ERROR)</span>' : '<span style="color:#34C759;">✓ result</span>';
|
||||
const tail = p.truncatedBytes ? `<span style="color:#777799;"> ...(+${p.truncatedBytes} bytes)</span>` : '';
|
||||
return `<div style="color:#9090A0;">${time} ${head}<div style="white-space:pre-wrap;padding-left:14px;border-left:2px solid #2A2A3E;margin-top:2px;">${_ariaEsc(p.content || '')}${tail}</div></div>`;
|
||||
}
|
||||
return `<div style="color:#AAAACC;">${time} <span>${_ariaEsc(kind)}: ${_ariaEsc(JSON.stringify(p).slice(0, 500))}</span></div>`;
|
||||
}
|
||||
function ariaPanicStop() {
|
||||
if (!confirm('Wirklich NOT-AUS? Alle aktiven Claude-Subprocesses werden sofort gekillt.')) return;
|
||||
send({ action: 'aria_panic_stop' });
|
||||
@@ -5454,6 +5608,9 @@
|
||||
|
||||
loadThoughtStream();
|
||||
connectWS();
|
||||
// ARIA-Live ist beim Page-Load schon der aktive Sub-Tab.
|
||||
// History gleich nach Seitenstart laden damit Browser-Reload nichts verliert.
|
||||
loadAriaStreamHistory();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
+105
-1
@@ -29,6 +29,40 @@ const RVS_TLS_FALLBACK = process.env.RVS_TLS_FALLBACK || "true";
|
||||
const RVS_TOKEN = process.env.RVS_TOKEN || "";
|
||||
const PROXY_URL = process.env.PROXY_URL || "http://proxy:3456";
|
||||
|
||||
// ── Persistenz fuer agent_stream-Events ──────────────────
|
||||
// Jeder agent_stream-Event wird parallel zum Broadcast in eine .jsonl
|
||||
// geschrieben. Live-View laedt beim Tab-Oeffnen die letzten ~200 Zeilen,
|
||||
// damit Browser-Reload / Standby den Verlauf nicht wegwerfen. Rotation
|
||||
// haendelt logrotate / manual cleanup — wir cappen hier nur weichweich.
|
||||
const AGENT_STREAM_LOG = process.env.AGENT_STREAM_LOG || "/shared/logs/agent_stream.jsonl";
|
||||
const AGENT_STREAM_MAX_BYTES = 50 * 1024 * 1024; // 50 MB → halten den File handlebar
|
||||
function appendAgentStream(payload) {
|
||||
if (!payload || typeof payload !== "object") return;
|
||||
try {
|
||||
const line = JSON.stringify({ ts: Date.now(), ...payload }) + "\n";
|
||||
// Soft-Cap: bei >50 MB ein Truncate auf den letzten ~25 MB Inhalt
|
||||
try {
|
||||
const st = fs.statSync(AGENT_STREAM_LOG);
|
||||
if (st.size > AGENT_STREAM_MAX_BYTES) {
|
||||
const half = Math.floor(AGENT_STREAM_MAX_BYTES / 2);
|
||||
const fd = fs.openSync(AGENT_STREAM_LOG, "r");
|
||||
const buf = Buffer.alloc(half);
|
||||
fs.readSync(fd, buf, 0, half, st.size - half);
|
||||
fs.closeSync(fd);
|
||||
// bis zum naechsten Newline springen damit wir keine halbe Zeile haben
|
||||
const firstNl = buf.indexOf(0x0a);
|
||||
const start = firstNl >= 0 ? firstNl + 1 : 0;
|
||||
fs.writeFileSync(AGENT_STREAM_LOG, buf.slice(start));
|
||||
}
|
||||
} catch {}
|
||||
// Verzeichnis sicherstellen
|
||||
try { fs.mkdirSync(path.dirname(AGENT_STREAM_LOG), { recursive: true }); } catch {}
|
||||
fs.appendFileSync(AGENT_STREAM_LOG, line);
|
||||
} catch (e) {
|
||||
// Schweigend ignorieren — Persistence darf den Stream nicht blockieren
|
||||
}
|
||||
}
|
||||
|
||||
// ── State ───────────────────────────────────────────────
|
||||
const state = {
|
||||
gateway: { status: "disconnected", lastError: null, handshakeOk: false },
|
||||
@@ -637,6 +671,9 @@ function connectRVS(forcePlain) {
|
||||
// Voller Live-Stream der Claude-Code-Session (assistant_text +
|
||||
// tool_use mit Input + tool_result mit truncated Output). Geht
|
||||
// 1:1 an Browser durch — die ARIA-Live-View rendert's.
|
||||
// Zusaetzlich persistieren damit Browser-Reload / Standby den
|
||||
// History-Verlauf nicht wegwirft.
|
||||
try { appendAgentStream(msg.payload); } catch {}
|
||||
broadcast({ type: "agent_stream", payload: msg.payload });
|
||||
} else if (msg.type === "memory_saved") {
|
||||
// ARIA hat selber etwas in die Qdrant-DB gespeichert (via memory_save Tool).
|
||||
@@ -1469,7 +1506,12 @@ const server = http.createServer((req, res) => {
|
||||
log("error", "server", `zip exit ${code}: ${stderr.slice(0, 200)}`);
|
||||
}
|
||||
});
|
||||
req.on("close", () => { if (!zip.killed) zip.kill("SIGTERM"); });
|
||||
// SIGTERM an zip nur wenn der Client wirklich disconnected
|
||||
// (res.close vor res.end). req.on("close") feuert auch wenn
|
||||
// der Request-Body durch ist — das wuerde zip vorzeitig killen.
|
||||
res.on("close", () => {
|
||||
if (!res.writableEnded && !zip.killed) zip.kill("SIGTERM");
|
||||
});
|
||||
});
|
||||
return;
|
||||
} else if (req.url === "/api/files-delete-batch" && req.method === "POST") {
|
||||
@@ -1714,6 +1756,68 @@ const server = http.createServer((req, res) => {
|
||||
});
|
||||
req.pipe(proxyReq);
|
||||
return;
|
||||
} else if (req.url.startsWith("/api/chat-backup") && req.method === "GET") {
|
||||
// Tail des chat_backup.jsonl — fuer Debug-Sessions (was hat ARIA wirklich
|
||||
// gesagt/getan). ?lines=N (Default 200, Max 5000).
|
||||
try {
|
||||
const u = new URL(req.url, "http://localhost");
|
||||
const lines = Math.max(1, Math.min(5000, parseInt(u.searchParams.get("lines") || "200", 10) || 200));
|
||||
const file = "/shared/config/chat_backup.jsonl";
|
||||
let raw = "";
|
||||
try { raw = fs.readFileSync(file, "utf-8"); } catch {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({ ok: true, file, lines: [] }));
|
||||
}
|
||||
const all = raw.split("\n").filter(l => l.trim());
|
||||
const tail = all.slice(-lines);
|
||||
const parsed = tail.map(l => { try { return JSON.parse(l); } catch { return { _raw: l }; } });
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({ ok: true, file, count: parsed.length, total: all.length, lines: parsed }));
|
||||
} catch (e) {
|
||||
res.writeHead(500, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({ ok: false, error: e.message }));
|
||||
}
|
||||
} else if (req.url.startsWith("/api/agent-stream") && req.method === "GET") {
|
||||
// Tail / paginierter Slice des persistierten agent_stream.jsonl.
|
||||
// Modi:
|
||||
// ?lines=N → letzte N Zeilen (Live-View Initial-Load)
|
||||
// ?page=P&perPage=M → 1-indexed Pagination (Modal-Browser);
|
||||
// page=1 = neueste Seite, hoehere Pages = aelter
|
||||
try {
|
||||
const u = new URL(req.url, "http://localhost");
|
||||
const linesParam = u.searchParams.get("lines");
|
||||
const pageParam = u.searchParams.get("page");
|
||||
const perPageParam = u.searchParams.get("perPage");
|
||||
const file = AGENT_STREAM_LOG;
|
||||
let raw = "";
|
||||
try { raw = fs.readFileSync(file, "utf-8"); } catch {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({ ok: true, file, total: 0, lines: [] }));
|
||||
}
|
||||
const all = raw.split("\n").filter(l => l.trim());
|
||||
let slice, page = 1, perPage = 0, pagesTotal = 1;
|
||||
if (pageParam || perPageParam) {
|
||||
perPage = Math.max(10, Math.min(5000, parseInt(perPageParam || "100", 10) || 100));
|
||||
pagesTotal = Math.max(1, Math.ceil(all.length / perPage));
|
||||
page = Math.max(1, Math.min(pagesTotal, parseInt(pageParam || "1", 10) || 1));
|
||||
// page=1 = juengste Seite → vom Ende her slicen
|
||||
const end = all.length - (page - 1) * perPage;
|
||||
const start = Math.max(0, end - perPage);
|
||||
slice = all.slice(start, end);
|
||||
} else {
|
||||
const lines = Math.max(1, Math.min(5000, parseInt(linesParam || "200", 10) || 200));
|
||||
slice = all.slice(-lines);
|
||||
}
|
||||
const parsed = slice.map(l => { try { return JSON.parse(l); } catch { return { _raw: l }; } });
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({
|
||||
ok: true, file, total: all.length, count: parsed.length,
|
||||
page, perPage, pagesTotal, lines: parsed,
|
||||
}));
|
||||
} catch (e) {
|
||||
res.writeHead(500, { "Content-Type": "application/json" });
|
||||
return res.end(JSON.stringify({ ok: false, error: e.message }));
|
||||
}
|
||||
} else if (req.url === "/api/brain-export" && req.method === "GET") {
|
||||
// Komplettes Gehirn als tar.gz streamen.
|
||||
// Schritte: Brain + Qdrant stoppen (saubere Bytes) → tar streamen → wieder starten.
|
||||
|
||||
+4
-7
@@ -20,7 +20,7 @@ services:
|
||||
volumes:
|
||||
- ~/.claude:/root/.claude # Claude CLI Auth (Credentials in /root/.claude/.credentials.json)
|
||||
- ./aria-data/ssh:/root/.ssh # SSH Keys fuer VM-Zugriff (aria-wohnung, rw fuer ARIA)
|
||||
- aria-shared:/shared # Shared Volume fuer Datei-Austausch (Uploads von App)
|
||||
- ./aria-shared:/shared # Shared Volume fuer Datei-Austausch (Uploads von App)
|
||||
- ./proxy-patches:/proxy-patches:ro # Tool-Use-Adapter (ueberschreibt npm-Version, read-only)
|
||||
# Claude Code's eingebautes Auto-Memory liegt in ~/.claude/projects/.
|
||||
# Wir ueberlagern das mit tmpfs damit ARIA nicht parallel zu ARIAs eigener
|
||||
@@ -87,7 +87,7 @@ services:
|
||||
- ./aria-data/brain/data:/data # Memory-Cache + Skills + Models (bind-mount fuer Export)
|
||||
- ./aria-data/brain-import:/import:ro # Quell-MDs fuer den initialen Memory-Import (read-only)
|
||||
- ./aria-data/ssh:/root/.ssh # SSH-Keys fuer aria-wohnung (geteilt mit Proxy)
|
||||
- aria-shared:/shared # gleicher Austausch-Speicher wie Bridge
|
||||
- ./aria-shared:/shared # gleicher Austausch-Speicher wie Bridge
|
||||
restart: unless-stopped
|
||||
networks:
|
||||
- aria-net
|
||||
@@ -103,7 +103,7 @@ services:
|
||||
ports:
|
||||
- "3001:3001" # Diagnostic Web-UI (Diagnostic teilt Netzwerk mit Bridge)
|
||||
volumes:
|
||||
- aria-shared:/shared # Shared Volume fuer Datei-Austausch
|
||||
- ./aria-shared:/shared # Shared Volume fuer Datei-Austausch
|
||||
# Audio-Zugriff
|
||||
- /run/user/1000/pulse:/run/user/1000/pulse
|
||||
- /dev/snd:/dev/snd
|
||||
@@ -132,7 +132,7 @@ services:
|
||||
volumes:
|
||||
- /var/run/docker.sock:/var/run/docker.sock # Container Restart + Brain-Export/Import
|
||||
- ./aria-data/config/diag-state:/data # Persistenter State (aktive Session etc.)
|
||||
- aria-shared:/shared # Shared Volume (Uploads + Config + Voices)
|
||||
- ./aria-shared:/shared # Shared Volume (Uploads + Config + Voices)
|
||||
- ./aria-data/brain:/brain # Brain-Export/Import (tar.gz aus Bind-Mount)
|
||||
environment:
|
||||
- ARIA_AUTH_TOKEN=${ARIA_AUTH_TOKEN:-}
|
||||
@@ -145,9 +145,6 @@ services:
|
||||
- RVS_TOKEN=${RVS_TOKEN:-}
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
aria-shared: # Datei-Austausch zwischen Bridge / Brain / Diagnostic
|
||||
|
||||
networks:
|
||||
aria-net:
|
||||
driver: bridge
|
||||
|
||||
@@ -38,6 +38,10 @@ const ALLOWED_TYPES = new Set([
|
||||
"xtts_delete_voice",
|
||||
"voice_preload", "voice_ready",
|
||||
"stt_request", "stt_response",
|
||||
// Streaming-STT (Phase 1+2): App schickt PCM live an whisper-bridge,
|
||||
// die feuert stt_endpoint mit dem finalen Text — kein Audio-Roundtrip.
|
||||
"stt_stream_start", "stt_audio_chunk", "stt_stream_end",
|
||||
"stt_partial", "stt_endpoint", "stt_stream_done",
|
||||
"service_status",
|
||||
"config_request",
|
||||
"flux_request", "flux_response",
|
||||
|
||||
@@ -375,6 +375,41 @@ async def _send(ws, mtype: str, payload: dict) -> None:
|
||||
logger.warning("Send fehlgeschlagen (%s): %s", mtype, e)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# DEBUG-LOG ueber RVS → /shared/logs/app.log
|
||||
#
|
||||
# Gleiches Pattern wie in whisper-bridge: Stefan's Gamebox ist
|
||||
# Windows (kein SSH), in Zukunft koennten whisper + f5tts auf
|
||||
# unterschiedlichen Hosts laufen. Logs ueber RVS heisst: ein Pfad.
|
||||
#
|
||||
# Toggle via aria-bridge config broadcast: f5ttsDebugLog (bool).
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
_DEBUG_LOG_TO_BRIDGE: bool = False # default OFF — TTS-Renders sind teurer
|
||||
# zu debuggen, normalerweise nicht noetig
|
||||
|
||||
|
||||
async def _debug_log(ws, scope: str, message: str, level: str = "info") -> None:
|
||||
"""Schickt einen app_log via RVS → /shared/logs/app.log mit platform='f5tts'.
|
||||
No-op wenn Toggle aus."""
|
||||
if not _DEBUG_LOG_TO_BRIDGE:
|
||||
return
|
||||
try:
|
||||
await ws.send(json.dumps({
|
||||
"type": "app_log",
|
||||
"payload": {
|
||||
"ts": int(time.time() * 1000),
|
||||
"platform": "f5tts",
|
||||
"level": level,
|
||||
"scope": scope,
|
||||
"message": str(message)[:2000],
|
||||
"stack": "",
|
||||
},
|
||||
"timestamp": int(time.time() * 1000),
|
||||
}))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ── Interne Transkription via whisper-bridge ────────────────
|
||||
|
||||
_pending_stt: dict[str, asyncio.Future] = {}
|
||||
@@ -867,6 +902,30 @@ async def run_loop(runner: F5Runner) -> None:
|
||||
else:
|
||||
fut.set_result(payload.get("text") or "")
|
||||
elif mtype == "config":
|
||||
# Debug-Toggle (gleiche Semantik wie in whisper-bridge)
|
||||
if "f5ttsDebugLog" in payload:
|
||||
global _DEBUG_LOG_TO_BRIDGE
|
||||
old = _DEBUG_LOG_TO_BRIDGE
|
||||
_DEBUG_LOG_TO_BRIDGE = bool(payload.get("f5ttsDebugLog", False))
|
||||
if old != _DEBUG_LOG_TO_BRIDGE:
|
||||
logger.info("Debug-Log-to-Bridge: %s", "ON" if _DEBUG_LOG_TO_BRIDGE else "OFF")
|
||||
# Last gasp wenn ausgeschaltet wird
|
||||
if not _DEBUG_LOG_TO_BRIDGE:
|
||||
try:
|
||||
await ws.send(json.dumps({
|
||||
"type": "app_log",
|
||||
"payload": {
|
||||
"ts": int(time.time() * 1000),
|
||||
"platform": "f5tts",
|
||||
"level": "info",
|
||||
"scope": "config",
|
||||
"message": "debug-log OFF (toggle aus)",
|
||||
"stack": "",
|
||||
},
|
||||
"timestamp": int(time.time() * 1000),
|
||||
}))
|
||||
except Exception:
|
||||
pass
|
||||
# F5-TTS-Settings aktualisieren (Modell, cfg_strength, nfe)
|
||||
async def _update_with_status(p):
|
||||
# Schaut ob ein Modell-Wechsel ansteht — falls ja:
|
||||
@@ -912,6 +971,12 @@ async def run_loop(runner: F5Runner) -> None:
|
||||
continue
|
||||
await asyncio.sleep(min(retry_s, 30))
|
||||
retry_s = min(retry_s * 2, 30)
|
||||
# Sticky-Fallback verhindern: nach jedem Disconnect-Cycle wieder
|
||||
# mit wss anfangen. Sonst klebt der Client nach einem temporaeren
|
||||
# TLS-Hick auf ws:// fest und kommt nie mehr auf wss zurueck —
|
||||
# genau das Problem das die App + Bridge frueher schon hatten.
|
||||
use_tls = RVS_TLS
|
||||
tls_fallback_tried = False
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
|
||||
+448
-25
@@ -2,8 +2,19 @@
|
||||
"""
|
||||
ARIA Whisper Bridge — laeuft auf der Gamebox (RTX 3060).
|
||||
|
||||
Empfaengt stt_request via RVS → FFmpeg-Konvertierung → faster-whisper auf GPU
|
||||
→ sendet stt_response zurueck an die aria-bridge.
|
||||
Zwei Modi:
|
||||
|
||||
1) Legacy One-Shot: stt_request mit komplettem Audio (mp4/wav/ogg base64)
|
||||
→ ffmpeg → faster-whisper → stt_response. Bleibt fuer Fallback/alte App.
|
||||
|
||||
2) Streaming + ML-Endpointer (neu): App schickt live PCM-Chunks waehrend
|
||||
der Aufnahme. Bridge transkribiert alle ~700ms auf dem Ringbuffer und
|
||||
feuert stt_endpoint sobald der Transkript-String N ms nicht mehr
|
||||
waechst. Ersetzt dB/VAD-Stille — endpointet auf SEMANTISCHE Stille,
|
||||
funktioniert im Auto / mit Musik im Hintergrund.
|
||||
|
||||
Erwartetes PCM-Format vom App-Native-Modul: 16 kHz mono s16le (genau
|
||||
das was OpenWakeWord/AudioRecord schon liefert — kein Resampling).
|
||||
|
||||
Env:
|
||||
RVS_HOST, RVS_PORT, RVS_TLS, RVS_TLS_FALLBACK, RVS_TOKEN
|
||||
@@ -21,6 +32,7 @@ import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Optional
|
||||
|
||||
import numpy as np
|
||||
@@ -47,6 +59,13 @@ WHISPER_LANGUAGE = os.getenv("WHISPER_LANGUAGE", "de")
|
||||
|
||||
ALLOWED_MODELS = {"tiny", "base", "small", "medium", "large-v3"}
|
||||
|
||||
# Streaming-Parameter (Defaults — koennen pro Session vom App-Payload ueberschrieben werden)
|
||||
STREAM_TRANSCRIBE_INTERVAL_MS = 700 # alle 700ms transkribieren waehrend Stream laeuft
|
||||
STREAM_DEFAULT_ENDPOINT_MS = 1500 # nach 1.5s ohne neuen Text → Endpoint
|
||||
STREAM_DEFAULT_HARD_CAP_MS = 60000 # nach 60s Audio: harter Cut egal was
|
||||
STREAM_MIN_AUDIO_MS = 600 # erst transkribieren wenn min 600ms Audio da
|
||||
STREAM_SESSION_TTL_S = 120 # tote Sessions nach 2 min aufraeumen
|
||||
|
||||
|
||||
class WhisperRunner:
|
||||
"""Haelt das Whisper-Modell. Hot-Swap bei Konfig-Wechsel via ensure_loaded()."""
|
||||
@@ -55,6 +74,9 @@ class WhisperRunner:
|
||||
self.model_size: str = WHISPER_MODEL
|
||||
self.model: Optional[WhisperModel] = None
|
||||
self._lock = asyncio.Lock()
|
||||
# Serialisiert transcribe()-Calls — faster-whisper ist nicht
|
||||
# parallel-safe auf einer GPU-Instanz, plus VRAM-Fragmentierung.
|
||||
self._transcribe_lock = asyncio.Lock()
|
||||
|
||||
def _load_blocking(self, size: str) -> None:
|
||||
logger.info(
|
||||
@@ -78,19 +100,21 @@ class WhisperRunner:
|
||||
loop = asyncio.get_event_loop()
|
||||
await loop.run_in_executor(None, self._load_blocking, desired_size)
|
||||
|
||||
async def transcribe(self, audio: np.ndarray, language: str) -> tuple[str, float]:
|
||||
async def transcribe(self, audio: np.ndarray, language: str,
|
||||
beam_size: int = 5, vad_filter: bool = True) -> tuple[str, float]:
|
||||
if self.model is None:
|
||||
return "", 0.0
|
||||
|
||||
def _run():
|
||||
segments, info = self.model.transcribe(
|
||||
audio, language=language, beam_size=5, vad_filter=True,
|
||||
audio, language=language, beam_size=beam_size, vad_filter=vad_filter,
|
||||
)
|
||||
text = " ".join(seg.text.strip() for seg in segments)
|
||||
return text, info.duration
|
||||
|
||||
loop = asyncio.get_event_loop()
|
||||
return await loop.run_in_executor(None, _run)
|
||||
async with self._transcribe_lock:
|
||||
return await loop.run_in_executor(None, _run)
|
||||
|
||||
|
||||
def ffmpeg_to_float32(audio_b64: str, mime_type: str) -> np.ndarray:
|
||||
@@ -128,6 +152,14 @@ def ffmpeg_to_float32(audio_b64: str, mime_type: str) -> np.ndarray:
|
||||
pass
|
||||
|
||||
|
||||
def pcm_s16le_to_float32(pcm_bytes: bytes) -> np.ndarray:
|
||||
"""16-bit signed little-endian PCM → float32 in [-1, 1]. Whisper-Format."""
|
||||
if not pcm_bytes:
|
||||
return np.zeros(0, dtype=np.float32)
|
||||
arr = np.frombuffer(pcm_bytes, dtype=np.int16).astype(np.float32) / 32768.0
|
||||
return arr
|
||||
|
||||
|
||||
async def _send(ws, mtype: str, payload: dict) -> None:
|
||||
try:
|
||||
await ws.send(json.dumps({
|
||||
@@ -139,14 +171,326 @@ async def _send(ws, mtype: str, payload: dict) -> None:
|
||||
logger.warning("Send fehlgeschlagen (%s): %s", mtype, e)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# DEBUG-LOG ueber RVS → /shared/logs/app.log
|
||||
#
|
||||
# Stefan's Gamebox ist Windows, kein SSH → wir brauchen Whisper-Bridge-
|
||||
# Logs ueber den gleichen Pfad wie die App: app_log-Messages via RVS,
|
||||
# aria-bridge schreibt sie in /shared/logs/app.log. Diagnostic / App-
|
||||
# Logs-Tab zeigen sie dann mit platform="whisper".
|
||||
#
|
||||
# Toggle via aria-bridge config broadcast: whisperDebugLog (bool).
|
||||
# Default ON solange wir Phase-1/2-Pipeline einfahren — danach
|
||||
# defaultet aria-bridge ihn aus damit kein Spam.
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
_DEBUG_LOG_TO_BRIDGE: bool = True
|
||||
|
||||
|
||||
async def _debug_log(ws, scope: str, message: str, level: str = "info") -> None:
|
||||
"""Schickt einen app_log via RVS → landet in /shared/logs/app.log mit
|
||||
platform='whisper'. Idempotent: wenn Toggle aus → no-op."""
|
||||
if not _DEBUG_LOG_TO_BRIDGE:
|
||||
return
|
||||
try:
|
||||
await ws.send(json.dumps({
|
||||
"type": "app_log",
|
||||
"payload": {
|
||||
"ts": int(time.time() * 1000),
|
||||
"platform": "whisper",
|
||||
"level": level,
|
||||
"scope": scope,
|
||||
"message": str(message)[:2000],
|
||||
"stack": "",
|
||||
},
|
||||
"timestamp": int(time.time() * 1000),
|
||||
}))
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# STREAMING-SESSIONS
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
@dataclass
|
||||
class StreamSession:
|
||||
"""State pro laufendem Streaming-STT-Request."""
|
||||
request_id: str
|
||||
audio_request_id: str
|
||||
language: str
|
||||
model: str
|
||||
endpoint_ms: int
|
||||
hard_cap_ms: int
|
||||
voice: str = "" # echoed back via stt_endpoint fuer ChatScreen → TTS-Override
|
||||
speed: float = 1.0
|
||||
interrupted: bool = False # Barge-In
|
||||
location: Optional[dict] = None
|
||||
sample_rate: int = 16000
|
||||
pcm_buffer: bytearray = field(default_factory=bytearray)
|
||||
started_at: float = field(default_factory=time.time)
|
||||
last_chunk_at: float = field(default_factory=time.time)
|
||||
last_partial: str = ""
|
||||
last_growth_at: float = 0.0
|
||||
last_transcribe_at: float = 0.0
|
||||
closed: bool = False # nach stream_end gesetzt
|
||||
endpoint_sent: bool = False # Endpoint nur einmal feuern
|
||||
|
||||
|
||||
class SessionManager:
|
||||
"""Haelt alle aktiven Streaming-Sessions + Endpointer-Loop."""
|
||||
|
||||
def __init__(self, runner: WhisperRunner) -> None:
|
||||
self.runner = runner
|
||||
self._sessions: dict[str, StreamSession] = {}
|
||||
self._ws = None # wird vom run_loop gesetzt
|
||||
self._loop_task: Optional[asyncio.Task] = None
|
||||
|
||||
def attach_ws(self, ws) -> None:
|
||||
self._ws = ws
|
||||
|
||||
def detach_ws(self) -> None:
|
||||
self._ws = None
|
||||
# Sessions ueberleben Disconnect — der naechste Reconnect kann sie weiter
|
||||
# fuettern, falls die App das gleiche requestId nochmal schickt.
|
||||
# Aber unsere App startet nach Reconnect eine neue Aufnahme; alte Sessions
|
||||
# werden vom Cleanup-Task entsorgt nach STREAM_SESSION_TTL_S.
|
||||
|
||||
def start_session(self, payload: dict) -> Optional[StreamSession]:
|
||||
request_id = payload.get("requestId", "").strip()
|
||||
if not request_id:
|
||||
logger.warning("stt_stream_start ohne requestId — ignoriert")
|
||||
return None
|
||||
if request_id in self._sessions:
|
||||
logger.warning("stt_stream_start: requestId %s schon aktiv — alte Session wird ersetzt",
|
||||
request_id[:8])
|
||||
try:
|
||||
endpoint_ms = int(payload.get("endpointMs") or STREAM_DEFAULT_ENDPOINT_MS)
|
||||
except (TypeError, ValueError):
|
||||
endpoint_ms = STREAM_DEFAULT_ENDPOINT_MS
|
||||
try:
|
||||
hard_cap_ms = int(payload.get("hardCapMs") or STREAM_DEFAULT_HARD_CAP_MS)
|
||||
except (TypeError, ValueError):
|
||||
hard_cap_ms = STREAM_DEFAULT_HARD_CAP_MS
|
||||
try:
|
||||
speed = float(payload.get("speed") or 1.0)
|
||||
except (TypeError, ValueError):
|
||||
speed = 1.0
|
||||
session = StreamSession(
|
||||
request_id=request_id,
|
||||
audio_request_id=payload.get("audioRequestId", "") or "",
|
||||
language=payload.get("language") or WHISPER_LANGUAGE,
|
||||
model=payload.get("model") or self.runner.model_size or WHISPER_MODEL,
|
||||
endpoint_ms=endpoint_ms,
|
||||
hard_cap_ms=hard_cap_ms,
|
||||
voice=payload.get("voice", "") or "",
|
||||
speed=speed,
|
||||
interrupted=bool(payload.get("interrupted", False)),
|
||||
location=payload.get("location") or None,
|
||||
sample_rate=int(payload.get("sampleRate") or 16000),
|
||||
)
|
||||
self._sessions[request_id] = session
|
||||
logger.info("Stream-Session offen: id=%s lang=%s model=%s endpointMs=%d hardCapMs=%d voice=%r",
|
||||
request_id[:8], session.language, session.model,
|
||||
session.endpoint_ms, session.hard_cap_ms, session.voice or "(default)")
|
||||
return session
|
||||
|
||||
def feed_chunk(self, payload: dict) -> bool:
|
||||
request_id = payload.get("requestId", "")
|
||||
session = self._sessions.get(request_id)
|
||||
if session is None or session.closed:
|
||||
return False
|
||||
pcm_b64 = payload.get("pcm", "")
|
||||
if not pcm_b64:
|
||||
return False
|
||||
try:
|
||||
pcm_bytes = base64.b64decode(pcm_b64)
|
||||
except Exception:
|
||||
logger.warning("Stream %s: ungueltige base64-PCM-Daten", request_id[:8])
|
||||
return False
|
||||
session.pcm_buffer.extend(pcm_bytes)
|
||||
session.last_chunk_at = time.time()
|
||||
return True
|
||||
|
||||
def end_session(self, request_id: str) -> Optional[StreamSession]:
|
||||
"""Markiert Session als geschlossen. Der Endpointer-Loop macht das
|
||||
Final-Transcribe + Cleanup."""
|
||||
session = self._sessions.get(request_id)
|
||||
if session is None:
|
||||
return None
|
||||
session.closed = True
|
||||
return session
|
||||
|
||||
def drop(self, request_id: str) -> None:
|
||||
self._sessions.pop(request_id, None)
|
||||
|
||||
async def run_endpointer(self) -> None:
|
||||
"""Background-Loop: alle ~200ms ueber alle Sessions iterieren."""
|
||||
logger.info("Endpointer-Loop gestartet (transcribe-interval=%dms, default-endpoint=%dms)",
|
||||
STREAM_TRANSCRIBE_INTERVAL_MS, STREAM_DEFAULT_ENDPOINT_MS)
|
||||
while True:
|
||||
await asyncio.sleep(0.2)
|
||||
now = time.time()
|
||||
# Snapshot — sonst RuntimeError wenn wir waehrend Iteration sessions[]
|
||||
# mutieren (Endpoint-Drop).
|
||||
for sid, sess in list(self._sessions.items()):
|
||||
try:
|
||||
await self._tick_session(sess, now)
|
||||
except Exception:
|
||||
logger.exception("Endpointer-Tick crashed (session=%s)", sid[:8])
|
||||
|
||||
# Cleanup: tote Sessions (ohne Chunk seit STREAM_SESSION_TTL_S)
|
||||
for sid, sess in list(self._sessions.items()):
|
||||
if now - sess.last_chunk_at > STREAM_SESSION_TTL_S:
|
||||
logger.info("Stream %s: TTL ueberschritten (ohne Daten seit %.0fs) — drop",
|
||||
sid[:8], now - sess.last_chunk_at)
|
||||
self.drop(sid)
|
||||
|
||||
async def _tick_session(self, sess: StreamSession, now: float) -> None:
|
||||
ws = self._ws
|
||||
if ws is None:
|
||||
return # disconnected — Endpointer pausiert bis Reconnect
|
||||
|
||||
audio_ms = self._buffer_duration_ms(sess)
|
||||
|
||||
# Hard-Cap erreicht → wie Endpoint behandeln (egal ob neuer Text)
|
||||
elapsed_ms = (now - sess.started_at) * 1000.0
|
||||
if elapsed_ms > sess.hard_cap_ms and not sess.endpoint_sent and not sess.closed:
|
||||
logger.info("Stream %s: HardCap %dms erreicht — forciere Endpoint",
|
||||
sess.request_id[:8], sess.hard_cap_ms)
|
||||
await self._finalize(sess, ws, reason="hardcap")
|
||||
return
|
||||
|
||||
# Closed (stream_end empfangen) → finalisieren mit dem gesammelten Buffer
|
||||
if sess.closed and not sess.endpoint_sent:
|
||||
await self._finalize(sess, ws, reason="stream_end")
|
||||
return
|
||||
|
||||
# Noch zu wenig Audio fuer eine erste Transkription
|
||||
if audio_ms < STREAM_MIN_AUDIO_MS:
|
||||
return
|
||||
|
||||
# Transcribe-Throttling
|
||||
since_last = (now - sess.last_transcribe_at) * 1000.0
|
||||
if since_last < STREAM_TRANSCRIBE_INTERVAL_MS:
|
||||
return
|
||||
|
||||
sess.last_transcribe_at = now
|
||||
try:
|
||||
audio = pcm_s16le_to_float32(bytes(sess.pcm_buffer))
|
||||
except Exception:
|
||||
logger.exception("Stream %s: PCM-Decode fehlgeschlagen", sess.request_id[:8])
|
||||
return
|
||||
|
||||
try:
|
||||
# Kleinere beam_size fuer Streaming-Partials — wir wollen Latenz,
|
||||
# nicht maximale Genauigkeit. Final-Transcribe (in _finalize) faehrt
|
||||
# dann mit beam_size=5.
|
||||
text, _dur = await self.runner.transcribe(audio, sess.language, beam_size=1, vad_filter=True)
|
||||
except Exception:
|
||||
logger.exception("Stream %s: Partial-Transcribe crashed", sess.request_id[:8])
|
||||
return
|
||||
|
||||
text = text.strip()
|
||||
grew = bool(text) and text != sess.last_partial
|
||||
if grew:
|
||||
sess.last_partial = text
|
||||
sess.last_growth_at = now
|
||||
# Optional: stt_partial broadcasten fuer UI-Feedback. Wir schicken's
|
||||
# mit damit Diagnostic / ChatScreen Live-Text zeigen kann.
|
||||
await _send(ws, "stt_partial", {
|
||||
"requestId": sess.request_id,
|
||||
"audioRequestId": sess.audio_request_id,
|
||||
"text": text,
|
||||
})
|
||||
await _debug_log(ws, "stream.partial",
|
||||
f"id={sess.request_id[:12]} text={text[:80]!r}")
|
||||
else:
|
||||
# Stagnation pruefen — Endpoint-Bedingung
|
||||
if sess.last_growth_at == 0.0:
|
||||
# Noch gar kein Text erkannt. Wenn der User gar nichts sagt
|
||||
# springt Brain irgendwann aus eigenem Conversation-Window-
|
||||
# Timeout in der App raus; wir machen hier nix.
|
||||
return
|
||||
silence_ms = (now - sess.last_growth_at) * 1000.0
|
||||
if silence_ms >= sess.endpoint_ms and not sess.endpoint_sent:
|
||||
logger.info("Stream %s: Endpoint nach %dms ohne neuen Text — Text=%r",
|
||||
sess.request_id[:8], int(silence_ms), sess.last_partial[:80])
|
||||
await self._finalize(sess, ws, reason="endpoint")
|
||||
|
||||
def _buffer_duration_ms(self, sess: StreamSession) -> float:
|
||||
# 16-bit s16le mono → 2 bytes pro Sample
|
||||
samples = len(sess.pcm_buffer) // 2
|
||||
if samples == 0:
|
||||
return 0.0
|
||||
return (samples / sess.sample_rate) * 1000.0
|
||||
|
||||
async def _finalize(self, sess: StreamSession, ws, reason: str) -> None:
|
||||
"""Endgueltige Transkription auf dem vollen Buffer (beam_size=5),
|
||||
feuert stt_endpoint + stt_stream_done, droppt Session."""
|
||||
if sess.endpoint_sent:
|
||||
return
|
||||
sess.endpoint_sent = True
|
||||
audio = pcm_s16le_to_float32(bytes(sess.pcm_buffer))
|
||||
if audio.size == 0:
|
||||
logger.info("Stream %s: leere Audio-Daten — final text leer", sess.request_id[:8])
|
||||
final_text = ""
|
||||
stt_ms = 0
|
||||
duration_s = 0.0
|
||||
else:
|
||||
t0 = time.time()
|
||||
try:
|
||||
final_text, _dur = await self.runner.transcribe(audio, sess.language, beam_size=5, vad_filter=True)
|
||||
except Exception:
|
||||
logger.exception("Stream %s: Final-Transcribe crashed", sess.request_id[:8])
|
||||
final_text = sess.last_partial # fallback auf letzten Partial
|
||||
stt_ms = int((time.time() - t0) * 1000)
|
||||
duration_s = audio.size / 16000.0
|
||||
final_text = final_text.strip()
|
||||
|
||||
logger.info("Stream %s: FINAL (reason=%s, %.1fs Audio, %dms): %r",
|
||||
sess.request_id[:8], reason, duration_s, stt_ms, final_text[:120])
|
||||
await _debug_log(ws, "stream.final",
|
||||
f"id={sess.request_id[:12]} reason={reason} "
|
||||
f"audio={duration_s:.1f}s stt={stt_ms}ms text={final_text[:80]!r}")
|
||||
|
||||
# stt_endpoint: das ist DAS Event auf das aria-bridge horcht fuer den
|
||||
# Brain-Shortcut. Enthaelt alle Felder die bisher in 'audio' lagen,
|
||||
# ohne den Audio-Roundtrip (App → aria-bridge → whisper → aria-bridge).
|
||||
endpoint_payload = {
|
||||
"requestId": sess.request_id,
|
||||
"audioRequestId": sess.audio_request_id,
|
||||
"text": final_text,
|
||||
"reason": reason,
|
||||
"durationS": duration_s,
|
||||
"sttMs": stt_ms,
|
||||
"voice": sess.voice,
|
||||
"speed": sess.speed,
|
||||
"interrupted": sess.interrupted,
|
||||
}
|
||||
if sess.location:
|
||||
endpoint_payload["location"] = sess.location
|
||||
await _send(ws, "stt_endpoint", endpoint_payload)
|
||||
|
||||
# stt_stream_done: an die App — damit sie ihre Recording-State-Machine
|
||||
# zurueck auf armed setzt (Mikro aus, ggf. Wake-Word wieder an).
|
||||
await _send(ws, "stt_stream_done", {
|
||||
"requestId": sess.request_id,
|
||||
"audioRequestId": sess.audio_request_id,
|
||||
"text": final_text,
|
||||
"reason": reason,
|
||||
})
|
||||
|
||||
self.drop(sess.request_id)
|
||||
|
||||
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# LEGACY ONE-SHOT (unveraendert)
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
async def handle_stt_request(ws, payload: dict, runner: WhisperRunner) -> None:
|
||||
request_id = payload.get("requestId", "")
|
||||
audio_b64 = payload.get("audio", "")
|
||||
mime_type = payload.get("mimeType", "audio/mp4")
|
||||
# Modell-Auswahl:
|
||||
# payload.model gesetzt → nimm das (aria-bridge sendet's basierend auf Config)
|
||||
# sonst + Modell geladen → behalt das aktuelle (kein sinnloser Swap)
|
||||
# sonst → fallback auf ENV-Default
|
||||
model = payload.get("model") or (runner.model_size if runner.model is not None else WHISPER_MODEL)
|
||||
language = payload.get("language") or WHISPER_LANGUAGE
|
||||
|
||||
@@ -156,8 +500,6 @@ async def handle_stt_request(ws, payload: dict, runner: WhisperRunner) -> None:
|
||||
|
||||
try:
|
||||
t_load = time.time()
|
||||
# Falls Modell noch nicht geladen (Race-Condition: stt_request vor config)
|
||||
# → Status-Broadcast loading→ready damit der App-Banner aufpoppt
|
||||
needs_load = runner.model is None or runner.model_size != model
|
||||
if needs_load:
|
||||
await _broadcast_status(ws, "loading", model=model)
|
||||
@@ -205,7 +547,11 @@ async def _broadcast_status(ws, state: str, **extra) -> None:
|
||||
await _send(ws, "service_status", payload)
|
||||
|
||||
|
||||
async def run_loop(runner: WhisperRunner) -> None:
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
# WS-LOOP
|
||||
# ──────────────────────────────────────────────────────────────
|
||||
|
||||
async def run_loop(runner: WhisperRunner, sessions: SessionManager) -> None:
|
||||
use_tls = RVS_TLS
|
||||
retry_s = 2
|
||||
tls_fallback_tried = False
|
||||
@@ -216,20 +562,12 @@ async def run_loop(runner: WhisperRunner) -> None:
|
||||
masked = url.replace(RVS_TOKEN, "***") if RVS_TOKEN else url
|
||||
try:
|
||||
logger.info("Verbinde zu RVS: %s", masked)
|
||||
# max_size 50MB damit grosse stt_request (Voice-Cloning-WAVs als
|
||||
# base64 koennen mehrere MB werden) nicht das Frame-Limit sprengen
|
||||
# und die Verbindung mit 1009 'message too big' killen.
|
||||
async with websockets.connect(url, ping_interval=20, ping_timeout=10, max_size=50 * 1024 * 1024) as ws:
|
||||
logger.info("RVS verbunden")
|
||||
retry_s = 2
|
||||
tls_fallback_tried = False
|
||||
sessions.attach_ws(ws)
|
||||
|
||||
# Initialer Status-Broadcast — uebertont alten "ready"-State
|
||||
# im App/Diagnostic Banner (sonst denkt der User noch alles ist
|
||||
# gut von vorher). Wenn Modell schon geladen → ready, sonst
|
||||
# loading mit aktuellem (Default-)Namen.
|
||||
# Plus: config_request an aria-bridge — wir wissen nicht ob
|
||||
# sie auch grad reconnected hat oder schon laenger online ist.
|
||||
async def _initial_handshake():
|
||||
try:
|
||||
if runner.model is not None:
|
||||
@@ -241,6 +579,11 @@ async def run_loop(runner: WhisperRunner) -> None:
|
||||
await _broadcast_status(ws, "loading", model=init_model)
|
||||
logger.info("Initial: sende config_request an aria-bridge")
|
||||
await _send(ws, "config_request", {"service": "whisper"})
|
||||
# Startup-Marker — App-Logs zeigen damit ob Streaming-Code
|
||||
# ueberhaupt aktiv ist (Stefan baut auf Gamebox via PS,
|
||||
# Build/Restart kann unbeabsichtigt alte Version weiterfahren).
|
||||
await _debug_log(ws, "boot",
|
||||
"whisper-bridge online — streaming-mode ENABLED, debug-log ON")
|
||||
except Exception as e:
|
||||
logger.exception("Initial-Handshake crashed: %s", e)
|
||||
asyncio.create_task(_initial_handshake())
|
||||
@@ -259,9 +602,84 @@ async def run_loop(runner: WhisperRunner) -> None:
|
||||
logger.info("stt_request empfangen (id=%s, %dKB Audio)",
|
||||
req_id[:8] if req_id != "?" else "?", audio_len // 1365)
|
||||
asyncio.create_task(handle_stt_request(ws, payload, runner))
|
||||
|
||||
elif mtype == "stt_stream_start":
|
||||
await _debug_log(ws, "stream.start",
|
||||
f"received id={payload.get('requestId', '?')[:12]} "
|
||||
f"audioReqId={payload.get('audioRequestId', '?')[:16]} "
|
||||
f"endpointMs={payload.get('endpointMs')} "
|
||||
f"hardCapMs={payload.get('hardCapMs')}")
|
||||
# Ggf. Modell sicherstellen — sonst antwortet der erste
|
||||
# transcribe-Call mit Leerstring weil Model None.
|
||||
target_model = payload.get("model") or runner.model_size or WHISPER_MODEL
|
||||
needs_load = (runner.model is None) or (target_model != runner.model_size)
|
||||
if needs_load:
|
||||
async def _load_then_start(p, target):
|
||||
await _broadcast_status(ws, "loading", model=target)
|
||||
try:
|
||||
await runner.ensure_loaded(target)
|
||||
await _broadcast_status(ws, "ready", model=runner.model_size)
|
||||
except Exception as e:
|
||||
await _broadcast_status(ws, "error", error=str(e)[:200])
|
||||
return
|
||||
sessions.start_session(p)
|
||||
asyncio.create_task(_load_then_start(payload, target_model))
|
||||
else:
|
||||
sessions.start_session(payload)
|
||||
|
||||
elif mtype == "stt_audio_chunk":
|
||||
ok = sessions.feed_chunk(payload)
|
||||
if not ok:
|
||||
# Sehr verbose im Schlimmstfall — debug-Level reicht.
|
||||
logger.debug("stt_audio_chunk: unbekannte/closed session %s",
|
||||
payload.get("requestId", "")[:8])
|
||||
await _debug_log(ws, "stream.chunk.reject",
|
||||
f"unknown/closed session id={payload.get('requestId', '?')[:12]}",
|
||||
level="warn")
|
||||
else:
|
||||
# Nur alle 25 Chunks loggen (=5s Audio) — sonst Spam.
|
||||
try:
|
||||
seq = int(payload.get("seq", 0) or 0)
|
||||
if seq % 25 == 0:
|
||||
await _debug_log(ws, "stream.chunk",
|
||||
f"id={payload.get('requestId', '?')[:12]} seq={seq}")
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
|
||||
elif mtype == "stt_stream_end":
|
||||
req_id = payload.get("requestId", "")
|
||||
logger.info("stt_stream_end empfangen: id=%s reason=%s",
|
||||
req_id[:8], payload.get("reason", ""))
|
||||
await _debug_log(ws, "stream.end",
|
||||
f"received id={req_id[:12]} reason={payload.get('reason', '')}")
|
||||
sessions.end_session(req_id)
|
||||
|
||||
elif mtype == "config":
|
||||
# Debug-Toggle: aria-bridge broadcastet jetzt whisperDebugLog
|
||||
# damit Stefan im laufenden Betrieb via Diagnostic-Settings
|
||||
# die Logs an/aus schalten kann.
|
||||
if "whisperDebugLog" in payload:
|
||||
global _DEBUG_LOG_TO_BRIDGE
|
||||
old = _DEBUG_LOG_TO_BRIDGE
|
||||
_DEBUG_LOG_TO_BRIDGE = bool(payload.get("whisperDebugLog", False))
|
||||
if old != _DEBUG_LOG_TO_BRIDGE:
|
||||
logger.info("Debug-Log-to-Bridge: %s", "ON" if _DEBUG_LOG_TO_BRIDGE else "OFF")
|
||||
# Last gasp wenn ausgeschaltet wird damit Stefan im Log sieht
|
||||
# dass der Toggle griff.
|
||||
if not _DEBUG_LOG_TO_BRIDGE:
|
||||
await ws.send(json.dumps({
|
||||
"type": "app_log",
|
||||
"payload": {
|
||||
"ts": int(time.time() * 1000),
|
||||
"platform": "whisper",
|
||||
"level": "info",
|
||||
"scope": "config",
|
||||
"message": "debug-log OFF (toggle aus)",
|
||||
"stack": "",
|
||||
},
|
||||
"timestamp": int(time.time() * 1000),
|
||||
}))
|
||||
new_model = payload.get("whisperModel") or WHISPER_MODEL
|
||||
# Laden wenn (a) noch nix geladen, oder (b) Modell wechselt
|
||||
needs_load = (runner.model is None) or (new_model != runner.model_size)
|
||||
if needs_load:
|
||||
logger.info("Config-Broadcast: Whisper-Modell -> %s%s",
|
||||
@@ -280,11 +698,10 @@ async def run_loop(runner: WhisperRunner) -> None:
|
||||
await _broadcast_status(ws, "error", error=str(e)[:200])
|
||||
asyncio.create_task(_swap_with_status(new_model))
|
||||
else:
|
||||
# Alle anderen Nachrichten debug-loggen — hilft beim Diagnostizieren,
|
||||
# ob stt_request ueberhaupt durch den RVS kommt
|
||||
logger.debug("Unbeachteter Type: %s", mtype)
|
||||
except Exception as e:
|
||||
logger.warning("Verbindung verloren: %s", e)
|
||||
sessions.detach_ws()
|
||||
if use_tls and RVS_TLS_FALLBACK and not tls_fallback_tried:
|
||||
logger.info("TLS-Verbindung fehlgeschlagen — Fallback auf ws://")
|
||||
use_tls = False
|
||||
@@ -292,6 +709,8 @@ async def run_loop(runner: WhisperRunner) -> None:
|
||||
continue
|
||||
await asyncio.sleep(min(retry_s, 30))
|
||||
retry_s = min(retry_s * 2, 30)
|
||||
use_tls = RVS_TLS
|
||||
tls_fallback_tried = False
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
@@ -299,7 +718,11 @@ async def main() -> None:
|
||||
logger.error("RVS_HOST ist nicht gesetzt — Abbruch")
|
||||
sys.exit(1)
|
||||
runner = WhisperRunner()
|
||||
await run_loop(runner)
|
||||
sessions = SessionManager(runner)
|
||||
# Endpointer-Loop nebenbei laufen lassen — er pruefst _ws is None und
|
||||
# schlaeft solange das nicht gesetzt ist.
|
||||
asyncio.create_task(sessions.run_endpointer())
|
||||
await run_loop(runner, sessions)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
Reference in New Issue
Block a user