Compare commits

..

67 Commits

Author SHA1 Message Date
duffyduck cd390a4115 release: bump version to 0.0.3.8 2026-04-18 11:41:12 +02:00
duffyduck a65ed579d2 feat: Whisper model selector + 16kHz mono recording
- App: AudioSamplingRateAndroid 16000 + AudioChannelsAndroid 1
  → Whisper bekommt direkt sein Ziel-Format, kein Resample mehr
- Bridge: STTEngine.reload() laedt Modell zur Laufzeit neu
  (tiny/base/small/medium/large-v3)
- Bridge: Config-Message triggert Hot-Reload wenn whisperModel sich aendert
- Bridge: Default auf 'medium' (besser als 'small' bei aehnlicher Latenz)
- Diagnostic: Neue Sektion "Whisper (Spracherkennung)" mit Dropdown,
  auto-save bei Auswahl, beim Laden wird der gespeicherte Wert gesetzt
- Diagnostic/Server: send_voice_config merged whisperModel in voice_config.json
- aria.env.example: WHISPER_MODEL + WHISPER_LANGUAGE dokumentiert

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:37:27 +02:00
duffyduck 2ad1f57382 feat: Thinking indicator + cancel button in the app
- Bridge: _emit_activity() spiegelt OpenClaw agent events als agent_activity
  an RVS, dedupliziert State-Wechsel. chat:final/error senden idle.
- Bridge: Neuer cancel_request-Handler ruft Diagnostic /api/cancel per HTTP.
- Diagnostic: Neuer POST /api/cancel Endpoint (gleiche Logik wie WS-Cancel).
- RVS: agent_activity + cancel_request in ALLOWED_TYPES.
- App: Gelber Indicator ueber der Input-Bar mit Text je nach Activity,
  roter Abbrechen-Button. Cancel sendet cancel_request via RVS.
- issue.md: Erledigte Bugfixes + Features konsolidiert.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:22:02 +02:00
duffyduck 58e3cfd3e6 feat: Session export as markdown in Diagnostic
- ⬇ Button per Session-Zeile — exportiert auch inaktive Sessions
- Server parst JSONL, extrahiert User/Assistant-Nachrichten mit Timestamp
- Metadata-Prefix wird entfernt, Markdown mit # Session-Header generiert
- Browser-Download via Blob + download-Attribut

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:14:15 +02:00
duffyduck 7de4ee8f5b fix: Stuck "ARIA denkt..." indicator after pipeline ends
- pipelineEnd() now broadcasts agent_activity: idle unconditionally
- chat:error and chat:final paths broadcast idle outside of active pipeline
- Gateway close event ends active pipeline + broadcasts idle
- Prevents indicator from hanging after timeout/error/disconnect

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:11:12 +02:00
duffyduck 213edac3a7 fix: Session persistence - respect user choice across container restarts
- sessionFromFile flag prevents auto-pick after first start
- Atomic write (temp + rename) with loud error logging
- Auto-pick filters out aria-bridge/aria-diagnostic when user sessions exist
- handleSetActiveSession reports persistence failures to client

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:03:26 +02:00
duffyduck acc13aef6b fix: Speech gate - only send recording if actual speech detected
- VAD_SPEECH_THRESHOLD_DB = -35 (louder than silence threshold)
- Needs 300ms of speech before counting as real speech
- Recording discarded if only background noise detected
- Prevents sending garbage to Whisper in conversation mode

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 18:20:05 +02:00
duffyduck 4bbc6f7787 release: bump version to 0.0.3.7 2026-04-11 13:18:17 +02:00
duffyduck 20f2ea1829 fix: Conversation mode starts recording immediately when ear button tapped
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 13:15:26 +02:00
duffyduck 2d23f0668b docs: update README with conversation mode, multi-attachments, markdown cleanup
- Conversation mode (ear button) documented in App Features
- Multiple attachments + paste support
- Markdown cleanup for TTS
- Auto-Update FileProvider + check button
- Roadmap: 22 items in Phase 1 completed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:43:09 +02:00
duffyduck d6030a06b7 docs: update issue.md - move completed items, clean up open list
28 items completed, 10 remaining open

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:23:04 +02:00
duffyduck 0df76e2af6 release: bump version to 0.0.3.6 2026-04-11 12:19:00 +02:00
duffyduck f80fe1df93 fix: Inverted FlatList - newest messages always visible at bottom
- No more scrollToEnd/scrollToIndex needed
- FlatList inverted=true with reversed data
- New messages appear at bottom automatically
- User scrolls up to see history (natural chat behavior)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:17:32 +02:00
duffyduck cff421bc53 release: bump version to 0.0.3.5 2026-04-11 12:13:41 +02:00
duffyduck bca925d385 fix: Use scrollToIndex with viewPosition:1 for reliable bottom scroll
- scrollToIndex targets last message at bottom of viewport
- onScrollToIndexFailed fallback to scrollToEnd
- More reliable than scrollToEnd with dynamic heights

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:12:24 +02:00
duffyduck 9abde89805 release: bump version to 0.0.3.4 2026-04-11 12:09:23 +02:00
duffyduck ea4f639fcb fix: Auto-scroll retry with multiple delays (100, 300, 600, 1000ms)
FlatList needs time to render - single setTimeout(150) was unreliable.
Now tries 4 times on initial load, 2 times for new messages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:07:54 +02:00
duffyduck 64cd5f7d52 release: bump version to 0.0.3.3 2026-04-11 12:04:37 +02:00
duffyduck 843ebe1d8f fix: Remove duplicate closure ending in ChatScreen (build error)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 12:03:20 +02:00
duffyduck 764619f076 fix: Comprehensive markdown/formatting cleanup for TTS (Piper + XTTS)
- Remove **bold**, *italic*, `code`, code blocks, links, headers, quotes, lists
- Replace newlines with natural pauses (period/comma)
- Remove quotation marks, empty brackets
- Fixes text being swallowed/garbled by TTS engines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:47:04 +02:00
duffyduck e3a0cfb55a docs: mark conversation mode as done, keep Porcupine as Phase 2
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:42:33 +02:00
duffyduck 2929749314 feat: Conversation mode (ear button) - auto-record after ARIA speaks
- Ear button activates conversation mode (green dot)
- After TTS playback finishes → 800ms pause → auto-start recording
- VAD stops recording on silence → sends to ARIA → ARIA answers → TTS → loop
- Like a natural conversation / walkie-talkie mode
- Audio service fires onPlaybackFinished when queue empty

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:40:55 +02:00
duffyduck 51b9512f4e docs: mark scroll bugs as fixed in issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:37:53 +02:00
duffyduck ffcfa44eef fix: Auto-scroll to last message on app start + new messages
- useEffect on messages array instead of onContentSizeChange
- Instant jump (no animation) when loading history
- Animated scroll for single new messages
- Scroll pauses when user scrolls up, resumes at bottom

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:37:30 +02:00
duffyduck 6363da97b1 feat: Multiple attachments + paste support (App + Diagnostic)
App:
- Multiple pending attachments (horizontal scroll preview)
- Individual remove (X) or clear all
- Send button shows when any attachment pending
- All files sent before text message

Diagnostic:
- Clip icon for file selection (multiple)
- Paste images/files from clipboard (Ctrl+V)
- Pending preview with thumbnails
- Files sent via RVS before text message

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 11:34:33 +02:00
duffyduck 07ed2cdcf6 docs: mark attachment text feature as done in issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 10:06:13 +02:00
duffyduck 5ad68b7dfc feat: Attachments not sent immediately - add text/voice before sending
- File/photo selection stores as pending (not sent immediately)
- Preview bar shows pending attachment above input field
- User can add text message before sending (e.g. "Was siehst du?")
- Send button appears when attachment is pending (even without text)
- Placeholder changes to "Text zum Anhang (optional)..."
- X button to cancel pending attachment
- File + text sent together (file first, then chat message)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 10:05:50 +02:00
duffyduck 8a6ee018ea docs: mark text message bug as fixed in issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 09:59:48 +02:00
duffyduck b42590ff95 docs: mark auto-update bugs as fixed in issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 09:54:57 +02:00
duffyduck 056b579c47 release: bump version to 0.0.3.2 2026-04-11 09:53:54 +02:00
duffyduck 576e612cd0 fix: release.sh clears Metro + Gradle cache before build (version consistency)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-11 09:51:41 +02:00
duffyduck c2faa06a15 release: bump version to 0.0.3.1 2026-04-10 23:19:40 +02:00
duffyduck d3ed3556eb fix: Bridge chat handler was missing send_to_core (text messages ignored)
The chat handler checked sender but never forwarded the text to aria-core.
Only voice messages worked because they went through the audio→STT→send_to_core path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 23:13:29 +02:00
duffyduck d960d125c0 release: bump version to 0.0.3.0 2026-04-10 09:07:20 +02:00
duffyduck 89d5d7ec0a release: bump version to 0.0.2.9 2026-04-10 09:01:47 +02:00
duffyduck ea0c13936b fix: release.sh deletes old APKs on RVS before uploading new one
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 09:00:45 +02:00
duffyduck 773c976822 fix: Auto-update APK install via FileProvider + dynamic version
- Native ApkInstallerModule: FileProvider content:// URI for Android 7+
- REQUEST_INSTALL_PACKAGES permission in AndroidManifest
- file_paths.xml for FileProvider cache access
- APP_VERSION reads from package.json (not hardcoded)
- "Auf Updates pruefen" button in Settings
- Version display reads from package.json dynamically

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:59:52 +02:00
duffyduck cd05ed2379 docs: add auto-update FileProvider bug + update check button to issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:55:30 +02:00
duffyduck 054e4057d8 release: bump version to 0.0.2.8 2026-04-10 08:49:47 +02:00
duffyduck 3943e79bb1 docs: document .env.example with detailed comments, explain both tokens in README
- ARIA_AUTH_TOKEN: Gateway auth (who can talk to ARIA)
- RVS_TOKEN: Pairing token (same room in RVS relay)
- RVS_UPDATE_HOST: SSH target for auto-update APK copy
- All variables with German comments and examples

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:45:26 +02:00
duffyduck 87f4317c15 docs: add auto-update APK not reaching RVS bug to issue.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:39:59 +02:00
duffyduck 50aa793910 fix: Proxy SSH volume read-write (ARIA can manage keys without -F workaround)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:34:35 +02:00
duffyduck 5efc9865a8 docs: add 6 new bugs/features to issue.md
- Session persistence on container restart
- App: text/image/attachment messages not working (only voice)
- App: audio stops randomly
- App: auto-scroll to last message on start + new messages
- App: add text/voice to attachments
- Prioritized bugs section

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 08:31:49 +02:00
duffyduck 949c573c49 fix: XTTS chunk size 150 chars (faster render, preload overlaps playback)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:52:56 +02:00
duffyduck f7f450a09d fix: XTTS streaming mode - send each chunk immediately, comma between sentences
- Back to streaming: render chunk → send immediately → next chunk
- App plays with preloading queue (no waiting for all chunks)
- Comma instead of dot between sentences in chunk (no "Punkt" read aloud)
- Sentence-ending dots already removed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:48:50 +02:00
duffyduck 81f7c38383 fix: XTTS splits concatenated audio into ~8s parts (seamless with preload)
- All chunks rendered and PCM concatenated (consistent voice)
- Split into ~8 second WAV parts (not per-sentence)
- 8s is long enough for preload overlap, small enough for WebSocket
- Parts include part/totalParts metadata

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:41:14 +02:00
duffyduck 2c785cb37a feat: XTTS concatenates chunks into seamless WAV (no stuttering)
- All chunks rendered sequentially, PCM data concatenated
- Single WAV with proper header sent back (no queue needed in app)
- If total > 800KB, split into parts (WebSocket limit)
- Eliminates stuttering between sentences

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:40:16 +02:00
duffyduck 57e65b061c docs: update issue.md with XTTS streaming as next priority
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:38:21 +02:00
duffyduck aa54765b03 release: bump version to 0.0.2.7 2026-04-10 02:24:58 +02:00
duffyduck 8929bc99bb fix: XTTS groups sentences into ~250 char chunks for consistent voice quality
- 2-3 sentences per chunk (more context = stable voice/volume)
- Max 250 chars per chunk (keeps WebSocket packets manageable)
- Dots re-added between sentences within a chunk (natural pauses)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:23:29 +02:00
duffyduck 0428c06612 fix: Audio preloading to prevent stuttering, remove trailing dots for XTTS
- Preload next audio while current plays (eliminates gap between sentences)
- Remove trailing dots from sentences (XTTS reads them aloud)
- stopPlayback cleans up preloaded audio

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:21:19 +02:00
duffyduck a7eb3cf433 release: bump version to 0.0.2.6 2026-04-10 02:11:04 +02:00
duffyduck e4e0e793a8 fix: Audio queue for sequential TTS playback (no overlap/skip)
- Audio packets queued instead of stopping previous
- _playNext() plays sequentially, each sentence after the previous
- stopPlayback() clears queue
- Fixes overlapping/skipping with XTTS sentence-by-sentence rendering

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:09:35 +02:00
duffyduck b3d3b8b6bc fix: XTTS bridge splits text into sentences sequentially
- XTTS-Bridge does sentence splitting (not ARIA-Bridge)
- Sequential rendering: correct order guaranteed
- Each sentence sent as separate xtts_response
- Markdown removal before splitting
- App starts playback after first sentence (faster UX)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 02:03:29 +02:00
duffyduck 06bc456221 fix: XTTS splits long text into sentences before sending (WebSocket size limit)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:56:25 +02:00
duffyduck 3461f45207 docs: update README with XTTS v2 setup details, voice cloning guide
- Architecture diagram for XTTS flow (Gaming-PC ↔ RVS ↔ ARIA-VM)
- Port 8020 (not 8000), token must match, model caching
- Voice cloning step-by-step guide
- TTS engine switching (Piper/XTTS) with fallback
- Known limitation: RVS zombie connections

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:49:08 +02:00
duffyduck a17d4acc13 fix: XTTS bridge shares /voices volume with XTTS server
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:40:41 +02:00
duffyduck 62fd9193a1 fix: XTTS voice dropdown shows saved voice after page reload
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:34:00 +02:00
duffyduck 2329645df4 fix: XTTS voices list + upload use fresh RVS connection with response wait
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:24:55 +02:00
duffyduck 8a435ddf6c fix: voice upload uses send() via server, not client-side sendToRVS_raw
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:15:29 +02:00
duffyduck 25b754ba31 fix: voice upload Base64 conversion (chunked, no stack overflow)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 01:08:32 +02:00
duffyduck b734593bf2 fix: Bridge _send_to_rvs ping-check before send, force reconnect on zombie
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 00:37:22 +02:00
duffyduck 16847ce6f7 fix: TTS toggle global above engine selector, health check /docs
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 00:27:55 +02:00
duffyduck 6300829317 fix: XTTS model cache volume path /app/xtts_models
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:44:29 +02:00
duffyduck a1e1ee31bd fix: XTTS bridge port 8020, longer startup wait
- XTTS API runs on port 8020 (not 8000)
- Bridge waits up to 5min for model download (30x10s)
- Health check uses / instead of /docs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-07 23:39:45 +02:00
duffyduck 7ed70b876d updated image public path 2026-04-07 23:06:26 +02:00
duffyduck 3ca85da906 release: bump version to 0.0.2.5 2026-04-05 20:12:56 +02:00
24 changed files with 1197 additions and 276 deletions
+37 -7
View File
@@ -1,20 +1,50 @@
# ARIA Environment Configuration
# Copy to .env and fill in values
# ════════════════════════════════════════════════
# ARIA — Umgebungsvariablen
# Kopieren nach .env und Werte eintragen
# ════════════════════════════════════════════════
# Auth token for ARIA Core (generate a long random string)
# openssl rand -hex 32
# ── ARIA Auth Token ──────────────────────────────
# Authentifizierung fuer den OpenClaw Gateway (aria-core).
# Wird von Diagnostic, Bridge und App genutzt um sich am Gateway anzumelden.
# Alle Services die mit aria-core kommunizieren brauchen diesen Token.
# Generieren: openssl rand -hex 32
ARIA_AUTH_TOKEN=change-me-to-a-long-random-string
# RVS — Rendezvous-Server (Bridge + App verbinden sich hierüber)
# ── RVS — Rendezvous-Server ─────────────────────
# Der RVS ist ein WebSocket-Relay im Rechenzentrum.
# App, Bridge, Diagnostic und XTTS-Bridge verbinden sich hierueber.
# Alle muessen den gleichen Host, Port und Token nutzen.
# Hostname des RVS-Servers (z.B. rvs.example.de oder mobil.hacker-net.de)
RVS_HOST=rvs.example.de
# Port auf dem der RVS laeuft (muss mit rvs/docker-compose.yml uebereinstimmen)
RVS_PORT=443
# TLS (wss://) verwenden? true = verschluesselt, false = unverschluesselt (ws://)
RVS_TLS=true
# Bei TLS-Fehler automatisch auf ws:// (ohne TLS) fallback?
# true = Fallback erlaubt, false = nur mit TLS verbinden
# Nuetzlich wenn kein TLS-Zertifikat vorhanden (z.B. Entwicklung)
RVS_TLS_FALLBACK=true
# Pairing-Token: Wer den gleichen Token hat, landet im gleichen RVS-Room.
# Wird von generate-token.sh automatisch generiert und hier eingetragen.
# Die Android App bekommt den Token per QR-Code beim Pairing.
# WICHTIG: Muss auf ARIA-VM, Gaming-PC (xtts/.env) und App identisch sein!
# Generieren: ./generate-token.sh (traegt den Token automatisch ein)
RVS_TOKEN=
# Gitea (for release.sh — Kennwort wird interaktiv abgefragt)
# ── Gitea — Release-Verwaltung ───────────────────
# Wird von release.sh genutzt um APKs auf Gitea zu veroeffentlichen.
# Kennwort wird beim Release interaktiv abgefragt (nicht in .env!).
GITEA_URL=https://git.hacker-net.de
GITEA_REPO=Hacker-Software/ARIA-AGENT
GITEA_USER=duffyduck
# ── Auto-Update — APK auf RVS-Server kopieren ───
# SSH-Ziel fuer scp: release.sh kopiert die APK dorthin.
# Der RVS-Server stellt sie dann per WebSocket an die App bereit.
# Format: user@host (z.B. root@aria-rvs oder root@rvs.example.de)
# Leer lassen = Auto-Update ueberspringen, APK manuell auf RVS kopieren.
RVS_UPDATE_HOST=
+69 -14
View File
@@ -103,16 +103,31 @@ cd ~/ARIA-AGENT
cp .env.example .env
```
`.env` Datei editieren:
`.env` Datei editieren (Details siehe `.env.example`):
```bash
# Gateway-Auth: Alle Services die mit aria-core reden brauchen diesen Token
# Diagnostic, Bridge, App nutzen ihn fuer den WebSocket-Handshake
ARIA_AUTH_TOKEN= # openssl rand -hex 32
# RVS-Verbindung: Hostname + Port deines Rendezvous-Servers
RVS_HOST= # z.B. rvs.hackersoft.de
RVS_PORT=443
RVS_TLS=true
RVS_TLS_FALLBACK=true
RVS_TOKEN= # wird von generate-token.sh automatisch gesetzt
# Pairing-Token: Verbindet App, Bridge, Diagnostic und XTTS im gleichen RVS-Room
# MUSS auf allen Geraeten identisch sein (ARIA-VM, Gaming-PC, App)
# Wird von generate-token.sh automatisch generiert und eingetragen
RVS_TOKEN= # ./generate-token.sh
# Optional: SSH-Host des RVS-Servers fuer Auto-Update (z.B. root@aria-rvs)
RVS_UPDATE_HOST=
```
**Zwei Tokens, zwei Zwecke:**
- **ARIA_AUTH_TOKEN**: Authentifizierung am OpenClaw Gateway (aria-core). Wer diesen Token hat, kann ARIA Befehle geben.
- **RVS_TOKEN**: Pairing-Token fuer den Rendezvous-Server. Alle Geraete mit dem gleichen Token landen im gleichen "Room" und koennen kommunizieren. Die App bekommt diesen Token per QR-Code.
### 2. Claude CLI einloggen (Proxy-Auth)
Der Proxy-Container nutzt deine Claude Max Subscription. Die Credentials muessen
@@ -291,7 +306,8 @@ aria-core → Antwort → Gateway → Diagnostic → RVS → App
### Features
- **STT**: faster-whisper (lokal, offline, 16kHz mono)
- **TTS**: Piper (Ramona + Thorsten, offline)
- **TTS**: Piper (Ramona + Thorsten, offline) oder XTTS v2 (remote, GPU, Voice Cloning)
- **Markdown-Bereinigung**: Entfernt **fett**, *kursiv*, `code`, Links, Listen etc. vor TTS (natuerliche Sprache)
- **Wake-Word**: openwakeword (lokales Mikrofon auf der VM)
- **App-Audio**: Base64 Audio von App → FFmpeg → Whisper STT → Text an aria-core
- **Modi**: Normal, Nicht stoeren, Fluestern, Hangar, Gaming
@@ -352,15 +368,17 @@ API-Endpoint fuer andere Services: `GET http://localhost:3001/api/session`
- Text-Chat mit ARIA
- **Sprachaufnahme**: Push-to-Talk (halten) oder Tap-to-Talk (tippen, Auto-Stop bei Stille)
- **Gespraechsmodus** (Ohr-Button): Nach jeder ARIA-Antwort startet automatisch die Aufnahme — wie ein natuerliches Gespraech hin und her, ohne Buttons druecken
- **VAD (Voice Activity Detection)**: Erkennt 1.8s Stille und stoppt automatisch
- **STT (Speech-to-Text)**: Audio wird in der Bridge per Whisper transkribiert, transkribierter Text erscheint im Chat
- **TTS-Wiedergabe**: ARIA antwortet per Lautsprecher (Piper oder XTTS v2)
- **TTS-Wiedergabe**: ARIA antwortet per Lautsprecher (Piper oder XTTS v2), Audio-Queue mit Preloading
- **Play-Button**: Jede ARIA-Nachricht kann nochmal vorgelesen werden
- **Chat-Suche**: Lupe in der Statusleiste filtert Nachrichten live
- **Datei- und Bild-Upload**: Bilder inline im Chat (Vollbild-Tap), Dateien mit Icon + Name + Groesse
- **Mehrere Anhaenge**: Bilder + Dateien sammeln, Text hinzufuegen, dann zusammen senden
- **Paste-Support**: Bilder aus Zwischenablage einfuegen (Diagnostic)
- **Anhaenge**: Bridge speichert in Shared Volume, ARIA kann darauf zugreifen, Re-Download ueber RVS
- **Einstellungen**: TTS Engine, Stimmen, Speed pro Stimme, Speicherort, Auto-Download, GPS
- **Auto-Update**: Prueft beim Start auf neue Version, Download + Installation ueber RVS
- **Auto-Update**: Prueft beim Start + per Button auf neue Version, Download + Installation ueber RVS (FileProvider)
- GPS-Position (optional)
- QR-Code Scanner fuer Token-Pairing
@@ -530,38 +548,68 @@ cp ARIA-v0.0.3.0.apk ~/ARIA-AGENT/rvs/updates/
## XTTS v2 — GPU TTS Server (optional)
Laeuft auf einem separaten Rechner mit NVIDIA GPU (z.B. Gaming-PC mit RTX 3060).
Verbindet sich ueber RVS mit der ARIA-Infrastruktur — kein VPN noetig.
Verbindet sich ueber RVS mit der ARIA-Infrastruktur — kein VPN noetig, funktioniert
ueber verschiedene Netze hinweg.
### Architektur
```
Gaming-PC (Windows, RTX 3060, Docker Desktop + WSL2)
├── aria-xtts XTTS v2 GPU Server (Port 8020 intern)
└── aria-xtts-bridge RVS-Relay (empfaengt Requests, sendet Audio)
└── Beide teilen ./voices/ Volume fuer Voice Cloning
↕ RVS (Rechenzentrum, WebSocket Relay)
ARIA-VM
└── aria-bridge: tts_engine="xtts" → xtts_request via RVS → wartet auf xtts_response
```
### Voraussetzungen
- Docker Desktop mit WSL2 (Windows) oder Docker mit NVIDIA Runtime (Linux)
- NVIDIA Container Toolkit
- GPU mit mindestens 4GB VRAM (6GB+ empfohlen)
- **Gleicher RVS_TOKEN wie auf der ARIA-VM!**
### Setup
```bash
cd xtts
cp .env.example .env
# .env mit RVS-Verbindungsdaten fuellen (gleiche wie auf der ARIA-VM)
# .env mit RVS-Verbindungsdaten fuellen (gleicher Token wie ARIA-VM!)
docker compose up -d
# Erster Start laedt ~2GB Model herunter
# Erster Start laedt ~2GB Model herunter (danach gecacht)
```
**Wichtig:** Der XTTS-Server laeuft intern auf Port **8020** (nicht 8000).
Das Model wird im Volume `xtts-models` gecacht und muss nur einmal geladen werden.
### Features
- **Natuerliche Stimmen**: Deutlich bessere Qualitaet als Piper
- **Voice Cloning**: Eigene Stimme mit 6-10s Audio-Sample
- **Voice Cloning**: Eigene Stimme mit 6-10s Audio-Sample (~2s Latenz auf RTX 3060)
- **16 Sprachen**: Deutsch, Englisch, Franzoesisch, etc.
- **RVS-Integration**: Bridge waehlt automatisch XTTS wenn verfuegbar
- **Fallback**: Wenn XTTS nicht erreichbar, nutzt die Bridge automatisch Piper
### TTS-Engine umschalten
In der Diagnostic unter Einstellungen → Sprachausgabe:
- **TTS aktiv**: Global An/Aus
- **TTS Engine**: Piper (lokal, CPU, schnell) oder XTTS v2 (remote, GPU, natuerlich)
- **Piper**: Standard-Stimme, Highlight-Stimme, Speed pro Stimme
- **XTTS**: Stimmen-Auswahl, Voice Cloning
### Stimme klonen
In der Diagnostic unter Einstellungen → Sprachausgabe → XTTS:
1. TTS Engine auf "XTTS v2" stellen
2. "Stimme klonen" → Audio-Dateien hochladen (WAV/MP3, min. 6-10s)
2. "Stimme klonen" → Audio-Dateien hochladen (WAV/MP3, 1-10 Dateien, min. 6-10s gesamt)
3. Name vergeben → "Stimme erstellen"
4. Neue Stimme in der Auswahl verfuegbar
4. "Laden" klicken → neue Stimme in der Auswahl
5. Stimme auswaehlen → Config wird automatisch gespeichert
> **Tipp:** Fuer beste Ergebnisse: saubere Aufnahme, eine Stimme, kein Hintergrund,
> 10-30 Sekunden Gesamtlaenge. Mehrere kurze Dateien werden zusammengefuegt.
---
@@ -633,6 +681,8 @@ docker exec aria-core ssh aria-wohnung hostname
- **Wake Word nur auf VM**: Die Bridge hoert auf "ARIA" ueber das lokale Mikrofon der VM.
In der App gibt es Energy-basierte Erkennung (Phase 1). On-device "ARIA"-Keyword (Porcupine) ist Phase 2.
- **Audio-Format**: App nimmt AAC/MP4 auf, Bridge konvertiert via FFmpeg zu 16kHz PCM.
- **RVS Zombie-Connections**: WebSocket-Verbindungen sterben gelegentlich ohne Fehlermeldung.
Bridge hat Ping-Check (5s), Diagnostic nutzt frische Verbindungen pro Request.
- **Bildanalyse eingeschraenkt**: Bilder werden in `/shared/uploads/` gespeichert. ARIA kann
sie per Bash/Read-Tool oeffnen, aber Claude Vision (direkte Bildanalyse) ist ueber den
Proxy-Pfad (`claude --print`) noch nicht moeglich. ARIA sieht den Dateipfad, nicht das Bild.
@@ -662,6 +712,11 @@ docker exec aria-core ssh aria-wohnung hostname
- [x] Auto-Update System (APK via RVS)
- [x] Chat-Suche, Play-Button, Abbrechen-Button
- [x] XTTS v2 Integration (GPU, Voice Cloning, remote ueber RVS)
- [x] Gespraechsmodus (Ohr-Button, automatische Aufnahme nach ARIA-Antwort)
- [x] Mehrere Anhaenge + Text vor dem Senden + Paste-Support
- [x] Markdown-Bereinigung fuer TTS
- [x] Auto-Update mit FileProvider + Update-Check Button
- [x] Inverted FlatList (zuverlaessiges Scroll-to-Bottom)
### Phase 2 — ARIA wird produktiv
+2 -2
View File
@@ -79,8 +79,8 @@ android {
applicationId "com.ariacockpit"
minSdkVersion rootProject.ext.minSdkVersion
targetSdkVersion rootProject.ext.targetSdkVersion
versionCode 204
versionName "0.0.2.4"
versionCode 308
versionName "0.0.3.8"
// Fallback fuer Libraries mit Product Flavors
missingDimensionStrategy 'react-native-camera', 'general'
}
@@ -3,6 +3,7 @@
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.REQUEST_INSTALL_PACKAGES" />
<application
android:name=".MainApplication"
@@ -24,5 +25,15 @@
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
</activity>
<provider
android:name="androidx.core.content.FileProvider"
android:authorities="${applicationId}.fileprovider"
android:exported="false"
android:grantUriPermissions="true">
<meta-data
android:name="android.support.FILE_PROVIDER_PATHS"
android:resource="@xml/file_paths" />
</provider>
</application>
</manifest>
@@ -0,0 +1,44 @@
package com.ariacockpit
import android.content.Intent
import android.net.Uri
import android.os.Build
import androidx.core.content.FileProvider
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.bridge.ReactContextBaseJavaModule
import com.facebook.react.bridge.ReactMethod
import com.facebook.react.bridge.Promise
import java.io.File
class ApkInstallerModule(reactContext: ReactApplicationContext) : ReactContextBaseJavaModule(reactContext) {
override fun getName() = "ApkInstaller"
@ReactMethod
fun install(filePath: String, promise: Promise) {
try {
val file = File(filePath)
if (!file.exists()) {
promise.reject("FILE_NOT_FOUND", "APK nicht gefunden: $filePath")
return
}
val context = reactApplicationContext
val uri: Uri = if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.N) {
FileProvider.getUriForFile(context, "${context.packageName}.fileprovider", file)
} else {
Uri.fromFile(file)
}
val intent = Intent(Intent.ACTION_VIEW).apply {
setDataAndType(uri, "application/vnd.android.package-archive")
addFlags(Intent.FLAG_ACTIVITY_NEW_TASK)
addFlags(Intent.FLAG_GRANT_READ_URI_PERMISSION)
}
context.startActivity(intent)
promise.resolve(true)
} catch (e: Exception) {
promise.reject("INSTALL_ERROR", e.message, e)
}
}
}
@@ -0,0 +1,16 @@
package com.ariacockpit
import com.facebook.react.ReactPackage
import com.facebook.react.bridge.NativeModule
import com.facebook.react.bridge.ReactApplicationContext
import com.facebook.react.uimanager.ViewManager
class ApkInstallerPackage : ReactPackage {
override fun createNativeModules(reactContext: ReactApplicationContext): List<NativeModule> {
return listOf(ApkInstallerModule(reactContext))
}
override fun createViewManagers(reactContext: ReactApplicationContext): List<ViewManager<*, *>> {
return emptyList()
}
}
@@ -18,8 +18,7 @@ class MainApplication : Application(), ReactApplication {
object : DefaultReactNativeHost(this) {
override fun getPackages(): List<ReactPackage> =
PackageList(this).packages.apply {
// Packages that cannot be autolinked yet can be added manually here, for example:
// add(MyReactNativePackage())
add(ApkInstallerPackage())
}
override fun getJSMainModuleName(): String = "index"
@@ -0,0 +1,4 @@
<?xml version="1.0" encoding="utf-8"?>
<paths>
<cache-path name="cache" path="." />
</paths>
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "aria-cockpit",
"version": "0.0.2.4",
"version": "0.0.3.8",
"private": true,
"scripts": {
"android": "react-native run-android",
+218 -93
View File
@@ -5,7 +5,7 @@
* Datei- und Kamera-Upload.
*/
import React, { useState, useEffect, useRef, useCallback } from 'react';
import React, { useState, useEffect, useRef, useCallback, useMemo } from 'react';
import {
View,
Text,
@@ -16,6 +16,7 @@ import {
Platform,
StyleSheet,
Image,
ScrollView,
Modal,
} from 'react-native';
import AsyncStorage from '@react-native-async-storage/async-storage';
@@ -94,6 +95,8 @@ const ChatScreen: React.FC = () => {
const [fullscreenImage, setFullscreenImage] = useState<string | null>(null);
const [searchQuery, setSearchQuery] = useState('');
const [searchVisible, setSearchVisible] = useState(false);
const [pendingAttachments, setPendingAttachments] = useState<{file: any, isPhoto: boolean}[]>([]);
const [agentActivity, setAgentActivity] = useState<{activity: string, tool: string}>({activity: 'idle', tool: ''});
const flatListRef = useRef<FlatList>(null);
const messageIdCounter = useRef(0);
@@ -248,6 +251,13 @@ const ChatScreen: React.FC = () => {
if (message.type === 'audio' && message.payload.base64) {
audioService.playAudio(message.payload.base64 as string);
}
// Thinking-Indicator Status von der Bridge
if (message.type === 'agent_activity') {
const activity = (message.payload.activity as string) || 'idle';
const tool = (message.payload.tool as string) || '';
setAgentActivity({ activity, tool });
}
});
const unsubState = rvs.onStateChange((state) => {
@@ -273,12 +283,20 @@ const ChatScreen: React.FC = () => {
return () => { unsubUpdate(); clearTimeout(timer); };
}, []);
// Wake Word: "ARIA" Erkennung → Auto-Aufnahme starten
// Gespraechsmodus: Nach TTS-Wiedergabe automatisch Aufnahme starten
useEffect(() => {
const unsubPlayback = audioService.onPlaybackFinished(() => {
if (wakeWordService.isActive()) {
wakeWordService.resume();
}
});
return () => unsubPlayback();
}, []);
// Wake Word / Gespraechsmodus: Auto-Aufnahme starten
useEffect(() => {
const unsubWake = wakeWordService.onWakeWord(async () => {
console.log('[Chat] Wake Word erkannt — starte Auto-Aufnahme');
// TTS stoppen damit ARIA sich nicht selbst hoert
audioService.stopPlayback();
console.log('[Chat] Gespraechsmodus — starte Auto-Aufnahme');
// Aufnahme mit Auto-Stop (VAD) starten
const started = await audioService.startRecording(true);
if (!started) {
@@ -359,22 +377,8 @@ const ChatScreen: React.FC = () => {
return () => { if (saveTimer.current) clearTimeout(saveTimer.current); };
}, [messages]);
// Auto-Scroll wird ueber onContentSizeChange der FlatList gesteuert
const shouldAutoScroll = useRef(true);
const handleContentSizeChange = useCallback(() => {
if (shouldAutoScroll.current) {
flatListRef.current?.scrollToEnd({ animated: false });
}
}, []);
const handleScrollBeginDrag = useCallback(() => {
shouldAutoScroll.current = false;
}, []);
const handleScrollEndDrag = useCallback((e: any) => {
// Auto-Scroll wieder aktivieren wenn User ganz unten ist
const { contentOffset, contentSize, layoutMeasurement } = e.nativeEvent;
const isAtBottom = contentOffset.y + layoutMeasurement.height >= contentSize.height - 50;
shouldAutoScroll.current = isAtBottom;
}, []);
// Inverted FlatList: neueste Nachrichten unten, kein manuelles Scrollen noetig
const invertedMessages = useMemo(() => [...messages].reverse(), [messages]);
// GPS-Position holen (optional)
const getCurrentLocation = useCallback((): Promise<{ lat: number; lon: number } | null> => {
@@ -400,6 +404,13 @@ const ChatScreen: React.FC = () => {
const sendTextMessage = useCallback(async () => {
const text = inputText.trim();
// Wenn pending Anhaenge vorhanden → Anhaenge + Text zusammen senden
if (pendingAttachments.length > 0) {
sendPendingAttachments(text);
return;
}
if (!text) return;
setInputText('');
@@ -419,7 +430,13 @@ const ChatScreen: React.FC = () => {
text,
...(location && { location }),
});
}, [inputText, getCurrentLocation]);
}, [inputText, getCurrentLocation, pendingAttachments, sendPendingAttachments]);
// Anfrage abbrechen — sofort lokalen Indicator weg, Bridge triggert doctor --fix
const cancelRequest = useCallback(() => {
setAgentActivity({ activity: 'idle', tool: '' });
rvs.send('cancel_request' as any, {});
}, []);
// Sprachaufnahme abgeschlossen
const handleVoiceRecording = useCallback(async (result: RecordingResult) => {
@@ -441,88 +458,91 @@ const ChatScreen: React.FC = () => {
});
}, [getCurrentLocation]);
// Datei senden
// Datei auswaehlen → zur Pending-Liste hinzufuegen
const handleFileSelected = useCallback(async (file: FileData) => {
setShowFileUpload(false);
const location = await getCurrentLocation();
setPendingAttachments(prev => [...prev, { file, isPhoto: false }]);
}, []);
const isImage = file.type.startsWith('image/');
const msgId = nextId();
let imageUri = isImage && file.base64 ? `data:${file.type};base64,${file.base64}` : file.uri;
const userMsg: ChatMessage = {
id: msgId,
sender: 'user',
text: 'Anhang empfangen',
timestamp: Date.now(),
attachments: [{
type: isImage ? 'image' : 'file',
name: file.name,
size: file.size,
uri: imageUri,
mimeType: file.type,
}],
};
setMessages(prev => [...prev, userMsg]);
// Anhang auf Disk speichern fuer Persistenz
if (file.base64) {
persistAttachment(file.base64, msgId, file.name).then(filePath => {
setMessages(prev => prev.map(m =>
m.id === msgId ? { ...m, attachments: m.attachments?.map(a => ({ ...a, uri: filePath })) } : m
));
}).catch(() => {});
}
rvs.send('file', {
name: file.name,
type: file.type,
size: file.size,
base64: file.base64,
...(location && { location }),
});
}, [getCurrentLocation]);
// Foto senden
// Foto auswaehlen → zur Pending-Liste hinzufuegen
const handlePhotoSelected = useCallback(async (photo: PhotoData) => {
setShowCameraUpload(false);
setPendingAttachments(prev => [...prev, { file: photo, isPhoto: true }]);
}, []);
// Alle Pending Anhaenge + Text senden
const sendPendingAttachments = useCallback(async (messageText: string) => {
if (pendingAttachments.length === 0) return;
const location = await getCurrentLocation();
const msgId = nextId();
const dataUri = photo.base64 ? `data:${photo.type};base64,${photo.base64}` : undefined;
// Alle Attachments fuer die Chat-Nachricht sammeln
const attachments: Attachment[] = [];
for (const { file, isPhoto } of pendingAttachments) {
const isImage = isPhoto || (file.type && file.type.startsWith('image/'));
const name = isPhoto ? file.fileName : file.name;
const base64 = file.base64 || '';
const mimeType = file.type || '';
const imageUri = isImage && base64 ? `data:${mimeType};base64,${base64}` : file.uri;
attachments.push({
type: isImage ? 'image' : 'file',
name,
size: file.size,
uri: imageUri,
mimeType,
});
}
// Chat-Nachricht mit allen Anhaengen
const userMsg: ChatMessage = {
id: msgId,
sender: 'user',
text: 'Anhang empfangen',
text: messageText || `${pendingAttachments.length} Anhang/Anhaenge`,
timestamp: Date.now(),
attachments: [{
type: 'image',
name: photo.fileName,
uri: dataUri,
mimeType: photo.type,
}],
attachments,
};
setMessages(prev => [...prev, userMsg]);
// Foto auf Disk speichern fuer Persistenz
if (photo.base64) {
persistAttachment(photo.base64, msgId, photo.fileName).then(filePath => {
setMessages(prev => prev.map(m =>
m.id === msgId ? { ...m, attachments: m.attachments?.map(a => ({ ...a, uri: filePath })) } : m
));
}).catch(() => {});
// Alle Dateien an RVS senden + auf Disk speichern
for (const { file, isPhoto } of pendingAttachments) {
const name = isPhoto ? file.fileName : file.name;
const base64 = file.base64 || '';
const mimeType = file.type || '';
// Auf Disk speichern
if (base64) {
persistAttachment(base64, msgId + '_' + name, name).then(filePath => {
setMessages(prev => prev.map(m =>
m.id === msgId ? { ...m, attachments: m.attachments?.map(a =>
a.name === name && !a.uri?.startsWith('file://') ? { ...a, uri: filePath } : a
)} : m
));
}).catch(() => {});
}
// An RVS senden
rvs.send('file', {
name,
type: mimeType,
size: file.size,
base64,
...(isPhoto && file.width && { width: file.width, height: file.height }),
...(location && { location }),
});
}
rvs.send('file', {
name: photo.fileName,
type: photo.type,
base64: photo.base64,
width: photo.width,
height: photo.height,
...(location && { location }),
});
}, [getCurrentLocation]);
// Text als separate Nachricht (damit ARIA weiss was zu tun ist)
if (messageText) {
rvs.send('chat', {
text: messageText,
...(location && { location }),
});
}
setPendingAttachments([]);
setInputText('');
}, [pendingAttachments, getCurrentLocation]);
// --- Rendering ---
@@ -653,14 +673,12 @@ const ChatScreen: React.FC = () => {
{/* Nachrichtenliste */}
<FlatList
ref={flatListRef}
data={searchQuery ? messages.filter(m => m.text.toLowerCase().includes(searchQuery.toLowerCase())) : messages}
inverted
data={searchQuery ? messages.filter(m => m.text.toLowerCase().includes(searchQuery.toLowerCase())).reverse() : invertedMessages}
keyExtractor={item => item.id}
renderItem={renderMessage}
contentContainerStyle={styles.messageList}
showsVerticalScrollIndicator={false}
onContentSizeChange={handleContentSizeChange}
onScrollBeginDrag={handleScrollBeginDrag}
onScrollEndDrag={handleScrollEndDrag}
ListEmptyComponent={
<View style={styles.emptyContainer}>
<Text style={styles.emptyIcon}>{'\uD83E\uDD16'}</Text>
@@ -670,6 +688,56 @@ const ChatScreen: React.FC = () => {
}
/>
{/* Thinking-Indicator */}
{agentActivity.activity !== 'idle' && (
<View style={styles.thinkingBar}>
<Text style={styles.thinkingText}>
{agentActivity.activity === 'tool' && agentActivity.tool
? `\uD83D\uDD27 ${agentActivity.tool}`
: agentActivity.activity === 'assistant'
? '\u270D\uFE0F ARIA schreibt...'
: '\uD83D\uDCAD ARIA denkt...'}
</Text>
<TouchableOpacity style={styles.thinkingCancel} onPress={cancelRequest}>
<Text style={styles.thinkingCancelText}>Abbrechen</Text>
</TouchableOpacity>
</View>
)}
{/* Pending Anhaenge Vorschau */}
{pendingAttachments.length > 0 && (
<View style={styles.pendingBar}>
<ScrollView horizontal showsHorizontalScrollIndicator={false} style={{flex: 1}}>
{pendingAttachments.map((att, idx) => (
<View key={idx} style={styles.pendingItem}>
{att.file.type?.startsWith('image/') || att.isPhoto ? (
<Image
source={{ uri: att.file.base64
? `data:${att.file.type};base64,${att.file.base64}`
: att.file.uri }}
style={styles.pendingThumb}
/>
) : (
<View style={[styles.pendingThumb, {justifyContent: 'center', alignItems: 'center'}]}>
<Text style={{fontSize: 20}}>{'\uD83D\uDCC4'}</Text>
</View>
)}
<TouchableOpacity
style={styles.pendingRemove}
onPress={() => setPendingAttachments(prev => prev.filter((_, i) => i !== idx))}
>
<Text style={{color: '#fff', fontSize: 10, fontWeight: 'bold'}}>X</Text>
</TouchableOpacity>
</View>
))}
</ScrollView>
<Text style={{color: '#8888AA', fontSize: 11, marginLeft: 8}}>{pendingAttachments.length}</Text>
<TouchableOpacity onPress={() => setPendingAttachments([])}>
<Text style={{color: '#FF3B30', fontSize: 14, paddingHorizontal: 8}}>Alle X</Text>
</TouchableOpacity>
</View>
)}
{/* Eingabebereich */}
<View style={styles.inputContainer}>
{/* Datei-Buttons */}
@@ -692,7 +760,7 @@ const ChatScreen: React.FC = () => {
style={styles.textInput}
value={inputText}
onChangeText={setInputText}
placeholder="Nachricht an ARIA..."
placeholder={pendingAttachments.length > 0 ? "Text zu den Anhaengen (optional)..." : "Nachricht an ARIA..."}
placeholderTextColor="#555570"
multiline
maxLength={4000}
@@ -701,7 +769,7 @@ const ChatScreen: React.FC = () => {
/>
{/* Senden oder Sprache */}
{inputText.trim() ? (
{inputText.trim() || pendingAttachments.length > 0 ? (
<TouchableOpacity style={styles.sendButton} onPress={sendTextMessage}>
<Text style={styles.sendIcon}>{'\u2B06\uFE0F'}</Text>
</TouchableOpacity>
@@ -932,6 +1000,63 @@ const styles = StyleSheet.create({
wakeWordIcon: {
fontSize: 16,
},
thinkingBar: {
flexDirection: 'row',
alignItems: 'center',
justifyContent: 'space-between',
backgroundColor: '#1E1E2E',
paddingHorizontal: 12,
paddingVertical: 6,
borderTopWidth: 1,
borderTopColor: '#2A2A3E',
},
thinkingText: {
color: '#FFD60A',
fontSize: 12,
flex: 1,
},
thinkingCancel: {
paddingHorizontal: 10,
paddingVertical: 4,
borderWidth: 1,
borderColor: '#FF3B30',
borderRadius: 4,
},
thinkingCancelText: {
color: '#FF3B30',
fontSize: 11,
fontWeight: 'bold',
},
pendingBar: {
flexDirection: 'row',
alignItems: 'center',
backgroundColor: '#1E1E2E',
paddingHorizontal: 12,
paddingVertical: 8,
borderTopWidth: 1,
borderTopColor: '#2A2A3E',
},
pendingItem: {
position: 'relative',
marginRight: 8,
},
pendingThumb: {
width: 50,
height: 50,
borderRadius: 6,
backgroundColor: '#0D0D1A',
},
pendingRemove: {
position: 'absolute',
top: -4,
right: -4,
width: 18,
height: 18,
borderRadius: 9,
backgroundColor: '#FF3B30',
justifyContent: 'center',
alignItems: 'center',
},
searchBar: {
flexDirection: 'row',
alignItems: 'center',
+11 -1
View File
@@ -748,11 +748,21 @@ const SettingsScreen: React.FC = () => {
<Text style={styles.sectionTitle}>{'\u00DC'}ber</Text>
<View style={styles.card}>
<Text style={styles.aboutTitle}>ARIA Cockpit</Text>
<Text style={styles.aboutVersion}>Version 0.0.2.4 </Text>
<Text style={styles.aboutVersion}>Version {require('../../package.json').version}</Text>
<Text style={styles.aboutInfo}>
Stefans Kommandozentrale f{'\u00FC'}r ARIA.{'\n'}
Gebaut mit React Native + TypeScript.
</Text>
<TouchableOpacity
style={[styles.connectButton, {marginTop: 12}]}
onPress={() => {
const updateService = require('../services/updater').default;
updateService.checkForUpdate();
Alert.alert('Update-Check', 'Pruefe auf neue Version...');
}}
>
<Text style={styles.connectButtonText}>Auf Updates pr{'\u00FC'}fen</Text>
</TouchableOpacity>
</View>
{/* Platz am Ende */}
+133 -31
View File
@@ -42,6 +42,8 @@ const AUDIO_ENCODING = 'audio/wav';
// VAD (Voice Activity Detection) — Stille-Erkennung
const VAD_SILENCE_THRESHOLD_DB = -45; // dB unter dem als "Stille" gilt
const VAD_SILENCE_DURATION_MS = 1800; // ms Stille bevor Auto-Stop
const VAD_SPEECH_THRESHOLD_DB = -35; // dB ueber dem als "Sprache" gilt (Sprach-Gate)
const VAD_SPEECH_MIN_MS = 300; // ms Sprache bevor Aufnahme zaehlt
// --- Audio-Service ---
@@ -55,6 +57,16 @@ class AudioService {
private recorder: AudioRecorderPlayer;
private recordingPath: string = '';
// Audio-Queue fuer sequentielle TTS-Wiedergabe
private audioQueue: string[] = [];
private isPlaying: boolean = false;
private preloadedSound: Sound | null = null;
private preloadedPath: string = '';
// Sprach-Gate: Aufnahme erst senden wenn tatsaechlich gesprochen wurde
private speechDetected: boolean = false;
private speechStartTime: number = 0;
// VAD State
private vadEnabled: boolean = false;
private lastSpeechTime: number = 0;
@@ -115,6 +127,8 @@ class AudioService {
AudioEncoderAndroid: AudioEncoderAndroidType.AAC,
AudioSourceAndroid: AudioSourceAndroidType.MIC,
OutputFormatAndroid: OutputFormatAndroidType.MPEG_4,
AudioSamplingRateAndroid: 16000,
AudioChannelsAndroid: 1,
}, true); // meteringEnabled = true
// Metering-Callback
@@ -122,7 +136,21 @@ class AudioService {
const db = e.currentMetering ?? -160;
this.meterListeners.forEach(cb => cb(db));
// VAD: Stille erkennen
// Sprach-Gate: Erkennen ob tatsaechlich gesprochen wird
if (db > VAD_SPEECH_THRESHOLD_DB) {
if (!this.speechDetected && this.speechStartTime === 0) {
this.speechStartTime = Date.now();
}
if (this.speechStartTime > 0 && Date.now() - this.speechStartTime >= VAD_SPEECH_MIN_MS) {
this.speechDetected = true;
}
} else {
if (!this.speechDetected) {
this.speechStartTime = 0; // Reset wenn noch nicht als Sprache erkannt
}
}
// VAD: Stille erkennen (nur wenn Sprache erkannt wurde)
if (this.vadEnabled) {
if (db > VAD_SILENCE_THRESHOLD_DB) {
this.lastSpeechTime = Date.now();
@@ -132,6 +160,8 @@ class AudioService {
this.recordingStartTime = Date.now();
this.lastSpeechTime = Date.now();
this.speechDetected = false;
this.speechStartTime = 0;
this.setState('recording');
// VAD aktivieren
@@ -174,6 +204,15 @@ class AudioService {
this.recorder.removeRecordBackListener();
const durationMs = Date.now() - this.recordingStartTime;
const hadSpeech = this.speechDetected;
// Sprach-Gate: Wenn keine Sprache erkannt → Aufnahme verwerfen
if (!hadSpeech) {
RNFS.unlink(this.recordingPath).catch(() => {});
this.setState('idle');
console.log('[Audio] Aufnahme verworfen — keine Sprache erkannt (nur Umgebungsgeraeusche)');
return null;
}
// Audio-Datei als Base64 lesen
const base64Data = await RNFS.readFile(this.recordingPath, 'base64');
@@ -182,7 +221,7 @@ class AudioService {
RNFS.unlink(this.recordingPath).catch(() => {});
this.setState('idle');
console.log(`[Audio] Aufnahme beendet (${durationMs}ms, ${Math.round(base64Data.length / 1024)}KB)`);
console.log(`[Audio] Aufnahme beendet (${durationMs}ms, ${Math.round(base64Data.length / 1024)}KB, Sprache erkannt)`);
return {
base64: base64Data,
@@ -198,47 +237,110 @@ class AudioService {
// --- Wiedergabe ---
/** Base64-kodiertes Audio abspielen (z.B. TTS-Antwort von ARIA) */
/** Base64-kodiertes Audio in die Queue stellen und abspielen */
async playAudio(base64Data: string): Promise<void> {
if (!base64Data) return;
// Laufende Wiedergabe stoppen
this.stopPlayback();
try {
// Base64 -> temporaere WAV-Datei -> Sound abspielen
const tmpPath = `${RNFS.CachesDirectoryPath}/aria_tts_${Date.now()}.wav`;
await RNFS.writeFile(tmpPath, base64Data, 'base64');
this.currentSound = new Sound(tmpPath, '', (error) => {
if (error) {
console.error('[Audio] Fehler beim Laden:', error);
RNFS.unlink(tmpPath).catch(() => {});
return;
}
this.currentSound?.play((success) => {
if (success) {
console.log('[Audio] Wiedergabe abgeschlossen');
} else {
console.warn('[Audio] Wiedergabe fehlgeschlagen');
}
this.currentSound?.release();
this.currentSound = null;
RNFS.unlink(tmpPath).catch(() => {});
});
});
} catch (err) {
console.error('[Audio] Wiedergabefehler:', err);
this.audioQueue.push(base64Data);
if (!this.isPlaying) {
this._playNext();
}
}
/** Laufende Wiedergabe stoppen */
// Callback wenn alle Audio-Teile abgespielt sind
private playbackFinishedListeners: (() => void)[] = [];
onPlaybackFinished(callback: () => void): () => void {
this.playbackFinishedListeners.push(callback);
return () => {
this.playbackFinishedListeners = this.playbackFinishedListeners.filter(cb => cb !== callback);
};
}
/** Naechstes Audio aus der Queue abspielen */
private async _playNext(): Promise<void> {
if (this.audioQueue.length === 0) {
this.isPlaying = false;
// Alle Audio-Teile abgespielt → Listener benachrichtigen
this.playbackFinishedListeners.forEach(cb => cb());
return;
}
this.isPlaying = true;
// Preloaded Sound verwenden wenn verfuegbar, sonst neu laden
let sound: Sound;
let soundPath: string;
if (this.preloadedSound) {
sound = this.preloadedSound;
soundPath = this.preloadedPath;
this.preloadedSound = null;
this.preloadedPath = '';
// Daten aus Queue entfernen (wurde schon preloaded)
this.audioQueue.shift();
} else {
const base64Data = this.audioQueue.shift()!;
try {
soundPath = `${RNFS.CachesDirectoryPath}/aria_tts_${Date.now()}.wav`;
await RNFS.writeFile(soundPath, base64Data, 'base64');
sound = await new Promise<Sound>((resolve, reject) => {
const s = new Sound(soundPath, '', (err) => err ? reject(err) : resolve(s));
});
} catch (err) {
console.error('[Audio] Laden fehlgeschlagen:', err);
this._playNext();
return;
}
}
this.currentSound = sound;
// Naechstes Audio schon vorbereiten waehrend dieses abspielt
this._preloadNext();
sound.play((success) => {
if (!success) console.warn('[Audio] Wiedergabe fehlgeschlagen');
sound.release();
this.currentSound = null;
RNFS.unlink(soundPath).catch(() => {});
this._playNext();
});
}
/** Naechstes Audio im Hintergrund vorladen (verhindert Stottern) */
private async _preloadNext(): Promise<void> {
if (this.audioQueue.length === 0 || this.preloadedSound) return;
const base64Data = this.audioQueue[0]; // Nicht shift — bleibt in Queue
try {
const tmpPath = `${RNFS.CachesDirectoryPath}/aria_tts_pre_${Date.now()}.wav`;
await RNFS.writeFile(tmpPath, base64Data, 'base64');
this.preloadedSound = await new Promise<Sound>((resolve, reject) => {
const s = new Sound(tmpPath, '', (err) => err ? reject(err) : resolve(s));
});
this.preloadedPath = tmpPath;
} catch {
this.preloadedSound = null;
this.preloadedPath = '';
}
}
/** Laufende Wiedergabe stoppen + Queue leeren */
stopPlayback(): void {
this.audioQueue = [];
this.isPlaying = false;
if (this.currentSound) {
this.currentSound.stop();
this.currentSound.release();
this.currentSound = null;
}
if (this.preloadedSound) {
this.preloadedSound.release();
this.preloadedSound = null;
if (this.preloadedPath) RNFS.unlink(this.preloadedPath).catch(() => {});
this.preloadedPath = '';
}
}
// --- Status & Callbacks ---
+14 -5
View File
@@ -7,12 +7,13 @@
* 3. App zeigt Benachrichtigung → User bestaetigt → Download + Install
*/
import { Alert, Linking, Platform } from 'react-native';
import { Alert, Linking, Platform, NativeModules } from 'react-native';
import RNFS from 'react-native-fs';
import rvs, { RVSMessage } from './rvs';
// Aktuelle App-Version (aus package.json via Build)
const APP_VERSION = '0.0.2.3'; // TODO: aus nativer Build-Config lesen
// Version aus package.json (wird beim Build eingebettet)
const packageJson = require('../../package.json');
const APP_VERSION = packageJson.version || '0.0.0.0';
type UpdateCallback = (info: UpdateInfo) => void;
@@ -116,9 +117,17 @@ class UpdateService {
const fileSize = await RNFS.stat(destPath);
console.log(`[Update] APK gespeichert: ${destPath} (${(parseInt(fileSize.size) / 1024 / 1024).toFixed(1)}MB)`);
// APK installieren (oeffnet Android-Installer)
// APK installieren via natives ApkInstaller Module (FileProvider + Intent)
if (Platform.OS === 'android') {
await Linking.openURL(`file://${destPath}`);
try {
const { ApkInstaller } = NativeModules;
await ApkInstaller.install(destPath);
} catch (installErr: any) {
Alert.alert(
'APK heruntergeladen',
`Version ${info.version} gespeichert.\n\nBitte manuell installieren:\nDateimanager → ${apkData.fileName} antippen.\n\n(${installErr.message})`,
);
}
}
} catch (err: any) {
console.error(`[Update] Fehler: ${err.message}`);
+29 -19
View File
@@ -1,10 +1,11 @@
/**
* Wake Word Service — "ARIA" Erkennung
* Gespraechsmodus — "Ohr-Button"
*
* Phase 1: Deaktiviert — react-native-live-audio-stream hat native Bridge-Probleme.
* Nutzt stattdessen Tap-to-Talk (VoiceButton) als primaeren Eingabemodus.
* Wenn aktiv: Nach jeder ARIA-Antwort (TTS fertig) startet automatisch die Aufnahme.
* Wie ein Walkie-Talkie / natuerliches Gespraech:
* ARIA spricht → Aufnahme startet → User spricht → VAD stoppt → ARIA antwortet → ...
*
* Phase 2: Porcupine on-device "ARIA" Keyword (geplant).
* Phase 2 (geplant): Porcupine "ARIA" Wake Word fuer passives Lauschen.
*/
type WakeWordCallback = () => void;
@@ -17,30 +18,39 @@ class WakeWordService {
private wakeCallbacks: WakeWordCallback[] = [];
private stateCallbacks: StateCallback[] = [];
/** Wake Word Erkennung starten */
/** Gespraechsmodus starten */
async start(): Promise<boolean> {
if (this.state === 'listening') return true;
try {
// Phase 1: LiveAudioStream deaktiviert (native Bridge instabil)
// Stattdessen: Tap-to-Talk als primaerer Modus
console.log('[WakeWord] Wake Word ist in Phase 1 noch nicht verfuegbar — nutze Tap-to-Talk');
this.setState('listening');
return true;
} catch (err) {
console.error('[WakeWord] Start fehlgeschlagen:', err);
return false;
}
console.log('[WakeWord] Gespraechsmodus aktiviert — starte sofort Aufnahme');
this.setState('listening');
// Sofort erste Aufnahme starten
setTimeout(() => {
if (this.state === 'listening') {
this.wakeCallbacks.forEach(cb => cb());
}
}, 500);
return true;
}
/** Wake Word Erkennung stoppen */
/** Gespraechsmodus stoppen */
stop(): void {
console.log('[WakeWord] Gespraechsmodus deaktiviert');
this.setState('off');
}
/** Nach Aufnahme erneut starten */
/** Nach ARIA-Antwort (TTS fertig): Aufnahme automatisch starten */
async resume(): Promise<void> {
// Nichts zu tun in Phase 1
if (this.state !== 'listening') return;
// Kurze Pause damit TTS-Audio nicht ins Mikrofon geht
await new Promise(resolve => setTimeout(resolve, 800));
if (this.state === 'listening') {
console.log('[WakeWord] TTS fertig — starte automatisch Aufnahme');
this.wakeCallbacks.forEach(cb => cb());
}
}
isActive(): boolean {
return this.state === 'listening';
}
// --- Callbacks ---
+7
View File
@@ -9,3 +9,10 @@ PIPER_THORSTEN=/voices/de_DE-thorsten-high.onnx
# Wake-Word
WAKE_WORD=aria
# Whisper STT — wird zur Laufzeit in der Diagnostic (Sektion "Whisper") umgeschaltet
# und in /shared/config/voice_config.json gespeichert. Der Wert hier ist nur der
# Initial-Default beim ersten Start.
# Optionen: tiny | base | small | medium | large-v3
WHISPER_MODEL=medium
WHISPER_LANGUAGE=de
+123 -9
View File
@@ -63,7 +63,7 @@ RVS_TLS = os.getenv("RVS_TLS", "true") # true = wss://, false = ws://
RVS_TLS_FALLBACK = os.getenv("RVS_TLS_FALLBACK", "true") # Bei TLS-Fehler ws:// versuchen
RVS_TOKEN = os.getenv("RVS_TOKEN", "") # Pairing-Token (gleich wie in der App)
DIAGNOSTIC_URL = os.getenv("DIAGNOSTIC_URL", "http://127.0.0.1:3001") # Diagnostic API
WHISPER_MODEL = os.getenv("WHISPER_MODEL", "small")
WHISPER_MODEL = os.getenv("WHISPER_MODEL", "medium")
WHISPER_LANGUAGE = os.getenv("WHISPER_LANGUAGE", "de")
# Audio-Parameter
@@ -201,11 +201,23 @@ class VoiceEngine:
return None
try:
# Langen Text in Saetze aufteilen (Piper hat Limits bei langen Texten)
# Markdown + Sonderzeichen entfernen fuer natuerliche Sprache
import re
sentences = re.split(r'(?<=[.!?])\s+', text.strip())
# Markdown-Formatierung entfernen
sentences = [re.sub(r'\*\*([^*]+)\*\*', r'\1', s).strip() for s in sentences if s.strip()]
clean = text.strip()
clean = re.sub(r'\*\*([^*]+)\*\*', r'\1', clean) # **fett**
clean = re.sub(r'\*([^*]+)\*', r'\1', clean) # *kursiv*
clean = re.sub(r'`[^`]+`', '', clean) # `code`
clean = re.sub(r'```[\s\S]*?```', '', clean) # Code-Bloecke
clean = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', clean) # [text](url)
clean = re.sub(r'#{1,6}\s*', '', clean) # ### Ueberschriften
clean = re.sub(r'>\s*', '', clean) # > Zitate
clean = re.sub(r'[-*]\s+', '', clean) # Listen
clean = re.sub(r'\n{2,}', '. ', clean) # Absaetze
clean = re.sub(r'\n', ', ', clean) # Zeilenumbrueche
clean = re.sub(r'\s{2,}', ' ', clean) # Mehrfach-Leerzeichen
clean = re.sub(r'["""„]', '', clean) # Anfuehrungszeichen
sentences = re.split(r'(?<=[.!?])\s+', clean)
sentences = [s.strip() for s in sentences if s.strip()]
if not sentences:
return None
@@ -318,6 +330,25 @@ class STTEngine:
self.model = WhisperModel(self.model_size, device="cpu", compute_type="int8")
logger.info("Whisper-Modell geladen")
def reload(self, model_size: str) -> bool:
"""Laedt ein anderes Whisper-Modell (bei Config-Aenderung)."""
if model_size == self.model_size and self.model is not None:
return False
allowed = {"tiny", "base", "small", "medium", "large-v3"}
if model_size not in allowed:
logger.warning("Ungueltiges Whisper-Modell: %s (erlaubt: %s)", model_size, allowed)
return False
logger.info("Lade Whisper-Modell neu: %s -> %s", self.model_size, model_size)
self.model_size = model_size
self.model = None
try:
self.model = WhisperModel(model_size, device="cpu", compute_type="int8")
logger.info("Whisper-Modell '%s' geladen", model_size)
return True
except Exception:
logger.exception("Whisper-Modell '%s' konnte nicht geladen werden", model_size)
return False
def transcribe(self, audio_data: np.ndarray) -> str:
"""Transkribiert Audio-Daten zu Text.
@@ -490,6 +521,7 @@ class ARIABridge:
# Komponenten
self.voice_engine = VoiceEngine(VOICES_DIR)
self.tts_enabled = True
vc: dict = {}
# Gespeicherte Voice-Config laden
try:
vc_path = "/shared/config/voice_config.json"
@@ -508,8 +540,10 @@ class ARIABridge:
logger.info("Voice-Config geladen: %s", vc)
except Exception as e:
logger.warning("Voice-Config laden fehlgeschlagen: %s", e)
# Whisper-Modell: Config hat Vorrang, dann env/Default (medium)
whisper_model = vc.get("whisperModel") or self.config.get("WHISPER_MODEL", WHISPER_MODEL)
self.stt_engine = STTEngine(
model_size=self.config.get("WHISPER_MODEL", WHISPER_MODEL),
model_size=whisper_model,
language=self.config.get("WHISPER_LANGUAGE", WHISPER_LANGUAGE),
)
self.wake_word = WakeWordDetector()
@@ -518,6 +552,9 @@ class ARIABridge:
self.ws_core: Optional[websockets.WebSocketClientProtocol] = None
self.ws_rvs: Optional[websockets.WebSocketClientProtocol] = None
# Letzter gesendeter agent_activity-State (zum Entduplizieren)
self._last_activity_state: Optional[tuple] = None
def initialize(self) -> None:
"""Initialisiert alle Komponenten.
@@ -722,8 +759,18 @@ class ARIABridge:
if event_name == "agent":
data = payload.get("data", {})
delta = data.get("delta", "")
if delta and payload.get("stream") == "assistant":
stream = payload.get("stream", "")
if delta and stream == "assistant":
logger.debug("[core] Delta: '%s'", delta[:40])
# Activity-Signal zur App (entdupliziert)
tool_name = data.get("name") or data.get("tool") or payload.get("tool") or ""
if stream == "tool_use" or data.get("type") == "tool_use":
activity = "tool"
elif stream == "assistant":
activity = "assistant"
else:
activity = "thinking"
await self._emit_activity(activity, tool_name)
return
# ── chat Events: Snapshots mit state=delta|final|error ──
@@ -732,6 +779,7 @@ class ARIABridge:
if state == "final":
text = self._extract_chat_text(payload)
await self._emit_activity("idle", "")
if not text:
logger.warning("[core] chat final ohne Text: %s", json.dumps(payload)[:200])
return
@@ -742,6 +790,7 @@ class ARIABridge:
if state == "error":
error = payload.get("error", "Unbekannt")
logger.error("[core] Chat-Fehler: %s", error)
await self._emit_activity("idle", "")
await self._send_to_rvs({
"type": "chat",
"payload": {
@@ -851,7 +900,7 @@ class ARIABridge:
tts_engine = getattr(self, 'tts_engine_type', 'piper')
if tts_engine == "xtts":
# XTTS: Request ueber RVS an Gaming-PC senden
# XTTS: Ganzen Text senden, XTTS-Bridge teilt satzweise auf
xtts_voice = getattr(self, 'xtts_voice', '')
try:
await self._send_to_rvs({
@@ -1045,6 +1094,17 @@ class ARIABridge:
sender = payload.get("sender", "")
if sender in ("aria", "stt"):
return
text = payload.get("text", "")
if text:
logger.info("[rvs] App-Chat: '%s'", text[:80])
await self.send_to_core(text, source="app")
return
if msg_type == "cancel_request":
logger.info("[rvs] Cancel-Request von App — rufe Diagnostic /api/cancel auf")
await self._cancel_via_diagnostic()
await self._emit_activity("idle", "")
return
elif msg_type == "xtts_response":
# XTTS-Audio vom Gaming-PC empfangen → an App weiterleiten
@@ -1125,6 +1185,15 @@ class ARIABridge:
self.voice_engine.speech_speed["thorsten"] = max(0.3, min(2.0, float(payload["speedThorsten"])))
logger.info("[rvs] Speed Thorsten: %.1f", self.voice_engine.speech_speed["thorsten"])
changed = True
whisper_reloaded = False
if "whisperModel" in payload:
new_model = payload["whisperModel"]
if new_model and new_model != self.stt_engine.model_size:
logger.info("[rvs] Whisper-Modell Wechsel: %s -> %s (laedt...)", self.stt_engine.model_size, new_model)
loop = asyncio.get_event_loop()
whisper_reloaded = await loop.run_in_executor(None, self.stt_engine.reload, new_model)
if whisper_reloaded:
changed = True
# Persistent speichern in Shared Volume
if changed:
try:
@@ -1137,6 +1206,7 @@ class ARIABridge:
"xttsVoice": getattr(self, "xtts_voice", ""),
"speedRamona": self.voice_engine.speech_speed.get("ramona", 1.0),
"speedThorsten": self.voice_engine.speech_speed.get("thorsten", 1.0),
"whisperModel": self.stt_engine.model_size,
}
with open("/shared/config/voice_config.json", "w") as f:
json.dump(config_data, f, indent=2)
@@ -1354,10 +1424,24 @@ class ARIABridge:
pass
async def _send_to_rvs(self, message: dict) -> None:
"""Sendet eine Nachricht an die App (via RVS)."""
"""Sendet eine Nachricht an die App (via RVS) mit Verbindungs-Check."""
if self.ws_rvs is None:
return
# Ping-Check: Verbindung wirklich aktiv?
try:
pong = await self.ws_rvs.ping()
await asyncio.wait_for(pong, timeout=5)
except Exception:
logger.warning("[rvs] Ping fehlgeschlagen — Verbindung tot, erzwinge Reconnect")
try:
await self.ws_rvs.close()
except Exception:
pass
self.ws_rvs = None
# Reconnect wird vom connect_to_rvs Loop uebernommen
return
try:
await self.ws_rvs.send(json.dumps(message))
except Exception:
@@ -1365,6 +1449,36 @@ class ARIABridge:
# ── Log-Streaming an die App ─────────────────────────────
async def _cancel_via_diagnostic(self) -> None:
"""Ruft das Diagnostic /api/cancel an — dort laeuft die volle Abbruch-Logik
(openclaw doctor --fix mit Docker-Socket)."""
def _do_request():
try:
req = urllib.request.Request(
f"{self._diagnostic_url}/api/cancel",
method="POST",
data=b"",
)
with urllib.request.urlopen(req, timeout=5) as resp:
return resp.status
except Exception as e:
return f"error: {e}"
status = await asyncio.get_event_loop().run_in_executor(None, _do_request)
logger.info("[cancel] Diagnostic /api/cancel: %s", status)
async def _emit_activity(self, activity: str, tool: str = "") -> None:
"""Sendet agent_activity an die App — nur wenn sich der State geaendert hat."""
state = (activity, tool)
if state == self._last_activity_state:
return
self._last_activity_state = state
await self._send_to_rvs({
"type": "agent_activity",
"payload": {"activity": activity, "tool": tool},
"timestamp": int(asyncio.get_event_loop().time() * 1000),
})
async def send_log_to_app(self, source: str, message: str, level: str = "info") -> None:
"""Sendet einen Log-Eintrag an die App (erscheint im Log-Viewer)."""
await self._send_to_rvs({
+176 -29
View File
@@ -205,8 +205,14 @@
<span><span style="animation:pulse 1s infinite;">&#x1F4AD;</span> <span id="thinking-text">ARIA denkt...</span></span>
<button class="btn secondary" onclick="cancelRequest()" style="padding:2px 10px;font-size:11px;color:#FF3B30;border-color:#FF3B30;">Abbrechen</button>
</div>
<div id="diag-pending-attachments" style="display:none;padding:6px 10px;background:#1E1E2E;border-radius:6px 6px 0 0;margin-bottom:-4px;display:flex;gap:6px;flex-wrap:wrap;align-items:center;">
</div>
<div class="input-row">
<input type="text" id="chat-input" placeholder="Nachricht an ARIA...">
<label class="btn secondary" style="padding:6px 10px;cursor:pointer;font-size:14px;" title="Datei anhaengen">
&#x1F4CE;
<input type="file" id="diag-file-input" multiple accept="image/*,application/pdf,.doc,.docx,.txt" style="display:none;" onchange="handleDiagFileSelect(this.files)">
</label>
<input type="text" id="chat-input" placeholder="Nachricht an ARIA..." onpaste="handleDiagPaste(event)">
<button class="btn" id="btn-gw" onclick="testGateway()">Gateway senden</button>
<button class="btn" id="btn-rvs" onclick="testRVS()">Via RVS senden</button>
</div>
@@ -401,6 +407,12 @@
<div class="settings-section">
<h2>Sprachausgabe</h2>
<div class="card" style="max-width:500px;">
<!-- TTS aktiv (global fuer alle Engines) -->
<div style="display:flex;align-items:center;gap:12px;margin-bottom:12px;">
<label style="color:#8888AA;font-size:12px;">TTS aktiv:</label>
<label class="toggle"><input type="checkbox" id="diag-tts-enabled" checked onchange="sendVoiceConfig()"><span class="slider"></span></label>
</div>
<!-- TTS Engine Auswahl -->
<div style="display:flex;align-items:center;gap:12px;margin-bottom:12px;">
<label style="color:#8888AA;font-size:12px;">TTS Engine:</label>
@@ -426,10 +438,6 @@
<option value="ramona">Ramona (weiblich)</option>
</select>
</div>
<div style="display:flex;align-items:center;gap:12px;margin-bottom:12px;">
<label style="color:#8888AA;font-size:12px;">TTS aktiv:</label>
<label class="toggle"><input type="checkbox" id="diag-tts-enabled" checked onchange="sendVoiceConfig()"><span class="slider"></span></label>
</div>
<div style="margin-bottom:4px;">
<label style="color:#8888AA;font-size:12px;">Ramona Speed: <span id="speed-ramona-label">1.0x</span></label>
</div>
@@ -491,6 +499,30 @@
</div>
</div>
<!-- Whisper (STT) -->
<div class="settings-section">
<h2>Whisper (Spracherkennung)</h2>
<div style="font-size:11px;color:#8888AA;margin-bottom:8px;">
Aenderungen werden sofort an die Bridge gesendet und das Modell neu geladen
(kann bei medium/large 10-30s dauern — waehrend dieser Zeit ist STT kurz pausiert).
</div>
<div class="card" style="max-width:500px;">
<div style="display:flex;align-items:center;gap:12px;margin-bottom:8px;">
<label style="color:#8888AA;font-size:12px;min-width:80px;">Modell:</label>
<select id="diag-whisper-model" onchange="sendVoiceConfig()" style="flex:1;background:#1E1E2E;color:#fff;border:1px solid #2A2A3E;border-radius:6px;padding:6px 10px;font-size:13px;">
<option value="tiny">tiny (39MB, schnell, niedrige Qualitaet)</option>
<option value="base">base (74MB, schnell, ok)</option>
<option value="small">small (244MB, mittel)</option>
<option value="medium" selected>medium (769MB, gut — Empfehlung)</option>
<option value="large-v3">large-v3 (1.5GB, beste Qualitaet, langsam auf CPU)</option>
</select>
</div>
<div style="font-size:10px;color:#555570;">
Tipp: <code>medium</code> ist der beste Kompromiss fuer CPU. <code>large-v3</code> nur bei GPU sinnvoll.
</div>
</div>
</div>
<!-- Highlight-Trigger -->
<div class="settings-section">
<h2>Highlight-Trigger</h2>
@@ -744,8 +776,22 @@
document.getElementById('diag-speed-thorsten').value = st;
document.getElementById('speed-thorsten-label').textContent = st + 'x';
document.getElementById('diag-tts-engine').value = msg.ttsEngine || 'piper';
document.getElementById('diag-xtts-voice').value = msg.xttsVoice || '';
// XTTS-Voice setzen — Option hinzufuegen falls nicht vorhanden
const xttsSelect = document.getElementById('diag-xtts-voice');
const xttsVoice = msg.xttsVoice || '';
if (xttsVoice && !Array.from(xttsSelect.options).some(o => o.value === xttsVoice)) {
const opt = document.createElement('option');
opt.value = xttsVoice;
opt.textContent = xttsVoice;
xttsSelect.appendChild(opt);
}
xttsSelect.value = xttsVoice;
toggleXTTSPanel();
// Whisper-Modell wiederherstellen (falls gesetzt)
if (msg.whisperModel) {
const wSel = document.getElementById('diag-whisper-model');
if (wSel) wSel.value = msg.whisperModel;
}
return;
}
@@ -874,6 +920,18 @@
else alert('Loeschen fehlgeschlagen: ' + (msg.error || '?'));
return;
}
if (msg.type === 'session_export') {
if (!msg.ok) { alert('Export fehlgeschlagen: ' + (msg.error || '?')); return; }
const blob = new Blob([msg.markdown], { type: 'text/markdown;charset=utf-8' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = msg.filename;
document.body.appendChild(a);
a.click();
setTimeout(() => { URL.revokeObjectURL(url); a.remove(); }, 100);
return;
}
if (msg.type === 'active_session') {
updateActiveSessionBar(msg.sessionKey);
loadSessions(); // Tabelle neu rendern
@@ -928,21 +986,39 @@
}
}
function sendDiagAttachments() {
// Alle pending Dateien an RVS senden
for (const f of diagPendingFiles) {
send({ action: 'send_file', name: f.name, type: f.type, size: f.size, base64: f.base64 });
}
if (diagPendingFiles.length > 0) {
addChat('sent', `${diagPendingFiles.length} Anhang/Anhaenge`, 'Datei');
}
diagPendingFiles = [];
renderDiagPending();
}
function testGateway() {
const input = document.getElementById('chat-input');
const text = input.value.trim();
if (!text) return;
addChat('sent', text, 'Gateway direkt');
send({ action: 'test_gateway', text });
if (!text && diagPendingFiles.length === 0) return;
if (diagPendingFiles.length > 0) sendDiagAttachments();
if (text) {
addChat('sent', text, 'Gateway direkt');
send({ action: 'test_gateway', text });
}
input.value = '';
}
function testRVS() {
const input = document.getElementById('chat-input');
const text = input.value.trim();
if (!text) return;
addChat('sent', text, 'via RVS');
send({ action: 'test_rvs', text });
if (!text && diagPendingFiles.length === 0) return;
if (diagPendingFiles.length > 0) sendDiagAttachments();
if (text) {
addChat('sent', text, 'via RVS');
send({ action: 'test_rvs', text });
}
input.value = '';
}
@@ -1247,7 +1323,16 @@
}
function loadXTTSVoices() {
sendToRVS_raw({ type: 'xtts_list_voices', payload: {}, timestamp: Date.now() });
send({ action: 'xtts_list_voices' });
}
function arrayBufferToBase64(buffer) {
const bytes = new Uint8Array(buffer);
let binary = '';
for (let i = 0; i < bytes.length; i += 8192) {
binary += String.fromCharCode.apply(null, bytes.subarray(i, i + 8192));
}
return btoa(binary);
}
async function uploadVoiceSamples() {
@@ -1255,25 +1340,81 @@
const files = document.getElementById('xtts-clone-files').files;
if (!name) { alert('Bitte einen Namen eingeben'); return; }
if (!files || files.length === 0) { alert('Bitte Audio-Dateien auswaehlen'); return; }
if (files.length > 10) { alert('Maximal 10 Dateien'); return; }
document.getElementById('xtts-clone-status').textContent = `Lade ${files.length} Datei(en) hoch...`;
const status = document.getElementById('xtts-clone-status');
status.textContent = `Lade ${files.length} Datei(en)...`;
status.style.color = '#FFD60A';
const samples = [];
for (const file of files) {
const buffer = await file.arrayBuffer();
const base64 = btoa(String.fromCharCode(...new Uint8Array(buffer)));
samples.push({ base64, name: file.name, size: file.size });
try {
const samples = [];
for (let i = 0; i < files.length; i++) {
status.textContent = `Lese Datei ${i + 1}/${files.length}: ${files[i].name}...`;
const buffer = await files[i].arrayBuffer();
const base64 = arrayBufferToBase64(buffer);
samples.push({ base64, name: files[i].name, size: files[i].size });
}
const totalSize = samples.reduce((s, f) => s + f.size, 0);
status.textContent = `Sende ${samples.length} Sample(s) (${(totalSize / 1024).toFixed(0)}KB)...`;
send({ action: 'voice_upload', name, samples });
status.textContent = `Gesendet — warte auf Bestaetigung vom XTTS-Server...`;
} catch (err) {
status.textContent = `Fehler: ${err.message}`;
status.style.color = '#FF3B30';
}
}
const totalSize = samples.reduce((s, f) => s + f.size, 0);
document.getElementById('xtts-clone-status').textContent =
`Sende ${samples.length} Sample(s) (${(totalSize / 1024).toFixed(0)}KB) an XTTS-Server...`;
// ── Diagnostic Anhang-Handling ─────────────
let diagPendingFiles = [];
sendToRVS_raw({
type: 'voice_upload',
payload: { name, samples },
timestamp: Date.now(),
});
function handleDiagFileSelect(files) {
for (const file of files) {
const reader = new FileReader();
reader.onload = () => {
const base64 = reader.result.split(',')[1];
diagPendingFiles.push({ name: file.name, type: file.type, size: file.size, base64 });
renderDiagPending();
};
reader.readAsDataURL(file);
}
}
function handleDiagPaste(event) {
const items = event.clipboardData?.items;
if (!items) return;
for (const item of items) {
if (item.kind === 'file') {
event.preventDefault();
const file = item.getAsFile();
if (file) handleDiagFileSelect([file]);
}
}
}
function renderDiagPending() {
const container = document.getElementById('diag-pending-attachments');
if (diagPendingFiles.length === 0) {
container.style.display = 'none';
return;
}
container.style.display = 'flex';
container.innerHTML = diagPendingFiles.map((f, i) => {
const isImage = f.type.startsWith('image/');
const preview = isImage ? `<img src="data:${f.type};base64,${f.base64}" style="width:40px;height:40px;border-radius:4px;object-fit:cover;">` : `<span style="font-size:20px;">&#x1F4C4;</span>`;
return `<div style="position:relative;display:inline-block;">
${preview}
<span onclick="removeDiagPending(${i})" style="position:absolute;top:-4px;right:-4px;width:16px;height:16px;border-radius:8px;background:#FF3B30;color:#fff;font-size:10px;cursor:pointer;display:flex;align-items:center;justify-content:center;">X</span>
</div>`;
}).join('') + `<span style="color:#8888AA;font-size:11px;margin-left:4px;">${diagPendingFiles.length} Datei(en)</span>
<span onclick="diagPendingFiles=[];renderDiagPending();" style="color:#FF3B30;font-size:11px;cursor:pointer;margin-left:8px;">Alle X</span>`;
}
function removeDiagPending(idx) {
diagPendingFiles.splice(idx, 1);
renderDiagPending();
}
// ── Abbrechen ──────────────────────────────
@@ -1292,7 +1433,8 @@
const speedThorsten = parseFloat(document.getElementById('diag-speed-thorsten').value);
const ttsEngine = document.getElementById('diag-tts-engine').value;
const xttsVoice = document.getElementById('diag-xtts-voice').value;
send({ action: 'send_voice_config', defaultVoice, highlightVoice, ttsEnabled, speedRamona, speedThorsten, ttsEngine, xttsVoice });
const whisperModel = document.getElementById('diag-whisper-model').value;
send({ action: 'send_voice_config', defaultVoice, highlightVoice, ttsEnabled, speedRamona, speedThorsten, ttsEngine, xttsVoice, whisperModel });
}
// ── Highlight-Trigger ────────────────────────
@@ -1579,7 +1721,8 @@
+ `<td style="padding:4px 6px;color:#8888AA;font-size:10px;">${date}</td>`
+ `<td style="padding:4px 6px;white-space:nowrap;">`
+ (isActive ? '' : `<button class="btn secondary" onclick="event.stopPropagation();activateSession('${escapeHtml(s.sessionKey)}')" style="padding:2px 6px;font-size:10px;color:#34C759;margin-right:2px;" title="Aktivieren">&#9654;</button>`)
+ `<button class="btn secondary" onclick="event.stopPropagation();deleteSession('${escapeHtml(s.path)}')" style="padding:2px 6px;font-size:10px;color:#FF6B6B;" title="Loeschen">X</button>`
+ `<button class="btn secondary" onclick="event.stopPropagation();deleteSession('${escapeHtml(s.path)}')" style="padding:2px 6px;font-size:10px;color:#FF6B6B;margin-right:2px;" title="Loeschen">X</button>`
+ `<button class="btn secondary" onclick="event.stopPropagation();exportSession('${escapeHtml(s.path)}','${escapeHtml(s.sessionKey)}')" style="padding:2px 6px;font-size:10px;color:#8888AA;" title="Als Markdown exportieren">&#x2B07;</button>`
+ `</td></tr>`;
}
html += '</table>';
@@ -1643,6 +1786,10 @@
send({ action: 'delete_session', sessionPath: path });
}
function exportSession(path, sessionKey) {
send({ action: 'export_session', sessionPath: path, sessionKey });
}
function activateSession(sessionKey) {
send({ action: 'set_active_session', sessionKey });
}
+182 -20
View File
@@ -37,15 +37,41 @@ const state = {
};
const SESSION_KEY_FILE = "/data/active-session";
// /data Verzeichnis sicherstellen (Volume Mount)
try { fs.mkdirSync("/data", { recursive: true }); } catch {}
try { fs.mkdirSync("/data", { recursive: true }); } catch (e) {
console.error(`[startup] /data mkdir fehlgeschlagen: ${e.message}`);
}
// sessionFromFile zeigt an, ob der aktive Key aus der Datei kam.
// Wenn true, darf resolveActiveSession NICHT mehr auto-picken (Wahl respektieren).
let sessionFromFile = false;
let activeSessionKey = (() => {
try {
const saved = fs.readFileSync(SESSION_KEY_FILE, "utf-8").trim();
if (saved) { console.log(`[startup] Gespeicherte Session geladen: '${saved}'`); return saved; }
} catch {}
if (saved) {
console.log(`[startup] Gespeicherte Session geladen: '${saved}'`);
sessionFromFile = true;
return saved;
}
} catch (e) {
console.error(`[startup] SESSION_KEY_FILE read: ${e.code || e.message}`);
}
console.log("[startup] Keine gespeicherte Session — Fallback 'main'");
return "main";
})();
// Atomic write: temp-file + rename, laute Logs bei Fehler.
function persistActiveSession(key) {
try {
const tmp = SESSION_KEY_FILE + ".tmp";
fs.writeFileSync(tmp, key);
fs.renameSync(tmp, SESSION_KEY_FILE);
sessionFromFile = true;
console.log(`[session] Aktive Session persistiert: '${key}'`);
return true;
} catch (e) {
console.error(`[session] FEHLER beim Persistieren von '${key}': ${e.message}`);
return false;
}
}
const logs = [];
let gatewayWs = null;
let rvsWs = null;
@@ -91,6 +117,9 @@ function pipelineEnd(ok, detail) {
}
plog(`━━━ Pipeline Ende ━━━`);
pipelineActive = false;
// Thinking-Indikator IMMER zuruecksetzen — auch bei Timeout/Fehler/Abbruch
broadcast({ type: "agent_activity", activity: "idle" });
pendingMessageTime = 0;
}
// ── Auto-Restart bei Netzwerk-Namespace-Verlust ──────
@@ -257,8 +286,10 @@ async function connectGateway() {
state.gateway.handshakeOk = false;
gatewayWs = null;
broadcastState();
// Stuck "ARIA denkt..." vermeiden, falls Gateway waehrend Pipeline abkackt
if (pipelineActive) pipelineEnd(false, `Gateway-Verbindung verloren (${code})`);
else broadcast({ type: "agent_activity", activity: "idle" });
checkGatewayHealth();
// Auto-Reconnect nach 5s
setTimeout(connectGateway, 5000);
});
@@ -372,6 +403,7 @@ function handleGatewayMessage(msg) {
const error = payload.error || text || "Unbekannt";
log("error", "gateway", `Chat-Fehler: ${error}`);
if (pipelineActive) pipelineEnd(false, error);
else broadcast({ type: "agent_activity", activity: "idle" });
broadcast({ type: "chat_error", error, payload });
return;
}
@@ -393,6 +425,7 @@ function handleGatewayMessage(msg) {
const text = extractChatText(payload) || payload.text || "";
log("info", "gateway", `ANTWORT: "${text.slice(0, 200)}"`);
if (pipelineActive) pipelineEnd(true, `"${text.slice(0, 120)}"`);
else broadcast({ type: "agent_activity", activity: "idle" });
broadcast({ type: "chat_final", text, payload });
return;
}
@@ -400,6 +433,7 @@ function handleGatewayMessage(msg) {
const error = payload.error || payload.message || "Unbekannt";
log("error", "gateway", `Chat-Fehler: ${error}`);
if (pipelineActive) pipelineEnd(false, error);
else broadcast({ type: "agent_activity", activity: "idle" });
broadcast({ type: "chat_error", error, payload });
return;
}
@@ -560,6 +594,31 @@ function connectRVS(forcePlain) {
});
}
function sendToRVS_withResponse(sendType, sendPayload, expectType, clientWs) {
if (!RVS_HOST || !RVS_TOKEN) return;
const proto = RVS_TLS === "true" ? "wss" : "ws";
const url = `${proto}://${RVS_HOST}:${RVS_PORT}?token=${RVS_TOKEN}`;
const freshWs = new WebSocket(url);
const timeout = setTimeout(() => {
try { freshWs.close(); } catch (_) {}
clientWs.send(JSON.stringify({ type: expectType, payload: { voices: [], error: "Timeout" }, timestamp: Date.now() }));
}, 15000);
freshWs.on("open", () => {
freshWs.send(JSON.stringify({ type: sendType, payload: sendPayload, timestamp: Date.now() }));
});
freshWs.on("message", (raw) => {
try {
const resp = JSON.parse(raw.toString());
if (resp.type === expectType) {
clearTimeout(timeout);
clientWs.send(JSON.stringify(resp));
setTimeout(() => { try { freshWs.close(); } catch (_) {} }, 1000);
}
} catch {}
});
freshWs.on("error", () => {});
}
function sendToRVS_raw(msgObj) {
if (!RVS_HOST || !RVS_TOKEN) return;
const proto = RVS_TLS === "true" ? "wss" : "ws";
@@ -1084,6 +1143,16 @@ const server = http.createServer((req, res) => {
} else if (req.url === "/api/session") {
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ sessionKey: activeSessionKey }));
} else if (req.url === "/api/cancel" && req.method === "POST") {
log("warn", "server", "HTTP /api/cancel — Cancel-Request (von Bridge)");
pendingMessageTime = 0;
watchdogWarned = false;
watchdogFixAttempted = false;
if (pipelineActive) pipelineEnd(false, "Vom Benutzer abgebrochen (App)");
else broadcast({ type: "agent_activity", activity: "idle" });
dockerExec("aria-core", "openclaw doctor --fix 2>/dev/null || true").catch(() => {});
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ ok: true }));
} else if (req.url.startsWith("/shared/")) {
// Dateien aus Shared Volume ausliefern (Bilder, Uploads)
const filePath = decodeURIComponent(req.url);
@@ -1156,6 +1225,14 @@ wss.on("connection", (ws) => {
if (ws._sshSock) ws._sshSock.write(msg.data);
} else if (msg.action === "live_ssh_close") {
if (ws._sshSock) { ws._sshSock.end(); ws._sshSock = null; }
} else if (msg.action === "send_file") {
// Datei von Diagnostic an Bridge via RVS senden
sendToRVS_raw({
type: "file",
payload: { name: msg.name, type: msg.type, size: msg.size, base64: msg.base64 },
timestamp: Date.now(),
});
log("info", "server", `Datei gesendet: ${msg.name} (${msg.type})`);
} else if (msg.action === "cancel_request") {
// Laufende Anfrage abbrechen — doctor --fix beendet stuck runs
log("warn", "server", "Anfrage abgebrochen — fuehre doctor --fix aus");
@@ -1165,11 +1242,22 @@ wss.on("connection", (ws) => {
if (pipelineActive) pipelineEnd(false, "Vom Benutzer abgebrochen");
broadcast({ type: "agent_activity", activity: "idle" });
dockerExec("aria-core", "openclaw doctor --fix 2>/dev/null || true").catch(() => {});
} else if (msg.action === "voice_upload") {
// Voice-Samples an XTTS-Bridge via RVS weiterleiten, auf Bestätigung warten
log("info", "server", `Voice-Upload '${msg.name}' (${(msg.samples || []).length} Samples) sende an RVS...`);
sendToRVS_withResponse("voice_upload", { name: msg.name, samples: msg.samples }, "xtts_voice_saved", ws);
} else if (msg.action === "xtts_list_voices") {
// Frische Verbindung die auf Antwort wartet
sendToRVS_withResponse("xtts_list_voices", {}, "xtts_voices_list", ws);
} else if (msg.action === "get_voice_config") {
handleGetVoiceConfig(ws);
} else if (msg.action === "send_voice_config") {
// Stimmen-Config persistent speichern + an Bridge via RVS senden
// Bestehende Config lesen um Felder zu mergen die dieser Call nicht setzt
let existing = {};
try { existing = JSON.parse(fs.readFileSync("/shared/config/voice_config.json", "utf-8")); } catch {}
const voiceConfig = {
...existing,
defaultVoice: msg.defaultVoice || "ramona",
highlightVoice: msg.highlightVoice || "thorsten",
ttsEnabled: msg.ttsEnabled !== false,
@@ -1178,12 +1266,13 @@ wss.on("connection", (ws) => {
speedRamona: msg.speedRamona || 1.0,
speedThorsten: msg.speedThorsten || 1.0,
};
if (msg.whisperModel !== undefined) voiceConfig.whisperModel = msg.whisperModel;
try {
fs.mkdirSync("/shared/config", { recursive: true });
fs.writeFileSync("/shared/config/voice_config.json", JSON.stringify(voiceConfig, null, 2));
} catch {}
sendToRVS_raw({ type: "config", payload: voiceConfig, timestamp: Date.now() });
log("info", "server", `Voice-Config gespeichert+gesendet: default=${voiceConfig.defaultVoice}, highlight=${voiceConfig.highlightVoice}, tts=${voiceConfig.ttsEnabled}`);
log("info", "server", `Voice-Config gespeichert+gesendet: default=${voiceConfig.defaultVoice}, whisper=${voiceConfig.whisperModel || "-"}`);
} else if (msg.action === "get_triggers") {
handleGetTriggers(ws);
} else if (msg.action === "save_triggers") {
@@ -1200,6 +1289,8 @@ wss.on("connection", (ws) => {
handleListSessions(ws);
} else if (msg.action === "read_session") {
handleReadSession(ws, msg.sessionPath);
} else if (msg.action === "export_session") {
handleExportSession(ws, msg.sessionPath, msg.sessionKey);
} else if (msg.action === "delete_session") {
handleDeleteSession(ws, msg.sessionPath);
} else if (msg.action === "set_active_session") {
@@ -1582,6 +1673,68 @@ async function handleReadSession(clientWs, sessionPath) {
}
}
async function handleExportSession(clientWs, sessionPath, sessionKey) {
if (!sessionPath || sessionPath.includes("..") || !sessionPath.startsWith(SESSIONS_DIR)) {
clientWs.send(JSON.stringify({ type: "session_export", ok: false, error: "Ungueltiger Pfad" }));
return;
}
try {
const safePath = sessionPath.replace(/'/g, "");
const raw = await dockerExec("aria-core", `cat '${safePath}'`);
const lines = raw.split("\n").filter(l => l.trim());
const blocks = [];
for (const line of lines) {
let obj;
try { obj = JSON.parse(line); } catch { continue; }
if (obj.type !== "message" || !obj.message) continue;
const role = obj.message.role;
if (role !== "user" && role !== "assistant") continue;
let text = "";
const content = obj.message.content;
if (typeof content === "string") text = content;
else if (Array.isArray(content)) text = content.filter(c => c.type === "text").map(c => c.text || "").join("\n");
if (!text) continue;
if (role === "user") {
text = text.replace(/^Sender \(untrusted metadata\):[\s\S]*?```[\s\S]*?```\s*\n*/m, "").trim();
text = text.replace(/^\[.*?\]\s*/, "").trim();
} else {
text = text.replace(/^\[\[reply_to_\w+\]\]\s*/g, "").trim();
}
if (!text) continue;
const ts = obj.message.timestamp || obj.timestamp || 0;
const when = ts ? new Date(ts).toISOString().replace("T", " ").slice(0, 19) : "";
const heading = role === "user" ? "## 🧑 User" : "## 🤖 ARIA";
blocks.push(`${heading}${when ? `${when}` : ""}\n\n${text}`);
}
const exportedAt = new Date().toISOString().replace("T", " ").slice(0, 19);
const title = sessionKey || sessionPath.split("/").pop().replace(".jsonl", "");
const markdown = [
`# Session: ${title}`,
``,
`Exportiert: ${exportedAt} `,
`Quelle: ${sessionPath}`,
``,
`---`,
``,
blocks.join("\n\n---\n\n"),
``,
].join("\n");
const safeKey = (sessionKey || "session").replace(/[^a-zA-Z0-9_-]/g, "_");
const filename = `${exportedAt.slice(0, 10)}_${safeKey}.md`;
clientWs.send(JSON.stringify({ type: "session_export", ok: true, filename, markdown }));
log("info", "server", `Session exportiert: ${filename} (${blocks.length} Nachrichten)`);
} catch (err) {
log("error", "server", `Session-Export fehlgeschlagen: ${err.message}`);
clientWs.send(JSON.stringify({ type: "session_export", ok: false, error: err.message }));
}
}
async function handleDeleteSession(clientWs, sessionPath) {
if (!sessionPath || sessionPath.includes("..") || !sessionPath.startsWith(SESSIONS_DIR)) {
clientWs.send(JSON.stringify({ type: "session_deleted", ok: false, error: "Ungueltiger Pfad" }));
@@ -1622,13 +1775,11 @@ async function handleDeleteSession(clientWs, sessionPath) {
}
// ── Session-Aufloesung: letzte aktive Session finden ────
// Wird nach Gateway-(Re-)Connect aufgerufen. Darf die explizit gewaehlte
// Session NIE ueberschreiben — nur beim absoluten Erststart auto-picken.
async function resolveActiveSession() {
// Nur bei Fallback-Key "main" automatisch aufloesen — gespeicherte Wahl respektieren
const hasSavedSession = (() => {
try { return !!fs.readFileSync(SESSION_KEY_FILE, "utf-8").trim(); } catch { return false; }
})();
if (hasSavedSession && activeSessionKey !== "main") {
log("info", "server", `Gespeicherte Session '${activeSessionKey}' wird beibehalten`);
if (sessionFromFile) {
log("info", "server", `Session '${activeSessionKey}' aus /data — keine Auto-Wahl`);
return;
}
@@ -1647,10 +1798,19 @@ async function resolveActiveSession() {
const keys = entries.map(e => (e.key || e.sessionKey || e.name || "?").replace(/^agent:main:/, ""));
log("info", "server", `Verfuegbare Sessions: [${keys.join(", ")}]`);
// Neueste Session nehmen
// Neueste Session nehmen — aber user-definierte bevorzugen.
// aria-bridge / aria-diagnostic werden von den Services auto-erstellt;
// bei erstem Start soll lieber eine "echte" Session gewaehlt werden,
// falls vorhanden.
const AUTO_KEYS = new Set(["aria-bridge", "aria-diagnostic"]);
const normalise = (e) => (e.key || e.sessionKey || e.name || "").replace(/^agent:main:/, "");
const userEntries = entries.filter(e => !AUTO_KEYS.has(normalise(e)));
const pool = userEntries.length > 0 ? userEntries : entries;
let newest = null;
let newestTime = 0;
for (const entry of entries) {
for (const entry of pool) {
const t = entry.updatedAt || entry.createdAt || 0;
if (t >= newestTime) {
newestTime = t;
@@ -1659,12 +1819,11 @@ async function resolveActiveSession() {
}
if (newest) {
const rawKey = newest.key || newest.sessionKey || newest.name || "";
const key = rawKey.replace(/^agent:main:/, "");
const key = normalise(newest);
if (key) {
activeSessionKey = key;
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
log("info", "server", `Aktive Session auf neueste gewechselt: '${activeSessionKey}'`);
persistActiveSession(activeSessionKey);
log("info", "server", `Auto-Wahl Erststart: '${activeSessionKey}'`);
for (const c of browserClients) {
c.send(JSON.stringify({ type: "active_session", sessionKey: activeSessionKey }));
}
@@ -1753,8 +1912,11 @@ function handleSetActiveSession(clientWs, sessionKey) {
return;
}
activeSessionKey = sessionKey;
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
log("info", "server", `Aktive Session: ${activeSessionKey}`);
const ok = persistActiveSession(activeSessionKey);
log("info", "server", `Aktive Session: ${activeSessionKey}${ok ? "" : " (WARN: nicht persistiert!)"}`);
if (!ok) {
clientWs.send(JSON.stringify({ type: "active_session", ok: false, sessionKey: activeSessionKey, error: "Persistierung fehlgeschlagen — /data Volume pruefen" }));
}
// Allen Clients mitteilen
for (const c of browserClients) {
c.send(JSON.stringify({ type: "active_session", sessionKey: activeSessionKey }));
@@ -1770,7 +1932,7 @@ async function handleCreateSession(clientWs, sessionName) {
try {
// Session wird automatisch erstellt wenn man die erste Nachricht sendet
activeSessionKey = sessionName;
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
persistActiveSession(activeSessionKey);
log("info", "server", `Neue Session erstellt und aktiviert: ${sessionName}`);
// Allen Clients mitteilen
for (const c of browserClients) {
+1 -1
View File
@@ -18,7 +18,7 @@ services:
claude-max-api"
volumes:
- ~/.claude:/root/.claude # Claude CLI Auth (Credentials in /root/.claude/.credentials.json)
- ./aria-data/ssh:/root/.ssh:ro # SSH Keys fuer VM-Zugriff (aria-wohnung)
- ./aria-data/ssh:/root/.ssh # SSH Keys fuer VM-Zugriff (aria-wohnung, rw fuer ARIA)
- aria-shared:/shared # Shared Volume fuer Datei-Austausch (Uploads von App)
environment:
- HOST=0.0.0.0
+32 -8
View File
@@ -6,9 +6,9 @@
- [x] Sprachnachrichten werden als Text angezeigt (STT → Chat-Bubble)
- [x] Cache leeren + Auto-Download von Anhaengen
- [x] ARIA liest Nachrichten vor (TTS via Piper)
- [x] Autoscroll zur letzten Nachricht
- [x] Autoscroll zur letzten Nachricht (inverted FlatList)
- [x] Bilder im Chat groesser + Vollbild-Vorschau
- [x] Ohr-Button Absturz gefixt (LiveAudioStream entfernt, Phase 1 Placeholder)
- [x] Ohr-Button → Gespraechsmodus (Auto-Aufnahme nach ARIA-Antwort)
- [x] Play-Button in ARIA-Nachrichten fuer Sprachwiedergabe
- [x] Chat-Suche in der App (Lupe in Statusleiste)
- [x] Watchdog mit Container-Restart (2min Warnung → 5min doctor --fix → 8min Restart)
@@ -18,19 +18,43 @@
- [x] RVS Nachrichten vom Smartphone gehen durch
- [x] Stimmen-Einstellungen (Ramona/Thorsten, Speed pro Stimme)
- [x] Highlight-Trigger konfigurierbar in Diagnostic
- [x] XTTS v2 Integration (Gaming-PC, GPU, Voice Cloning)
- [x] XTTS Voice Cloning (Audio-Samples hochladen, eigene Stimme)
- [x] TTS Engine waehlbar (Piper/XTTS) in Diagnostic + App
- [x] Auto-Update System (APK via RVS WebSocket)
- [x] Auto-Update: APK-Installation via FileProvider
- [x] Auto-Update: "Auf Updates pruefen" Button in App-Einstellungen
- [x] Audio-Queue (sequentielle Wiedergabe, kein Ueberlappen)
- [x] Textnachrichten werden von ARIA beantwortet (Bridge chat handler fix)
- [x] Mehrere Anhaenge + Text vor dem Senden (Pending-Vorschau)
- [x] Paste-Support fuer Bilder in Diagnostic Chat
- [x] Markdown-Bereinigung fuer TTS (fett, kursiv, code, links, etc.)
- [x] SSH Volume read-write fuer Proxy (kein -F Workaround mehr)
- [x] Diagnostic: Sessions als Markdown exportieren (Download-Button)
- [x] Speech Gate: Aufnahme wird verworfen wenn keine Sprache erkannt (verhindert dass Umgebungsgeraeusche an Whisper gehen)
- [x] Session-Persistenz: Gewaehlte Session bleibt ueber Container-Restarts erhalten (sessionFromFile-Flag, atomic write)
- [x] Diagnostic: "ARIA denkt..." bleibt nicht mehr stehen (pipelineEnd broadcastet immer idle, auch bei Timeout/Fehler/Disconnect)
- [x] App: "ARIA denkt..." Indicator + Abbrechen-Button (Bridge spiegelt agent_activity via RVS)
- [x] Whisper STT: Model-Auswahl in Diagnostic (tiny/base/small/medium/large-v3), Hot-Reload in Bridge, Default auf medium
- [x] App: Audio-Aufnahme explizit 16kHz mono (spart Resample, optimal fuer Whisper)
## Offen
### TTS / Stimmen
- [ ] TTS Engine waehlbar: Piper (CPU, schnell) oder Coqui XTTS v2 (GPU, natuerlicher)
- [ ] Piper Voices Download ueber Diagnostic (neue Sprachen/Stimmen)
- [ ] Coqui XTTS v2 Integration (braucht GPU, bessere deutsche Stimme)
### Bugs (Prioritaet)
- [ ] App: Audioausgabe hoert ab und zu einfach auf (mitten im Satz oder zwischen Chunks)
### App
- [ ] Wake Word on-device (Porcupine "ARIA" Keyword, Phase 2)
### App Features
- [ ] Wake Word on-device (Porcupine "ARIA" Keyword, Phase 2 — passives Lauschen)
- [ ] Chat-History zuverlaessiger laden (AsyncStorage Race Condition)
- [ ] Background Audio Service (TTS auch bei minimierter App)
### TTS / Audio
- [ ] XTTS Audio-Streaming (PCM-Stream statt WAV-Dateien, eliminiert Stottern komplett)
- [ ] Audio-Normalisierung (Lautstaerke zwischen Chunks angleichen)
- [ ] Piper Voices Download ueber Diagnostic (neue Sprachen/Stimmen)
### Architektur
- [ ] Bilder: Claude Vision direkt nutzen (aktuell nur Dateipfad an ARIA)
- [ ] Auto-Compacting und Memory/Brain Verwaltung (SQLite?)
- [ ] Diagnostic: System-Info Tab (Container-Status, Disk, RAM, CPU)
- [ ] RVS Zombie-Connections endgueltig loesen
+7 -2
View File
@@ -76,8 +76,11 @@ echo -e " ${GREEN}✓${NC} SettingsScreen → Version $VERSION"
echo ""
# ── APK bauen ─────────────────────────────────
echo -e "${GREEN}[2/5] APK bauen...${NC}"
echo -e "${GREEN}[2/5] APK bauen (Cache leeren + Build)...${NC}"
cd android
# Metro + Gradle Cache leeren damit neue Version sauber eingebettet wird
rm -rf node_modules/.cache 2>/dev/null
cd android && ./gradlew clean 2>/dev/null; cd ..
./build.sh release
cd ..
@@ -174,9 +177,11 @@ fi
RVS_UPDATE_HOST="${RVS_UPDATE_HOST:-}"
if [ -n "$RVS_UPDATE_HOST" ]; then
echo -e "${GREEN}[6/6] APK auf RVS-Server kopieren (Auto-Update)...${NC}"
# Alte APKs auf dem RVS loeschen, dann neue hochladen
ssh "$RVS_UPDATE_HOST" "rm -f ~/ARIA-AGENT/rvs/updates/ARIA-*.apk" 2>/dev/null
scp "$APK_PATH" "${RVS_UPDATE_HOST}:~/ARIA-AGENT/rvs/updates/${APK_NAME}" 2>/dev/null
if [ $? -eq 0 ]; then
echo -e " ${GREEN}${NC} APK auf RVS-Server kopiert — Apps werden benachrichtigt"
echo -e " ${GREEN}${NC} APK auf RVS-Server kopiert (alte Versionen geloescht)"
else
echo -e " ${YELLOW}APK konnte nicht auf RVS kopiert werden (RVS_UPDATE_HOST=$RVS_UPDATE_HOST)${NC}"
echo -e " ${YELLOW}Manuell: scp $APK_PATH $RVS_UPDATE_HOST:~/ARIA-AGENT/rvs/updates/${APK_NAME}${NC}"
+1
View File
@@ -16,6 +16,7 @@ const ALLOWED_TYPES = new Set([
"file_request", "file_response", "file_saved", "stt_result", "config", "tts_request",
"xtts_request", "xtts_response", "xtts_list_voices", "xtts_voices_list", "voice_upload", "xtts_voice_saved",
"update_check", "update_available", "update_download", "update_data",
"agent_activity", "cancel_request",
]);
// Token-Raum: token -> { clients: Set<ws> }
+72 -28
View File
@@ -97,39 +97,83 @@ async function handleTTSRequest(payload) {
const { text, voice, requestId, language } = payload;
if (!text) return;
log(`TTS-Request: "${text.slice(0, 60)}..." (voice: ${voice || "default"}, lang: ${language || "de"})`);
// Markdown + Sonderzeichen entfernen fuer natuerliche Sprache
let cleanText = text
.replace(/\*\*([^*]+)\*\*/g, "$1") // **fett** → fett
.replace(/\*([^*]+)\*/g, "$1") // *kursiv* → kursiv
.replace(/`([^`]+)`/g, "$1") // `code` → code
.replace(/```[\s\S]*?```/g, "") // Code-Bloecke entfernen
.replace(/\[([^\]]+)\]\([^)]+\)/g, "$1") // [text](url) → text
.replace(/#{1,6}\s*/g, "") // ### Ueberschriften → entfernen
.replace(/>\s*/g, "") // > Zitate → entfernen
.replace(/[-*]\s+/g, "") // - Listen → entfernen
.replace(/\n{2,}/g, ". ") // Mehrere Newlines → Punkt
.replace(/\n/g, ", ") // Einzelne Newlines → Komma
.replace(/\s{2,}/g, " ") // Mehrfach-Leerzeichen
.replace(/["""„]/g, "") // Anfuehrungszeichen entfernen
.replace(/\(\)/g, "") // Leere Klammern
.trim();
// Text in Saetze aufteilen, dann zu Chunks von 2-3 Saetzen zusammenfassen
// (mehr Kontext = konsistentere Stimme/Lautstaerke, aber nicht zu lang fuer WebSocket)
const sentences = cleanText.split(/(?<=[.!?])\s+/)
.map(s => s.trim())
.filter(s => s.length > 0)
.map(s => s.replace(/[.]+$/, '')); // Punkt am Ende entfernen
const MAX_CHUNK_CHARS = 150; // Max ~150 Zeichen pro Chunk (schnelles Rendering, Preloading reicht)
const chunks = [];
let currentChunk = '';
for (const sentence of sentences) {
if (currentChunk && (currentChunk.length + sentence.length + 2) > MAX_CHUNK_CHARS) {
chunks.push(currentChunk);
currentChunk = sentence;
} else {
currentChunk = currentChunk ? currentChunk + ', ' + sentence : sentence;
}
}
if (currentChunk) chunks.push(currentChunk);
if (chunks.length === 0) return;
log(`TTS-Request: "${cleanText.slice(0, 60)}..." (${sentences.length} Saetze → ${chunks.length} Chunks, voice: ${voice || "default"}, lang: ${language || "de"})`);
try {
// Voice-Sample Pfad bestimmen
const voiceSample = voice ? path.join(VOICES_DIR, `${voice}.wav`) : null;
const hasCustomVoice = voiceSample && fs.existsSync(voiceSample);
// XTTS API aufrufen
const audioBuffer = await callXTTSAPI(text, language || "de", hasCustomVoice ? voiceSample : null);
// Streaming: Chunk rendern → sofort senden → naechster Chunk
// App spielt mit Preloading-Queue nahtlos ab
let sentCount = 0;
if (audioBuffer && audioBuffer.length > 100) {
const base64 = audioBuffer.toString("base64");
log(`TTS fertig: ${audioBuffer.length} bytes (${(audioBuffer.length / 1024).toFixed(0)}KB)`);
for (let i = 0; i < chunks.length; i++) {
const chunk = chunks[i];
try {
const audioBuffer = await callXTTSAPI(chunk, language || "de", hasCustomVoice ? voiceSample : null);
sendToRVS({
type: "xtts_response",
payload: {
requestId: requestId || "",
base64,
mimeType: "audio/wav",
voice: voice || "default",
engine: "xtts",
},
timestamp: Date.now(),
});
} else {
log("TTS: Leeres Audio erhalten");
sendToRVS({
type: "xtts_response",
payload: { requestId, error: "Leeres Audio" },
timestamp: Date.now(),
});
if (audioBuffer && audioBuffer.length > 100) {
log(`TTS [${i + 1}/${chunks.length}]: ${(audioBuffer.length / 1024).toFixed(0)}KB — "${chunk.slice(0, 50)}"`);
sendToRVS({
type: "xtts_response",
payload: {
requestId: `${requestId || ""}_${i}`,
base64: audioBuffer.toString("base64"),
mimeType: "audio/wav",
voice: voice || "default",
engine: "xtts",
part: i + 1,
totalParts: chunks.length,
},
timestamp: Date.now(),
});
sentCount++;
}
} catch (chunkErr) {
log(`TTS [${i + 1}/${chunks.length}] Fehler: ${chunkErr.message} — ueberspringe`);
}
}
log(`TTS komplett: ${sentCount}/${chunks.length} Chunks gestreamt`);
} catch (err) {
log(`TTS Fehler: ${err.message}`);
sendToRVS({
@@ -257,12 +301,12 @@ log(`RVS: ${RVS_HOST}:${RVS_PORT}`);
function waitForXTTS(callback, attempts) {
if (attempts <= 0) { log("XTTS API nicht erreichbar — starte trotzdem"); callback(); return; }
http.get(`${XTTS_API_URL}/docs`, (res) => {
log("XTTS API erreichbar");
log(`XTTS API erreichbar (HTTP ${res.statusCode})`);
callback();
}).on("error", () => {
log(`XTTS API noch nicht bereit — warte (${attempts} Versuche uebrig)...`);
setTimeout(() => waitForXTTS(callback, attempts - 1), 5000);
setTimeout(() => waitForXTTS(callback, attempts - 1), 10000); // 10s statt 5s (Model laden dauert)
});
}
waitForXTTS(() => connectRVS(), 24); // Max 2min warten
waitForXTTS(() => connectRVS(), 30); // Max 5min warten
+6 -4
View File
@@ -17,7 +17,7 @@ services:
# ─── XTTS v2 API Server (GPU) ─────────────────
xtts:
image: ghcr.io/daswer123/xtts-api-server:latest
image: daswer123/xtts-api-server:latest
container_name: aria-xtts
deploy:
resources:
@@ -27,9 +27,9 @@ services:
count: 1
capabilities: [gpu]
ports:
- "8000:8000"
- "8000:8020"
volumes:
- xtts-models:/root/.local/share/tts # Model-Cache (~2GB)
- xtts-models:/app/xtts_models # Model-Cache (~2GB)
- ./voices:/voices # Custom Voice Samples
environment:
- COQUI_TOS_AGREED=1
@@ -41,8 +41,10 @@ services:
container_name: aria-xtts-bridge
depends_on:
- xtts
volumes:
- ./voices:/voices # Shared mit XTTS-Server
environment:
- XTTS_API_URL=http://xtts:8000
- XTTS_API_URL=http://xtts:8020
- RVS_HOST=${RVS_HOST}
- RVS_PORT=${RVS_PORT:-443}
- RVS_TLS=${RVS_TLS:-true}