Compare commits
37 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 08da28f475 | |||
| 8c1014d281 | |||
| 271fc4edf6 | |||
| cd390a4115 | |||
| a65ed579d2 | |||
| 2ad1f57382 | |||
| 58e3cfd3e6 | |||
| 7de4ee8f5b | |||
| 213edac3a7 | |||
| acc13aef6b | |||
| 4bbc6f7787 | |||
| 20f2ea1829 | |||
| 2d23f0668b | |||
| d6030a06b7 | |||
| 0df76e2af6 | |||
| f80fe1df93 | |||
| cff421bc53 | |||
| bca925d385 | |||
| 9abde89805 | |||
| ea4f639fcb | |||
| 64cd5f7d52 | |||
| 843ebe1d8f | |||
| 764619f076 | |||
| e3a0cfb55a | |||
| 2929749314 | |||
| 51b9512f4e | |||
| ffcfa44eef | |||
| 6363da97b1 | |||
| 07ed2cdcf6 | |||
| 5ad68b7dfc | |||
| 8a6ee018ea | |||
| b42590ff95 | |||
| 056b579c47 | |||
| 576e612cd0 | |||
| c2faa06a15 | |||
| d3ed3556eb | |||
| d960d125c0 |
@@ -306,7 +306,8 @@ aria-core → Antwort → Gateway → Diagnostic → RVS → App
|
||||
### Features
|
||||
|
||||
- **STT**: faster-whisper (lokal, offline, 16kHz mono)
|
||||
- **TTS**: Piper (Ramona + Thorsten, offline)
|
||||
- **TTS**: Piper (Ramona + Thorsten, offline) oder XTTS v2 (remote, GPU, Voice Cloning)
|
||||
- **Markdown-Bereinigung**: Entfernt **fett**, *kursiv*, `code`, Links, Listen etc. vor TTS (natuerliche Sprache)
|
||||
- **Wake-Word**: openwakeword (lokales Mikrofon auf der VM)
|
||||
- **App-Audio**: Base64 Audio von App → FFmpeg → Whisper STT → Text an aria-core
|
||||
- **Modi**: Normal, Nicht stoeren, Fluestern, Hangar, Gaming
|
||||
@@ -340,10 +341,10 @@ Erreichbar unter `http://<VM-IP>:3001`. Teilt das Netzwerk mit aria-core.
|
||||
- **Chat-Test**: Nachrichten direkt an ARIA senden (Gateway oder via RVS), Vollbild-Modus
|
||||
- **"ARIA denkt..." Indikator**: Zeigt live was ARIA gerade tut (Denken, Tool, Schreiben)
|
||||
- **Abbrechen-Button**: Stoppt laufende Anfragen + doctor --fix
|
||||
- **Session-Verwaltung**: Sessions auflisten, wechseln, erstellen, loeschen
|
||||
- **Session-Verwaltung**: Sessions auflisten, wechseln, erstellen, loeschen, als Markdown exportieren (⬇ Button)
|
||||
- **Chat-History**: Wird beim Laden und Session-Wechsel angezeigt (read-only aus JSONL)
|
||||
- **TTS-Diagnose Tab**: Stimmen testen, Status pruefen, Fehler anzeigen
|
||||
- **Einstellungen**: TTS-Engine (Piper/XTTS), Stimmen, Speed, Highlight-Trigger, Betriebsmodi
|
||||
- **Einstellungen**: TTS-Engine (Piper/XTTS), Stimmen, Speed, Highlight-Trigger, Betriebsmodi, Whisper-Modell (tiny…large-v3, Hot-Reload)
|
||||
- **XTTS Voice Cloning**: Audio-Samples hochladen, eigene Stimme erstellen
|
||||
- **Claude Login**: Browser-Terminal zum Einloggen in den Proxy
|
||||
- **Core Terminal**: Shell in aria-core (openclaw CLI)
|
||||
@@ -367,15 +368,19 @@ API-Endpoint fuer andere Services: `GET http://localhost:3001/api/session`
|
||||
|
||||
- Text-Chat mit ARIA
|
||||
- **Sprachaufnahme**: Push-to-Talk (halten) oder Tap-to-Talk (tippen, Auto-Stop bei Stille)
|
||||
- **Gespraechsmodus** (Ohr-Button): Nach jeder ARIA-Antwort startet automatisch die Aufnahme — wie ein natuerliches Gespraech hin und her, ohne Buttons druecken
|
||||
- **VAD (Voice Activity Detection)**: Erkennt 1.8s Stille und stoppt automatisch
|
||||
- **STT (Speech-to-Text)**: Audio wird in der Bridge per Whisper transkribiert, transkribierter Text erscheint im Chat
|
||||
- **TTS-Wiedergabe**: ARIA antwortet per Lautsprecher (Piper oder XTTS v2)
|
||||
- **Speech Gate**: Aufnahme wird verworfen wenn keine Sprache erkannt (kein Rauschen an Whisper)
|
||||
- **STT (Speech-to-Text)**: Audio wird als 16kHz mono aufgenommen und in der Bridge per Whisper transkribiert, transkribierter Text erscheint im Chat
|
||||
- **"ARIA denkt..." Indicator**: Zeigt live den Status vom Core (Denken, Tool, Schreiben) + Abbrechen-Button
|
||||
- **TTS-Wiedergabe**: ARIA antwortet per Lautsprecher (Piper oder XTTS v2), Audio-Queue mit Preloading
|
||||
- **Play-Button**: Jede ARIA-Nachricht kann nochmal vorgelesen werden
|
||||
- **Chat-Suche**: Lupe in der Statusleiste filtert Nachrichten live
|
||||
- **Datei- und Bild-Upload**: Bilder inline im Chat (Vollbild-Tap), Dateien mit Icon + Name + Groesse
|
||||
- **Mehrere Anhaenge**: Bilder + Dateien sammeln, Text hinzufuegen, dann zusammen senden
|
||||
- **Paste-Support**: Bilder aus Zwischenablage einfuegen (Diagnostic)
|
||||
- **Anhaenge**: Bridge speichert in Shared Volume, ARIA kann darauf zugreifen, Re-Download ueber RVS
|
||||
- **Einstellungen**: TTS Engine, Stimmen, Speed pro Stimme, Speicherort, Auto-Download, GPS
|
||||
- **Auto-Update**: Prueft beim Start auf neue Version, Download + Installation ueber RVS
|
||||
- **Auto-Update**: Prueft beim Start + per Button auf neue Version, Download + Installation ueber RVS (FileProvider)
|
||||
- GPS-Position (optional)
|
||||
- QR-Code Scanner fuer Token-Pairing
|
||||
|
||||
@@ -421,6 +426,17 @@ GITEA_USER=stefan
|
||||
RVS_UPDATE_HOST=root@aria-rvs # Optional: fuer Auto-Update
|
||||
```
|
||||
|
||||
### Docker-Cleanup
|
||||
|
||||
Das Bridge-Image zieht grosse ML-Deps (faster-whisper, ctranslate2, onnxruntime,
|
||||
openwakeword, piper-tts) — bei jedem Rebuild waechst der Docker-Build-Cache. Wenn
|
||||
die VM voll laeuft:
|
||||
|
||||
```bash
|
||||
./cleanup.sh # sicher: Build-Cache + ungenutzte Images
|
||||
./cleanup.sh --full # aggressiv: zusaetzlich ungenutzte Volumes (mit Rueckfrage)
|
||||
```
|
||||
|
||||
### Auto-Update
|
||||
|
||||
Die App prueft beim Start ob eine neuere Version auf dem RVS liegt.
|
||||
@@ -709,6 +725,17 @@ docker exec aria-core ssh aria-wohnung hostname
|
||||
- [x] Auto-Update System (APK via RVS)
|
||||
- [x] Chat-Suche, Play-Button, Abbrechen-Button
|
||||
- [x] XTTS v2 Integration (GPU, Voice Cloning, remote ueber RVS)
|
||||
- [x] Gespraechsmodus (Ohr-Button, automatische Aufnahme nach ARIA-Antwort)
|
||||
- [x] Mehrere Anhaenge + Text vor dem Senden + Paste-Support
|
||||
- [x] Markdown-Bereinigung fuer TTS
|
||||
- [x] Auto-Update mit FileProvider + Update-Check Button
|
||||
- [x] Inverted FlatList (zuverlaessiges Scroll-to-Bottom)
|
||||
- [x] Speech Gate (VAD verwirft Aufnahme ohne erkannte Sprache)
|
||||
- [x] Session-Persistenz ueber Container-Restarts (sessionFromFile + atomic write)
|
||||
- [x] Session-Export als Markdown-Datei (Download-Button pro Session)
|
||||
- [x] "ARIA denkt..."-Indicator + Abbrechen-Button in App (via Bridge → RVS)
|
||||
- [x] Whisper-Modell waehlbar in Diagnostic (tiny…large-v3, Hot-Reload)
|
||||
- [x] App-Aufnahme explizit 16kHz mono (optimal fuer Whisper, kein Resample)
|
||||
|
||||
### Phase 2 — ARIA wird produktiv
|
||||
|
||||
|
||||
@@ -79,8 +79,8 @@ android {
|
||||
applicationId "com.ariacockpit"
|
||||
minSdkVersion rootProject.ext.minSdkVersion
|
||||
targetSdkVersion rootProject.ext.targetSdkVersion
|
||||
versionCode 209
|
||||
versionName "0.0.2.9"
|
||||
versionCode 309
|
||||
versionName "0.0.3.9"
|
||||
// Fallback fuer Libraries mit Product Flavors
|
||||
missingDimensionStrategy 'react-native-camera', 'general'
|
||||
}
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "aria-cockpit",
|
||||
"version": "0.0.2.9",
|
||||
"version": "0.0.3.9",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"android": "react-native run-android",
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
* Datei- und Kamera-Upload.
|
||||
*/
|
||||
|
||||
import React, { useState, useEffect, useRef, useCallback } from 'react';
|
||||
import React, { useState, useEffect, useRef, useCallback, useMemo } from 'react';
|
||||
import {
|
||||
View,
|
||||
Text,
|
||||
@@ -16,6 +16,7 @@ import {
|
||||
Platform,
|
||||
StyleSheet,
|
||||
Image,
|
||||
ScrollView,
|
||||
Modal,
|
||||
} from 'react-native';
|
||||
import AsyncStorage from '@react-native-async-storage/async-storage';
|
||||
@@ -94,6 +95,8 @@ const ChatScreen: React.FC = () => {
|
||||
const [fullscreenImage, setFullscreenImage] = useState<string | null>(null);
|
||||
const [searchQuery, setSearchQuery] = useState('');
|
||||
const [searchVisible, setSearchVisible] = useState(false);
|
||||
const [pendingAttachments, setPendingAttachments] = useState<{file: any, isPhoto: boolean}[]>([]);
|
||||
const [agentActivity, setAgentActivity] = useState<{activity: string, tool: string}>({activity: 'idle', tool: ''});
|
||||
|
||||
const flatListRef = useRef<FlatList>(null);
|
||||
const messageIdCounter = useRef(0);
|
||||
@@ -248,6 +251,13 @@ const ChatScreen: React.FC = () => {
|
||||
if (message.type === 'audio' && message.payload.base64) {
|
||||
audioService.playAudio(message.payload.base64 as string);
|
||||
}
|
||||
|
||||
// Thinking-Indicator Status von der Bridge
|
||||
if (message.type === 'agent_activity') {
|
||||
const activity = (message.payload.activity as string) || 'idle';
|
||||
const tool = (message.payload.tool as string) || '';
|
||||
setAgentActivity({ activity, tool });
|
||||
}
|
||||
});
|
||||
|
||||
const unsubState = rvs.onStateChange((state) => {
|
||||
@@ -273,12 +283,20 @@ const ChatScreen: React.FC = () => {
|
||||
return () => { unsubUpdate(); clearTimeout(timer); };
|
||||
}, []);
|
||||
|
||||
// Wake Word: "ARIA" Erkennung → Auto-Aufnahme starten
|
||||
// Gespraechsmodus: Nach TTS-Wiedergabe automatisch Aufnahme starten
|
||||
useEffect(() => {
|
||||
const unsubPlayback = audioService.onPlaybackFinished(() => {
|
||||
if (wakeWordService.isActive()) {
|
||||
wakeWordService.resume();
|
||||
}
|
||||
});
|
||||
return () => unsubPlayback();
|
||||
}, []);
|
||||
|
||||
// Wake Word / Gespraechsmodus: Auto-Aufnahme starten
|
||||
useEffect(() => {
|
||||
const unsubWake = wakeWordService.onWakeWord(async () => {
|
||||
console.log('[Chat] Wake Word erkannt — starte Auto-Aufnahme');
|
||||
// TTS stoppen damit ARIA sich nicht selbst hoert
|
||||
audioService.stopPlayback();
|
||||
console.log('[Chat] Gespraechsmodus — starte Auto-Aufnahme');
|
||||
// Aufnahme mit Auto-Stop (VAD) starten
|
||||
const started = await audioService.startRecording(true);
|
||||
if (!started) {
|
||||
@@ -359,22 +377,8 @@ const ChatScreen: React.FC = () => {
|
||||
return () => { if (saveTimer.current) clearTimeout(saveTimer.current); };
|
||||
}, [messages]);
|
||||
|
||||
// Auto-Scroll wird ueber onContentSizeChange der FlatList gesteuert
|
||||
const shouldAutoScroll = useRef(true);
|
||||
const handleContentSizeChange = useCallback(() => {
|
||||
if (shouldAutoScroll.current) {
|
||||
flatListRef.current?.scrollToEnd({ animated: false });
|
||||
}
|
||||
}, []);
|
||||
const handleScrollBeginDrag = useCallback(() => {
|
||||
shouldAutoScroll.current = false;
|
||||
}, []);
|
||||
const handleScrollEndDrag = useCallback((e: any) => {
|
||||
// Auto-Scroll wieder aktivieren wenn User ganz unten ist
|
||||
const { contentOffset, contentSize, layoutMeasurement } = e.nativeEvent;
|
||||
const isAtBottom = contentOffset.y + layoutMeasurement.height >= contentSize.height - 50;
|
||||
shouldAutoScroll.current = isAtBottom;
|
||||
}, []);
|
||||
// Inverted FlatList: neueste Nachrichten unten, kein manuelles Scrollen noetig
|
||||
const invertedMessages = useMemo(() => [...messages].reverse(), [messages]);
|
||||
|
||||
// GPS-Position holen (optional)
|
||||
const getCurrentLocation = useCallback((): Promise<{ lat: number; lon: number } | null> => {
|
||||
@@ -400,6 +404,13 @@ const ChatScreen: React.FC = () => {
|
||||
|
||||
const sendTextMessage = useCallback(async () => {
|
||||
const text = inputText.trim();
|
||||
|
||||
// Wenn pending Anhaenge vorhanden → Anhaenge + Text zusammen senden
|
||||
if (pendingAttachments.length > 0) {
|
||||
sendPendingAttachments(text);
|
||||
return;
|
||||
}
|
||||
|
||||
if (!text) return;
|
||||
|
||||
setInputText('');
|
||||
@@ -419,7 +430,13 @@ const ChatScreen: React.FC = () => {
|
||||
text,
|
||||
...(location && { location }),
|
||||
});
|
||||
}, [inputText, getCurrentLocation]);
|
||||
}, [inputText, getCurrentLocation, pendingAttachments, sendPendingAttachments]);
|
||||
|
||||
// Anfrage abbrechen — sofort lokalen Indicator weg, Bridge triggert doctor --fix
|
||||
const cancelRequest = useCallback(() => {
|
||||
setAgentActivity({ activity: 'idle', tool: '' });
|
||||
rvs.send('cancel_request' as any, {});
|
||||
}, []);
|
||||
|
||||
// Sprachaufnahme abgeschlossen
|
||||
const handleVoiceRecording = useCallback(async (result: RecordingResult) => {
|
||||
@@ -441,88 +458,91 @@ const ChatScreen: React.FC = () => {
|
||||
});
|
||||
}, [getCurrentLocation]);
|
||||
|
||||
// Datei senden
|
||||
// Datei auswaehlen → zur Pending-Liste hinzufuegen
|
||||
const handleFileSelected = useCallback(async (file: FileData) => {
|
||||
setShowFileUpload(false);
|
||||
const location = await getCurrentLocation();
|
||||
setPendingAttachments(prev => [...prev, { file, isPhoto: false }]);
|
||||
}, []);
|
||||
|
||||
const isImage = file.type.startsWith('image/');
|
||||
const msgId = nextId();
|
||||
let imageUri = isImage && file.base64 ? `data:${file.type};base64,${file.base64}` : file.uri;
|
||||
|
||||
const userMsg: ChatMessage = {
|
||||
id: msgId,
|
||||
sender: 'user',
|
||||
text: 'Anhang empfangen',
|
||||
timestamp: Date.now(),
|
||||
attachments: [{
|
||||
type: isImage ? 'image' : 'file',
|
||||
name: file.name,
|
||||
size: file.size,
|
||||
uri: imageUri,
|
||||
mimeType: file.type,
|
||||
}],
|
||||
};
|
||||
setMessages(prev => [...prev, userMsg]);
|
||||
|
||||
// Anhang auf Disk speichern fuer Persistenz
|
||||
if (file.base64) {
|
||||
persistAttachment(file.base64, msgId, file.name).then(filePath => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === msgId ? { ...m, attachments: m.attachments?.map(a => ({ ...a, uri: filePath })) } : m
|
||||
));
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
rvs.send('file', {
|
||||
name: file.name,
|
||||
type: file.type,
|
||||
size: file.size,
|
||||
base64: file.base64,
|
||||
...(location && { location }),
|
||||
});
|
||||
}, [getCurrentLocation]);
|
||||
|
||||
// Foto senden
|
||||
// Foto auswaehlen → zur Pending-Liste hinzufuegen
|
||||
const handlePhotoSelected = useCallback(async (photo: PhotoData) => {
|
||||
setShowCameraUpload(false);
|
||||
setPendingAttachments(prev => [...prev, { file: photo, isPhoto: true }]);
|
||||
}, []);
|
||||
|
||||
// Alle Pending Anhaenge + Text senden
|
||||
const sendPendingAttachments = useCallback(async (messageText: string) => {
|
||||
if (pendingAttachments.length === 0) return;
|
||||
const location = await getCurrentLocation();
|
||||
|
||||
const msgId = nextId();
|
||||
const dataUri = photo.base64 ? `data:${photo.type};base64,${photo.base64}` : undefined;
|
||||
|
||||
// Alle Attachments fuer die Chat-Nachricht sammeln
|
||||
const attachments: Attachment[] = [];
|
||||
for (const { file, isPhoto } of pendingAttachments) {
|
||||
const isImage = isPhoto || (file.type && file.type.startsWith('image/'));
|
||||
const name = isPhoto ? file.fileName : file.name;
|
||||
const base64 = file.base64 || '';
|
||||
const mimeType = file.type || '';
|
||||
const imageUri = isImage && base64 ? `data:${mimeType};base64,${base64}` : file.uri;
|
||||
|
||||
attachments.push({
|
||||
type: isImage ? 'image' : 'file',
|
||||
name,
|
||||
size: file.size,
|
||||
uri: imageUri,
|
||||
mimeType,
|
||||
});
|
||||
}
|
||||
|
||||
// Chat-Nachricht mit allen Anhaengen
|
||||
const userMsg: ChatMessage = {
|
||||
id: msgId,
|
||||
sender: 'user',
|
||||
text: 'Anhang empfangen',
|
||||
text: messageText || `${pendingAttachments.length} Anhang/Anhaenge`,
|
||||
timestamp: Date.now(),
|
||||
attachments: [{
|
||||
type: 'image',
|
||||
name: photo.fileName,
|
||||
uri: dataUri,
|
||||
mimeType: photo.type,
|
||||
}],
|
||||
attachments,
|
||||
};
|
||||
setMessages(prev => [...prev, userMsg]);
|
||||
|
||||
// Foto auf Disk speichern fuer Persistenz
|
||||
if (photo.base64) {
|
||||
persistAttachment(photo.base64, msgId, photo.fileName).then(filePath => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === msgId ? { ...m, attachments: m.attachments?.map(a => ({ ...a, uri: filePath })) } : m
|
||||
));
|
||||
}).catch(() => {});
|
||||
// Alle Dateien an RVS senden + auf Disk speichern
|
||||
for (const { file, isPhoto } of pendingAttachments) {
|
||||
const name = isPhoto ? file.fileName : file.name;
|
||||
const base64 = file.base64 || '';
|
||||
const mimeType = file.type || '';
|
||||
|
||||
// Auf Disk speichern
|
||||
if (base64) {
|
||||
persistAttachment(base64, msgId + '_' + name, name).then(filePath => {
|
||||
setMessages(prev => prev.map(m =>
|
||||
m.id === msgId ? { ...m, attachments: m.attachments?.map(a =>
|
||||
a.name === name && !a.uri?.startsWith('file://') ? { ...a, uri: filePath } : a
|
||||
)} : m
|
||||
));
|
||||
}).catch(() => {});
|
||||
}
|
||||
|
||||
// An RVS senden
|
||||
rvs.send('file', {
|
||||
name,
|
||||
type: mimeType,
|
||||
size: file.size,
|
||||
base64,
|
||||
...(isPhoto && file.width && { width: file.width, height: file.height }),
|
||||
...(location && { location }),
|
||||
});
|
||||
}
|
||||
|
||||
rvs.send('file', {
|
||||
name: photo.fileName,
|
||||
type: photo.type,
|
||||
base64: photo.base64,
|
||||
width: photo.width,
|
||||
height: photo.height,
|
||||
...(location && { location }),
|
||||
});
|
||||
}, [getCurrentLocation]);
|
||||
// Text als separate Nachricht (damit ARIA weiss was zu tun ist)
|
||||
if (messageText) {
|
||||
rvs.send('chat', {
|
||||
text: messageText,
|
||||
...(location && { location }),
|
||||
});
|
||||
}
|
||||
|
||||
setPendingAttachments([]);
|
||||
setInputText('');
|
||||
}, [pendingAttachments, getCurrentLocation]);
|
||||
|
||||
// --- Rendering ---
|
||||
|
||||
@@ -653,14 +673,12 @@ const ChatScreen: React.FC = () => {
|
||||
{/* Nachrichtenliste */}
|
||||
<FlatList
|
||||
ref={flatListRef}
|
||||
data={searchQuery ? messages.filter(m => m.text.toLowerCase().includes(searchQuery.toLowerCase())) : messages}
|
||||
inverted
|
||||
data={searchQuery ? messages.filter(m => m.text.toLowerCase().includes(searchQuery.toLowerCase())).reverse() : invertedMessages}
|
||||
keyExtractor={item => item.id}
|
||||
renderItem={renderMessage}
|
||||
contentContainerStyle={styles.messageList}
|
||||
showsVerticalScrollIndicator={false}
|
||||
onContentSizeChange={handleContentSizeChange}
|
||||
onScrollBeginDrag={handleScrollBeginDrag}
|
||||
onScrollEndDrag={handleScrollEndDrag}
|
||||
ListEmptyComponent={
|
||||
<View style={styles.emptyContainer}>
|
||||
<Text style={styles.emptyIcon}>{'\uD83E\uDD16'}</Text>
|
||||
@@ -670,6 +688,56 @@ const ChatScreen: React.FC = () => {
|
||||
}
|
||||
/>
|
||||
|
||||
{/* Thinking-Indicator */}
|
||||
{agentActivity.activity !== 'idle' && (
|
||||
<View style={styles.thinkingBar}>
|
||||
<Text style={styles.thinkingText}>
|
||||
{agentActivity.activity === 'tool' && agentActivity.tool
|
||||
? `\uD83D\uDD27 ${agentActivity.tool}`
|
||||
: agentActivity.activity === 'assistant'
|
||||
? '\u270D\uFE0F ARIA schreibt...'
|
||||
: '\uD83D\uDCAD ARIA denkt...'}
|
||||
</Text>
|
||||
<TouchableOpacity style={styles.thinkingCancel} onPress={cancelRequest}>
|
||||
<Text style={styles.thinkingCancelText}>Abbrechen</Text>
|
||||
</TouchableOpacity>
|
||||
</View>
|
||||
)}
|
||||
|
||||
{/* Pending Anhaenge Vorschau */}
|
||||
{pendingAttachments.length > 0 && (
|
||||
<View style={styles.pendingBar}>
|
||||
<ScrollView horizontal showsHorizontalScrollIndicator={false} style={{flex: 1}}>
|
||||
{pendingAttachments.map((att, idx) => (
|
||||
<View key={idx} style={styles.pendingItem}>
|
||||
{att.file.type?.startsWith('image/') || att.isPhoto ? (
|
||||
<Image
|
||||
source={{ uri: att.file.base64
|
||||
? `data:${att.file.type};base64,${att.file.base64}`
|
||||
: att.file.uri }}
|
||||
style={styles.pendingThumb}
|
||||
/>
|
||||
) : (
|
||||
<View style={[styles.pendingThumb, {justifyContent: 'center', alignItems: 'center'}]}>
|
||||
<Text style={{fontSize: 20}}>{'\uD83D\uDCC4'}</Text>
|
||||
</View>
|
||||
)}
|
||||
<TouchableOpacity
|
||||
style={styles.pendingRemove}
|
||||
onPress={() => setPendingAttachments(prev => prev.filter((_, i) => i !== idx))}
|
||||
>
|
||||
<Text style={{color: '#fff', fontSize: 10, fontWeight: 'bold'}}>X</Text>
|
||||
</TouchableOpacity>
|
||||
</View>
|
||||
))}
|
||||
</ScrollView>
|
||||
<Text style={{color: '#8888AA', fontSize: 11, marginLeft: 8}}>{pendingAttachments.length}</Text>
|
||||
<TouchableOpacity onPress={() => setPendingAttachments([])}>
|
||||
<Text style={{color: '#FF3B30', fontSize: 14, paddingHorizontal: 8}}>Alle X</Text>
|
||||
</TouchableOpacity>
|
||||
</View>
|
||||
)}
|
||||
|
||||
{/* Eingabebereich */}
|
||||
<View style={styles.inputContainer}>
|
||||
{/* Datei-Buttons */}
|
||||
@@ -692,7 +760,7 @@ const ChatScreen: React.FC = () => {
|
||||
style={styles.textInput}
|
||||
value={inputText}
|
||||
onChangeText={setInputText}
|
||||
placeholder="Nachricht an ARIA..."
|
||||
placeholder={pendingAttachments.length > 0 ? "Text zu den Anhaengen (optional)..." : "Nachricht an ARIA..."}
|
||||
placeholderTextColor="#555570"
|
||||
multiline
|
||||
maxLength={4000}
|
||||
@@ -701,7 +769,7 @@ const ChatScreen: React.FC = () => {
|
||||
/>
|
||||
|
||||
{/* Senden oder Sprache */}
|
||||
{inputText.trim() ? (
|
||||
{inputText.trim() || pendingAttachments.length > 0 ? (
|
||||
<TouchableOpacity style={styles.sendButton} onPress={sendTextMessage}>
|
||||
<Text style={styles.sendIcon}>{'\u2B06\uFE0F'}</Text>
|
||||
</TouchableOpacity>
|
||||
@@ -932,6 +1000,63 @@ const styles = StyleSheet.create({
|
||||
wakeWordIcon: {
|
||||
fontSize: 16,
|
||||
},
|
||||
thinkingBar: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
justifyContent: 'space-between',
|
||||
backgroundColor: '#1E1E2E',
|
||||
paddingHorizontal: 12,
|
||||
paddingVertical: 6,
|
||||
borderTopWidth: 1,
|
||||
borderTopColor: '#2A2A3E',
|
||||
},
|
||||
thinkingText: {
|
||||
color: '#FFD60A',
|
||||
fontSize: 12,
|
||||
flex: 1,
|
||||
},
|
||||
thinkingCancel: {
|
||||
paddingHorizontal: 10,
|
||||
paddingVertical: 4,
|
||||
borderWidth: 1,
|
||||
borderColor: '#FF3B30',
|
||||
borderRadius: 4,
|
||||
},
|
||||
thinkingCancelText: {
|
||||
color: '#FF3B30',
|
||||
fontSize: 11,
|
||||
fontWeight: 'bold',
|
||||
},
|
||||
pendingBar: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
backgroundColor: '#1E1E2E',
|
||||
paddingHorizontal: 12,
|
||||
paddingVertical: 8,
|
||||
borderTopWidth: 1,
|
||||
borderTopColor: '#2A2A3E',
|
||||
},
|
||||
pendingItem: {
|
||||
position: 'relative',
|
||||
marginRight: 8,
|
||||
},
|
||||
pendingThumb: {
|
||||
width: 50,
|
||||
height: 50,
|
||||
borderRadius: 6,
|
||||
backgroundColor: '#0D0D1A',
|
||||
},
|
||||
pendingRemove: {
|
||||
position: 'absolute',
|
||||
top: -4,
|
||||
right: -4,
|
||||
width: 18,
|
||||
height: 18,
|
||||
borderRadius: 9,
|
||||
backgroundColor: '#FF3B30',
|
||||
justifyContent: 'center',
|
||||
alignItems: 'center',
|
||||
},
|
||||
searchBar: {
|
||||
flexDirection: 'row',
|
||||
alignItems: 'center',
|
||||
|
||||
@@ -42,6 +42,8 @@ const AUDIO_ENCODING = 'audio/wav';
|
||||
// VAD (Voice Activity Detection) — Stille-Erkennung
|
||||
const VAD_SILENCE_THRESHOLD_DB = -45; // dB unter dem als "Stille" gilt
|
||||
const VAD_SILENCE_DURATION_MS = 1800; // ms Stille bevor Auto-Stop
|
||||
const VAD_SPEECH_THRESHOLD_DB = -35; // dB ueber dem als "Sprache" gilt (Sprach-Gate)
|
||||
const VAD_SPEECH_MIN_MS = 300; // ms Sprache bevor Aufnahme zaehlt
|
||||
|
||||
// --- Audio-Service ---
|
||||
|
||||
@@ -61,6 +63,10 @@ class AudioService {
|
||||
private preloadedSound: Sound | null = null;
|
||||
private preloadedPath: string = '';
|
||||
|
||||
// Sprach-Gate: Aufnahme erst senden wenn tatsaechlich gesprochen wurde
|
||||
private speechDetected: boolean = false;
|
||||
private speechStartTime: number = 0;
|
||||
|
||||
// VAD State
|
||||
private vadEnabled: boolean = false;
|
||||
private lastSpeechTime: number = 0;
|
||||
@@ -121,6 +127,8 @@ class AudioService {
|
||||
AudioEncoderAndroid: AudioEncoderAndroidType.AAC,
|
||||
AudioSourceAndroid: AudioSourceAndroidType.MIC,
|
||||
OutputFormatAndroid: OutputFormatAndroidType.MPEG_4,
|
||||
AudioSamplingRateAndroid: 16000,
|
||||
AudioChannelsAndroid: 1,
|
||||
}, true); // meteringEnabled = true
|
||||
|
||||
// Metering-Callback
|
||||
@@ -128,7 +136,21 @@ class AudioService {
|
||||
const db = e.currentMetering ?? -160;
|
||||
this.meterListeners.forEach(cb => cb(db));
|
||||
|
||||
// VAD: Stille erkennen
|
||||
// Sprach-Gate: Erkennen ob tatsaechlich gesprochen wird
|
||||
if (db > VAD_SPEECH_THRESHOLD_DB) {
|
||||
if (!this.speechDetected && this.speechStartTime === 0) {
|
||||
this.speechStartTime = Date.now();
|
||||
}
|
||||
if (this.speechStartTime > 0 && Date.now() - this.speechStartTime >= VAD_SPEECH_MIN_MS) {
|
||||
this.speechDetected = true;
|
||||
}
|
||||
} else {
|
||||
if (!this.speechDetected) {
|
||||
this.speechStartTime = 0; // Reset wenn noch nicht als Sprache erkannt
|
||||
}
|
||||
}
|
||||
|
||||
// VAD: Stille erkennen (nur wenn Sprache erkannt wurde)
|
||||
if (this.vadEnabled) {
|
||||
if (db > VAD_SILENCE_THRESHOLD_DB) {
|
||||
this.lastSpeechTime = Date.now();
|
||||
@@ -138,6 +160,8 @@ class AudioService {
|
||||
|
||||
this.recordingStartTime = Date.now();
|
||||
this.lastSpeechTime = Date.now();
|
||||
this.speechDetected = false;
|
||||
this.speechStartTime = 0;
|
||||
this.setState('recording');
|
||||
|
||||
// VAD aktivieren
|
||||
@@ -180,6 +204,15 @@ class AudioService {
|
||||
this.recorder.removeRecordBackListener();
|
||||
|
||||
const durationMs = Date.now() - this.recordingStartTime;
|
||||
const hadSpeech = this.speechDetected;
|
||||
|
||||
// Sprach-Gate: Wenn keine Sprache erkannt → Aufnahme verwerfen
|
||||
if (!hadSpeech) {
|
||||
RNFS.unlink(this.recordingPath).catch(() => {});
|
||||
this.setState('idle');
|
||||
console.log('[Audio] Aufnahme verworfen — keine Sprache erkannt (nur Umgebungsgeraeusche)');
|
||||
return null;
|
||||
}
|
||||
|
||||
// Audio-Datei als Base64 lesen
|
||||
const base64Data = await RNFS.readFile(this.recordingPath, 'base64');
|
||||
@@ -188,7 +221,7 @@ class AudioService {
|
||||
RNFS.unlink(this.recordingPath).catch(() => {});
|
||||
|
||||
this.setState('idle');
|
||||
console.log(`[Audio] Aufnahme beendet (${durationMs}ms, ${Math.round(base64Data.length / 1024)}KB)`);
|
||||
console.log(`[Audio] Aufnahme beendet (${durationMs}ms, ${Math.round(base64Data.length / 1024)}KB, Sprache erkannt)`);
|
||||
|
||||
return {
|
||||
base64: base64Data,
|
||||
@@ -214,10 +247,22 @@ class AudioService {
|
||||
}
|
||||
}
|
||||
|
||||
// Callback wenn alle Audio-Teile abgespielt sind
|
||||
private playbackFinishedListeners: (() => void)[] = [];
|
||||
|
||||
onPlaybackFinished(callback: () => void): () => void {
|
||||
this.playbackFinishedListeners.push(callback);
|
||||
return () => {
|
||||
this.playbackFinishedListeners = this.playbackFinishedListeners.filter(cb => cb !== callback);
|
||||
};
|
||||
}
|
||||
|
||||
/** Naechstes Audio aus der Queue abspielen */
|
||||
private async _playNext(): Promise<void> {
|
||||
if (this.audioQueue.length === 0) {
|
||||
this.isPlaying = false;
|
||||
// Alle Audio-Teile abgespielt → Listener benachrichtigen
|
||||
this.playbackFinishedListeners.forEach(cb => cb());
|
||||
return;
|
||||
}
|
||||
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
/**
|
||||
* Wake Word Service — "ARIA" Erkennung
|
||||
* Gespraechsmodus — "Ohr-Button"
|
||||
*
|
||||
* Phase 1: Deaktiviert — react-native-live-audio-stream hat native Bridge-Probleme.
|
||||
* Nutzt stattdessen Tap-to-Talk (VoiceButton) als primaeren Eingabemodus.
|
||||
* Wenn aktiv: Nach jeder ARIA-Antwort (TTS fertig) startet automatisch die Aufnahme.
|
||||
* Wie ein Walkie-Talkie / natuerliches Gespraech:
|
||||
* ARIA spricht → Aufnahme startet → User spricht → VAD stoppt → ARIA antwortet → ...
|
||||
*
|
||||
* Phase 2: Porcupine on-device "ARIA" Keyword (geplant).
|
||||
* Phase 2 (geplant): Porcupine "ARIA" Wake Word fuer passives Lauschen.
|
||||
*/
|
||||
|
||||
type WakeWordCallback = () => void;
|
||||
@@ -17,30 +18,39 @@ class WakeWordService {
|
||||
private wakeCallbacks: WakeWordCallback[] = [];
|
||||
private stateCallbacks: StateCallback[] = [];
|
||||
|
||||
/** Wake Word Erkennung starten */
|
||||
/** Gespraechsmodus starten */
|
||||
async start(): Promise<boolean> {
|
||||
if (this.state === 'listening') return true;
|
||||
|
||||
try {
|
||||
// Phase 1: LiveAudioStream deaktiviert (native Bridge instabil)
|
||||
// Stattdessen: Tap-to-Talk als primaerer Modus
|
||||
console.log('[WakeWord] Wake Word ist in Phase 1 noch nicht verfuegbar — nutze Tap-to-Talk');
|
||||
this.setState('listening');
|
||||
return true;
|
||||
} catch (err) {
|
||||
console.error('[WakeWord] Start fehlgeschlagen:', err);
|
||||
return false;
|
||||
}
|
||||
console.log('[WakeWord] Gespraechsmodus aktiviert — starte sofort Aufnahme');
|
||||
this.setState('listening');
|
||||
// Sofort erste Aufnahme starten
|
||||
setTimeout(() => {
|
||||
if (this.state === 'listening') {
|
||||
this.wakeCallbacks.forEach(cb => cb());
|
||||
}
|
||||
}, 500);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Wake Word Erkennung stoppen */
|
||||
/** Gespraechsmodus stoppen */
|
||||
stop(): void {
|
||||
console.log('[WakeWord] Gespraechsmodus deaktiviert');
|
||||
this.setState('off');
|
||||
}
|
||||
|
||||
/** Nach Aufnahme erneut starten */
|
||||
/** Nach ARIA-Antwort (TTS fertig): Aufnahme automatisch starten */
|
||||
async resume(): Promise<void> {
|
||||
// Nichts zu tun in Phase 1
|
||||
if (this.state !== 'listening') return;
|
||||
// Kurze Pause damit TTS-Audio nicht ins Mikrofon geht
|
||||
await new Promise(resolve => setTimeout(resolve, 800));
|
||||
if (this.state === 'listening') {
|
||||
console.log('[WakeWord] TTS fertig — starte automatisch Aufnahme');
|
||||
this.wakeCallbacks.forEach(cb => cb());
|
||||
}
|
||||
}
|
||||
|
||||
isActive(): boolean {
|
||||
return this.state === 'listening';
|
||||
}
|
||||
|
||||
// --- Callbacks ---
|
||||
|
||||
@@ -9,3 +9,10 @@ PIPER_THORSTEN=/voices/de_DE-thorsten-high.onnx
|
||||
|
||||
# Wake-Word
|
||||
WAKE_WORD=aria
|
||||
|
||||
# Whisper STT — wird zur Laufzeit in der Diagnostic (Sektion "Whisper") umgeschaltet
|
||||
# und in /shared/config/voice_config.json gespeichert. Der Wert hier ist nur der
|
||||
# Initial-Default beim ersten Start.
|
||||
# Optionen: tiny | base | small | medium | large-v3
|
||||
WHISPER_MODEL=medium
|
||||
WHISPER_LANGUAGE=de
|
||||
|
||||
+119
-7
@@ -63,7 +63,7 @@ RVS_TLS = os.getenv("RVS_TLS", "true") # true = wss://, false = ws://
|
||||
RVS_TLS_FALLBACK = os.getenv("RVS_TLS_FALLBACK", "true") # Bei TLS-Fehler ws:// versuchen
|
||||
RVS_TOKEN = os.getenv("RVS_TOKEN", "") # Pairing-Token (gleich wie in der App)
|
||||
DIAGNOSTIC_URL = os.getenv("DIAGNOSTIC_URL", "http://127.0.0.1:3001") # Diagnostic API
|
||||
WHISPER_MODEL = os.getenv("WHISPER_MODEL", "small")
|
||||
WHISPER_MODEL = os.getenv("WHISPER_MODEL", "medium")
|
||||
WHISPER_LANGUAGE = os.getenv("WHISPER_LANGUAGE", "de")
|
||||
|
||||
# Audio-Parameter
|
||||
@@ -201,11 +201,23 @@ class VoiceEngine:
|
||||
return None
|
||||
|
||||
try:
|
||||
# Langen Text in Saetze aufteilen (Piper hat Limits bei langen Texten)
|
||||
# Markdown + Sonderzeichen entfernen fuer natuerliche Sprache
|
||||
import re
|
||||
sentences = re.split(r'(?<=[.!?])\s+', text.strip())
|
||||
# Markdown-Formatierung entfernen
|
||||
sentences = [re.sub(r'\*\*([^*]+)\*\*', r'\1', s).strip() for s in sentences if s.strip()]
|
||||
clean = text.strip()
|
||||
clean = re.sub(r'\*\*([^*]+)\*\*', r'\1', clean) # **fett**
|
||||
clean = re.sub(r'\*([^*]+)\*', r'\1', clean) # *kursiv*
|
||||
clean = re.sub(r'`[^`]+`', '', clean) # `code`
|
||||
clean = re.sub(r'```[\s\S]*?```', '', clean) # Code-Bloecke
|
||||
clean = re.sub(r'\[([^\]]+)\]\([^)]+\)', r'\1', clean) # [text](url)
|
||||
clean = re.sub(r'#{1,6}\s*', '', clean) # ### Ueberschriften
|
||||
clean = re.sub(r'>\s*', '', clean) # > Zitate
|
||||
clean = re.sub(r'[-*]\s+', '', clean) # Listen
|
||||
clean = re.sub(r'\n{2,}', '. ', clean) # Absaetze
|
||||
clean = re.sub(r'\n', ', ', clean) # Zeilenumbrueche
|
||||
clean = re.sub(r'\s{2,}', ' ', clean) # Mehrfach-Leerzeichen
|
||||
clean = re.sub(r'["""„]', '', clean) # Anfuehrungszeichen
|
||||
sentences = re.split(r'(?<=[.!?])\s+', clean)
|
||||
sentences = [s.strip() for s in sentences if s.strip()]
|
||||
|
||||
if not sentences:
|
||||
return None
|
||||
@@ -318,6 +330,25 @@ class STTEngine:
|
||||
self.model = WhisperModel(self.model_size, device="cpu", compute_type="int8")
|
||||
logger.info("Whisper-Modell geladen")
|
||||
|
||||
def reload(self, model_size: str) -> bool:
|
||||
"""Laedt ein anderes Whisper-Modell (bei Config-Aenderung)."""
|
||||
if model_size == self.model_size and self.model is not None:
|
||||
return False
|
||||
allowed = {"tiny", "base", "small", "medium", "large-v3"}
|
||||
if model_size not in allowed:
|
||||
logger.warning("Ungueltiges Whisper-Modell: %s (erlaubt: %s)", model_size, allowed)
|
||||
return False
|
||||
logger.info("Lade Whisper-Modell neu: %s -> %s", self.model_size, model_size)
|
||||
self.model_size = model_size
|
||||
self.model = None
|
||||
try:
|
||||
self.model = WhisperModel(model_size, device="cpu", compute_type="int8")
|
||||
logger.info("Whisper-Modell '%s' geladen", model_size)
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Whisper-Modell '%s' konnte nicht geladen werden", model_size)
|
||||
return False
|
||||
|
||||
def transcribe(self, audio_data: np.ndarray) -> str:
|
||||
"""Transkribiert Audio-Daten zu Text.
|
||||
|
||||
@@ -490,6 +521,7 @@ class ARIABridge:
|
||||
# Komponenten
|
||||
self.voice_engine = VoiceEngine(VOICES_DIR)
|
||||
self.tts_enabled = True
|
||||
vc: dict = {}
|
||||
# Gespeicherte Voice-Config laden
|
||||
try:
|
||||
vc_path = "/shared/config/voice_config.json"
|
||||
@@ -508,8 +540,10 @@ class ARIABridge:
|
||||
logger.info("Voice-Config geladen: %s", vc)
|
||||
except Exception as e:
|
||||
logger.warning("Voice-Config laden fehlgeschlagen: %s", e)
|
||||
# Whisper-Modell: Config hat Vorrang, dann env/Default (medium)
|
||||
whisper_model = vc.get("whisperModel") or self.config.get("WHISPER_MODEL", WHISPER_MODEL)
|
||||
self.stt_engine = STTEngine(
|
||||
model_size=self.config.get("WHISPER_MODEL", WHISPER_MODEL),
|
||||
model_size=whisper_model,
|
||||
language=self.config.get("WHISPER_LANGUAGE", WHISPER_LANGUAGE),
|
||||
)
|
||||
self.wake_word = WakeWordDetector()
|
||||
@@ -518,6 +552,12 @@ class ARIABridge:
|
||||
self.ws_core: Optional[websockets.WebSocketClientProtocol] = None
|
||||
self.ws_rvs: Optional[websockets.WebSocketClientProtocol] = None
|
||||
|
||||
# Letzter gesendeter agent_activity-State (zum Entduplizieren)
|
||||
self._last_activity_state: Optional[tuple] = None
|
||||
# Zeitstempel des letzten chat:final — waehrend 3s danach werden
|
||||
# trailing Agent-Events unterdrueckt (Core raeumt manchmal nach).
|
||||
self._last_chat_final_at: float = 0.0
|
||||
|
||||
def initialize(self) -> None:
|
||||
"""Initialisiert alle Komponenten.
|
||||
|
||||
@@ -722,8 +762,18 @@ class ARIABridge:
|
||||
if event_name == "agent":
|
||||
data = payload.get("data", {})
|
||||
delta = data.get("delta", "")
|
||||
if delta and payload.get("stream") == "assistant":
|
||||
stream = payload.get("stream", "")
|
||||
if delta and stream == "assistant":
|
||||
logger.debug("[core] Delta: '%s'", delta[:40])
|
||||
# Activity-Signal zur App (entdupliziert)
|
||||
tool_name = data.get("name") or data.get("tool") or payload.get("tool") or ""
|
||||
if stream == "tool_use" or data.get("type") == "tool_use":
|
||||
activity = "tool"
|
||||
elif stream == "assistant":
|
||||
activity = "assistant"
|
||||
else:
|
||||
activity = "thinking"
|
||||
await self._emit_activity(activity, tool_name)
|
||||
return
|
||||
|
||||
# ── chat Events: Snapshots mit state=delta|final|error ──
|
||||
@@ -732,6 +782,8 @@ class ARIABridge:
|
||||
|
||||
if state == "final":
|
||||
text = self._extract_chat_text(payload)
|
||||
self._last_chat_final_at = asyncio.get_event_loop().time()
|
||||
await self._emit_activity("idle", "")
|
||||
if not text:
|
||||
logger.warning("[core] chat final ohne Text: %s", json.dumps(payload)[:200])
|
||||
return
|
||||
@@ -742,6 +794,8 @@ class ARIABridge:
|
||||
if state == "error":
|
||||
error = payload.get("error", "Unbekannt")
|
||||
logger.error("[core] Chat-Fehler: %s", error)
|
||||
self._last_chat_final_at = asyncio.get_event_loop().time()
|
||||
await self._emit_activity("idle", "")
|
||||
await self._send_to_rvs({
|
||||
"type": "chat",
|
||||
"payload": {
|
||||
@@ -1045,6 +1099,17 @@ class ARIABridge:
|
||||
sender = payload.get("sender", "")
|
||||
if sender in ("aria", "stt"):
|
||||
return
|
||||
text = payload.get("text", "")
|
||||
if text:
|
||||
logger.info("[rvs] App-Chat: '%s'", text[:80])
|
||||
await self.send_to_core(text, source="app")
|
||||
return
|
||||
|
||||
if msg_type == "cancel_request":
|
||||
logger.info("[rvs] Cancel-Request von App — rufe Diagnostic /api/cancel auf")
|
||||
await self._cancel_via_diagnostic()
|
||||
await self._emit_activity("idle", "")
|
||||
return
|
||||
|
||||
elif msg_type == "xtts_response":
|
||||
# XTTS-Audio vom Gaming-PC empfangen → an App weiterleiten
|
||||
@@ -1125,6 +1190,15 @@ class ARIABridge:
|
||||
self.voice_engine.speech_speed["thorsten"] = max(0.3, min(2.0, float(payload["speedThorsten"])))
|
||||
logger.info("[rvs] Speed Thorsten: %.1f", self.voice_engine.speech_speed["thorsten"])
|
||||
changed = True
|
||||
whisper_reloaded = False
|
||||
if "whisperModel" in payload:
|
||||
new_model = payload["whisperModel"]
|
||||
if new_model and new_model != self.stt_engine.model_size:
|
||||
logger.info("[rvs] Whisper-Modell Wechsel: %s -> %s (laedt...)", self.stt_engine.model_size, new_model)
|
||||
loop = asyncio.get_event_loop()
|
||||
whisper_reloaded = await loop.run_in_executor(None, self.stt_engine.reload, new_model)
|
||||
if whisper_reloaded:
|
||||
changed = True
|
||||
# Persistent speichern in Shared Volume
|
||||
if changed:
|
||||
try:
|
||||
@@ -1137,6 +1211,7 @@ class ARIABridge:
|
||||
"xttsVoice": getattr(self, "xtts_voice", ""),
|
||||
"speedRamona": self.voice_engine.speech_speed.get("ramona", 1.0),
|
||||
"speedThorsten": self.voice_engine.speech_speed.get("thorsten", 1.0),
|
||||
"whisperModel": self.stt_engine.model_size,
|
||||
}
|
||||
with open("/shared/config/voice_config.json", "w") as f:
|
||||
json.dump(config_data, f, indent=2)
|
||||
@@ -1379,6 +1454,43 @@ class ARIABridge:
|
||||
|
||||
# ── Log-Streaming an die App ─────────────────────────────
|
||||
|
||||
async def _cancel_via_diagnostic(self) -> None:
|
||||
"""Ruft das Diagnostic /api/cancel an — dort laeuft die volle Abbruch-Logik
|
||||
(openclaw doctor --fix mit Docker-Socket)."""
|
||||
def _do_request():
|
||||
try:
|
||||
req = urllib.request.Request(
|
||||
f"{self._diagnostic_url}/api/cancel",
|
||||
method="POST",
|
||||
data=b"",
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=5) as resp:
|
||||
return resp.status
|
||||
except Exception as e:
|
||||
return f"error: {e}"
|
||||
|
||||
status = await asyncio.get_event_loop().run_in_executor(None, _do_request)
|
||||
logger.info("[cancel] Diagnostic /api/cancel: %s", status)
|
||||
|
||||
async def _emit_activity(self, activity: str, tool: str = "") -> None:
|
||||
"""Sendet agent_activity an die App — nur wenn sich der State geaendert hat.
|
||||
|
||||
Trailing Agent-Events nach chat:final werden 3s lang unterdrueckt
|
||||
(nur 'idle' kommt immer durch)."""
|
||||
if activity != "idle" and self._last_chat_final_at > 0:
|
||||
since_final = asyncio.get_event_loop().time() - self._last_chat_final_at
|
||||
if since_final < 3.0:
|
||||
return
|
||||
state = (activity, tool)
|
||||
if state == self._last_activity_state:
|
||||
return
|
||||
self._last_activity_state = state
|
||||
await self._send_to_rvs({
|
||||
"type": "agent_activity",
|
||||
"payload": {"activity": activity, "tool": tool},
|
||||
"timestamp": int(asyncio.get_event_loop().time() * 1000),
|
||||
})
|
||||
|
||||
async def send_log_to_app(self, source: str, message: str, level: str = "info") -> None:
|
||||
"""Sendet einen Log-Eintrag an die App (erscheint im Log-Viewer)."""
|
||||
await self._send_to_rvs({
|
||||
|
||||
Executable
+44
@@ -0,0 +1,44 @@
|
||||
#!/bin/bash
|
||||
# ARIA Docker Cleanup
|
||||
#
|
||||
# Standard: docker builder prune + image prune (sicher, loescht keine Volumes)
|
||||
# --full: Volle Reinigung inkl. --volumes (Vorsicht bei ungenutzten Volumes!)
|
||||
#
|
||||
# Usage:
|
||||
# ./cleanup.sh # sicherer Cleanup
|
||||
# ./cleanup.sh --full # aggressiver Cleanup (inkl. Volumes)
|
||||
|
||||
set -e
|
||||
|
||||
FULL=0
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--full|-f) FULL=1 ;;
|
||||
-h|--help)
|
||||
grep '^#' "$0" | sed 's/^# \{0,1\}//'
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
echo "── Docker Speicher VOR Cleanup ───────────────────"
|
||||
docker system df
|
||||
echo
|
||||
|
||||
if [ "$FULL" = "1" ]; then
|
||||
echo ">>> VOLLE Reinigung (inkl. ungenutzter Volumes)"
|
||||
read -p "Wirklich? [y/N] " -n 1 -r REPLY
|
||||
echo
|
||||
[[ ! $REPLY =~ ^[Yy]$ ]] && { echo "Abgebrochen."; exit 0; }
|
||||
docker system prune -a --volumes -f
|
||||
else
|
||||
echo ">>> Sicherer Cleanup (Build-Cache + ungenutzte Images)"
|
||||
docker builder prune -a -f
|
||||
docker image prune -a -f
|
||||
fi
|
||||
|
||||
echo
|
||||
echo "── Docker Speicher NACH Cleanup ──────────────────"
|
||||
docker system df
|
||||
echo
|
||||
df -h / | head -2
|
||||
+130
-9
@@ -205,8 +205,14 @@
|
||||
<span><span style="animation:pulse 1s infinite;">💭</span> <span id="thinking-text">ARIA denkt...</span></span>
|
||||
<button class="btn secondary" onclick="cancelRequest()" style="padding:2px 10px;font-size:11px;color:#FF3B30;border-color:#FF3B30;">Abbrechen</button>
|
||||
</div>
|
||||
<div id="diag-pending-attachments" style="display:none;padding:6px 10px;background:#1E1E2E;border-radius:6px 6px 0 0;margin-bottom:-4px;display:flex;gap:6px;flex-wrap:wrap;align-items:center;">
|
||||
</div>
|
||||
<div class="input-row">
|
||||
<input type="text" id="chat-input" placeholder="Nachricht an ARIA...">
|
||||
<label class="btn secondary" style="padding:6px 10px;cursor:pointer;font-size:14px;" title="Datei anhaengen">
|
||||
📎
|
||||
<input type="file" id="diag-file-input" multiple accept="image/*,application/pdf,.doc,.docx,.txt" style="display:none;" onchange="handleDiagFileSelect(this.files)">
|
||||
</label>
|
||||
<input type="text" id="chat-input" placeholder="Nachricht an ARIA..." onpaste="handleDiagPaste(event)">
|
||||
<button class="btn" id="btn-gw" onclick="testGateway()">Gateway senden</button>
|
||||
<button class="btn" id="btn-rvs" onclick="testRVS()">Via RVS senden</button>
|
||||
</div>
|
||||
@@ -493,6 +499,30 @@
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Whisper (STT) -->
|
||||
<div class="settings-section">
|
||||
<h2>Whisper (Spracherkennung)</h2>
|
||||
<div style="font-size:11px;color:#8888AA;margin-bottom:8px;">
|
||||
Aenderungen werden sofort an die Bridge gesendet und das Modell neu geladen
|
||||
(kann bei medium/large 10-30s dauern — waehrend dieser Zeit ist STT kurz pausiert).
|
||||
</div>
|
||||
<div class="card" style="max-width:500px;">
|
||||
<div style="display:flex;align-items:center;gap:12px;margin-bottom:8px;">
|
||||
<label style="color:#8888AA;font-size:12px;min-width:80px;">Modell:</label>
|
||||
<select id="diag-whisper-model" onchange="sendVoiceConfig()" style="flex:1;background:#1E1E2E;color:#fff;border:1px solid #2A2A3E;border-radius:6px;padding:6px 10px;font-size:13px;">
|
||||
<option value="tiny">tiny (39MB, schnell, niedrige Qualitaet)</option>
|
||||
<option value="base">base (74MB, schnell, ok)</option>
|
||||
<option value="small">small (244MB, mittel)</option>
|
||||
<option value="medium" selected>medium (769MB, gut — Empfehlung)</option>
|
||||
<option value="large-v3">large-v3 (1.5GB, beste Qualitaet, langsam auf CPU)</option>
|
||||
</select>
|
||||
</div>
|
||||
<div style="font-size:10px;color:#555570;">
|
||||
Tipp: <code>medium</code> ist der beste Kompromiss fuer CPU. <code>large-v3</code> nur bei GPU sinnvoll.
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<!-- Highlight-Trigger -->
|
||||
<div class="settings-section">
|
||||
<h2>Highlight-Trigger</h2>
|
||||
@@ -757,6 +787,11 @@
|
||||
}
|
||||
xttsSelect.value = xttsVoice;
|
||||
toggleXTTSPanel();
|
||||
// Whisper-Modell wiederherstellen (falls gesetzt)
|
||||
if (msg.whisperModel) {
|
||||
const wSel = document.getElementById('diag-whisper-model');
|
||||
if (wSel) wSel.value = msg.whisperModel;
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -885,6 +920,18 @@
|
||||
else alert('Loeschen fehlgeschlagen: ' + (msg.error || '?'));
|
||||
return;
|
||||
}
|
||||
if (msg.type === 'session_export') {
|
||||
if (!msg.ok) { alert('Export fehlgeschlagen: ' + (msg.error || '?')); return; }
|
||||
const blob = new Blob([msg.markdown], { type: 'text/markdown;charset=utf-8' });
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = msg.filename;
|
||||
document.body.appendChild(a);
|
||||
a.click();
|
||||
setTimeout(() => { URL.revokeObjectURL(url); a.remove(); }, 100);
|
||||
return;
|
||||
}
|
||||
if (msg.type === 'active_session') {
|
||||
updateActiveSessionBar(msg.sessionKey);
|
||||
loadSessions(); // Tabelle neu rendern
|
||||
@@ -939,21 +986,39 @@
|
||||
}
|
||||
}
|
||||
|
||||
function sendDiagAttachments() {
|
||||
// Alle pending Dateien an RVS senden
|
||||
for (const f of diagPendingFiles) {
|
||||
send({ action: 'send_file', name: f.name, type: f.type, size: f.size, base64: f.base64 });
|
||||
}
|
||||
if (diagPendingFiles.length > 0) {
|
||||
addChat('sent', `${diagPendingFiles.length} Anhang/Anhaenge`, 'Datei');
|
||||
}
|
||||
diagPendingFiles = [];
|
||||
renderDiagPending();
|
||||
}
|
||||
|
||||
function testGateway() {
|
||||
const input = document.getElementById('chat-input');
|
||||
const text = input.value.trim();
|
||||
if (!text) return;
|
||||
addChat('sent', text, 'Gateway direkt');
|
||||
send({ action: 'test_gateway', text });
|
||||
if (!text && diagPendingFiles.length === 0) return;
|
||||
if (diagPendingFiles.length > 0) sendDiagAttachments();
|
||||
if (text) {
|
||||
addChat('sent', text, 'Gateway direkt');
|
||||
send({ action: 'test_gateway', text });
|
||||
}
|
||||
input.value = '';
|
||||
}
|
||||
|
||||
function testRVS() {
|
||||
const input = document.getElementById('chat-input');
|
||||
const text = input.value.trim();
|
||||
if (!text) return;
|
||||
addChat('sent', text, 'via RVS');
|
||||
send({ action: 'test_rvs', text });
|
||||
if (!text && diagPendingFiles.length === 0) return;
|
||||
if (diagPendingFiles.length > 0) sendDiagAttachments();
|
||||
if (text) {
|
||||
addChat('sent', text, 'via RVS');
|
||||
send({ action: 'test_rvs', text });
|
||||
}
|
||||
input.value = '';
|
||||
}
|
||||
|
||||
@@ -1302,6 +1367,56 @@
|
||||
}
|
||||
}
|
||||
|
||||
// ── Diagnostic Anhang-Handling ─────────────
|
||||
let diagPendingFiles = [];
|
||||
|
||||
function handleDiagFileSelect(files) {
|
||||
for (const file of files) {
|
||||
const reader = new FileReader();
|
||||
reader.onload = () => {
|
||||
const base64 = reader.result.split(',')[1];
|
||||
diagPendingFiles.push({ name: file.name, type: file.type, size: file.size, base64 });
|
||||
renderDiagPending();
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
}
|
||||
}
|
||||
|
||||
function handleDiagPaste(event) {
|
||||
const items = event.clipboardData?.items;
|
||||
if (!items) return;
|
||||
for (const item of items) {
|
||||
if (item.kind === 'file') {
|
||||
event.preventDefault();
|
||||
const file = item.getAsFile();
|
||||
if (file) handleDiagFileSelect([file]);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function renderDiagPending() {
|
||||
const container = document.getElementById('diag-pending-attachments');
|
||||
if (diagPendingFiles.length === 0) {
|
||||
container.style.display = 'none';
|
||||
return;
|
||||
}
|
||||
container.style.display = 'flex';
|
||||
container.innerHTML = diagPendingFiles.map((f, i) => {
|
||||
const isImage = f.type.startsWith('image/');
|
||||
const preview = isImage ? `<img src="data:${f.type};base64,${f.base64}" style="width:40px;height:40px;border-radius:4px;object-fit:cover;">` : `<span style="font-size:20px;">📄</span>`;
|
||||
return `<div style="position:relative;display:inline-block;">
|
||||
${preview}
|
||||
<span onclick="removeDiagPending(${i})" style="position:absolute;top:-4px;right:-4px;width:16px;height:16px;border-radius:8px;background:#FF3B30;color:#fff;font-size:10px;cursor:pointer;display:flex;align-items:center;justify-content:center;">X</span>
|
||||
</div>`;
|
||||
}).join('') + `<span style="color:#8888AA;font-size:11px;margin-left:4px;">${diagPendingFiles.length} Datei(en)</span>
|
||||
<span onclick="diagPendingFiles=[];renderDiagPending();" style="color:#FF3B30;font-size:11px;cursor:pointer;margin-left:8px;">Alle X</span>`;
|
||||
}
|
||||
|
||||
function removeDiagPending(idx) {
|
||||
diagPendingFiles.splice(idx, 1);
|
||||
renderDiagPending();
|
||||
}
|
||||
|
||||
// ── Abbrechen ──────────────────────────────
|
||||
function cancelRequest() {
|
||||
send({ action: 'cancel_request' });
|
||||
@@ -1318,7 +1433,8 @@
|
||||
const speedThorsten = parseFloat(document.getElementById('diag-speed-thorsten').value);
|
||||
const ttsEngine = document.getElementById('diag-tts-engine').value;
|
||||
const xttsVoice = document.getElementById('diag-xtts-voice').value;
|
||||
send({ action: 'send_voice_config', defaultVoice, highlightVoice, ttsEnabled, speedRamona, speedThorsten, ttsEngine, xttsVoice });
|
||||
const whisperModel = document.getElementById('diag-whisper-model').value;
|
||||
send({ action: 'send_voice_config', defaultVoice, highlightVoice, ttsEnabled, speedRamona, speedThorsten, ttsEngine, xttsVoice, whisperModel });
|
||||
}
|
||||
|
||||
// ── Highlight-Trigger ────────────────────────
|
||||
@@ -1605,7 +1721,8 @@
|
||||
+ `<td style="padding:4px 6px;color:#8888AA;font-size:10px;">${date}</td>`
|
||||
+ `<td style="padding:4px 6px;white-space:nowrap;">`
|
||||
+ (isActive ? '' : `<button class="btn secondary" onclick="event.stopPropagation();activateSession('${escapeHtml(s.sessionKey)}')" style="padding:2px 6px;font-size:10px;color:#34C759;margin-right:2px;" title="Aktivieren">▶</button>`)
|
||||
+ `<button class="btn secondary" onclick="event.stopPropagation();deleteSession('${escapeHtml(s.path)}')" style="padding:2px 6px;font-size:10px;color:#FF6B6B;" title="Loeschen">X</button>`
|
||||
+ `<button class="btn secondary" onclick="event.stopPropagation();deleteSession('${escapeHtml(s.path)}')" style="padding:2px 6px;font-size:10px;color:#FF6B6B;margin-right:2px;" title="Loeschen">X</button>`
|
||||
+ `<button class="btn secondary" onclick="event.stopPropagation();exportSession('${escapeHtml(s.path)}','${escapeHtml(s.sessionKey)}')" style="padding:2px 6px;font-size:10px;color:#8888AA;" title="Als Markdown exportieren">⬇</button>`
|
||||
+ `</td></tr>`;
|
||||
}
|
||||
html += '</table>';
|
||||
@@ -1669,6 +1786,10 @@
|
||||
send({ action: 'delete_session', sessionPath: path });
|
||||
}
|
||||
|
||||
function exportSession(path, sessionKey) {
|
||||
send({ action: 'export_session', sessionPath: path, sessionKey });
|
||||
}
|
||||
|
||||
function activateSession(sessionKey) {
|
||||
send({ action: 'set_active_session', sessionKey });
|
||||
}
|
||||
|
||||
+166
-23
@@ -37,15 +37,41 @@ const state = {
|
||||
};
|
||||
const SESSION_KEY_FILE = "/data/active-session";
|
||||
// /data Verzeichnis sicherstellen (Volume Mount)
|
||||
try { fs.mkdirSync("/data", { recursive: true }); } catch {}
|
||||
try { fs.mkdirSync("/data", { recursive: true }); } catch (e) {
|
||||
console.error(`[startup] /data mkdir fehlgeschlagen: ${e.message}`);
|
||||
}
|
||||
// sessionFromFile zeigt an, ob der aktive Key aus der Datei kam.
|
||||
// Wenn true, darf resolveActiveSession NICHT mehr auto-picken (Wahl respektieren).
|
||||
let sessionFromFile = false;
|
||||
let activeSessionKey = (() => {
|
||||
try {
|
||||
const saved = fs.readFileSync(SESSION_KEY_FILE, "utf-8").trim();
|
||||
if (saved) { console.log(`[startup] Gespeicherte Session geladen: '${saved}'`); return saved; }
|
||||
} catch {}
|
||||
if (saved) {
|
||||
console.log(`[startup] Gespeicherte Session geladen: '${saved}'`);
|
||||
sessionFromFile = true;
|
||||
return saved;
|
||||
}
|
||||
} catch (e) {
|
||||
console.error(`[startup] SESSION_KEY_FILE read: ${e.code || e.message}`);
|
||||
}
|
||||
console.log("[startup] Keine gespeicherte Session — Fallback 'main'");
|
||||
return "main";
|
||||
})();
|
||||
|
||||
// Atomic write: temp-file + rename, laute Logs bei Fehler.
|
||||
function persistActiveSession(key) {
|
||||
try {
|
||||
const tmp = SESSION_KEY_FILE + ".tmp";
|
||||
fs.writeFileSync(tmp, key);
|
||||
fs.renameSync(tmp, SESSION_KEY_FILE);
|
||||
sessionFromFile = true;
|
||||
console.log(`[session] Aktive Session persistiert: '${key}'`);
|
||||
return true;
|
||||
} catch (e) {
|
||||
console.error(`[session] FEHLER beim Persistieren von '${key}': ${e.message}`);
|
||||
return false;
|
||||
}
|
||||
}
|
||||
const logs = [];
|
||||
let gatewayWs = null;
|
||||
let rvsWs = null;
|
||||
@@ -56,6 +82,12 @@ const browserClients = new Set();
|
||||
let pipelineActive = false;
|
||||
let pipelineStartTime = 0;
|
||||
|
||||
// Nach chat:final kommen oft noch Trailing Agent-Events. Waehrend dieses
|
||||
// Fensters unterdruecken wir agent_activity-Broadcasts, damit der
|
||||
// Thinking-Indicator nicht wieder anspringt.
|
||||
let lastChatFinalAt = 0;
|
||||
const SETTLED_WINDOW_MS = 3000;
|
||||
|
||||
function plog(message, level) {
|
||||
const elapsed = pipelineActive ? `+${Date.now() - pipelineStartTime}ms` : "";
|
||||
const entry = { ts: new Date().toISOString(), level: level || "info", source: "pipeline", message: `${elapsed ? `[${elapsed}] ` : ""}${message}` };
|
||||
@@ -91,6 +123,9 @@ function pipelineEnd(ok, detail) {
|
||||
}
|
||||
plog(`━━━ Pipeline Ende ━━━`);
|
||||
pipelineActive = false;
|
||||
// Thinking-Indikator IMMER zuruecksetzen — auch bei Timeout/Fehler/Abbruch
|
||||
broadcast({ type: "agent_activity", activity: "idle" });
|
||||
pendingMessageTime = 0;
|
||||
}
|
||||
|
||||
// ── Auto-Restart bei Netzwerk-Namespace-Verlust ──────
|
||||
@@ -257,8 +292,10 @@ async function connectGateway() {
|
||||
state.gateway.handshakeOk = false;
|
||||
gatewayWs = null;
|
||||
broadcastState();
|
||||
// Stuck "ARIA denkt..." vermeiden, falls Gateway waehrend Pipeline abkackt
|
||||
if (pipelineActive) pipelineEnd(false, `Gateway-Verbindung verloren (${code})`);
|
||||
else broadcast({ type: "agent_activity", activity: "idle" });
|
||||
checkGatewayHealth();
|
||||
// Auto-Reconnect nach 5s
|
||||
setTimeout(connectGateway, 5000);
|
||||
});
|
||||
|
||||
@@ -325,17 +362,22 @@ function handleGatewayMessage(msg) {
|
||||
broadcast({ type: "chat_delta", delta, payload });
|
||||
}
|
||||
|
||||
// Nach chat:final trickeln noch Aufraeum-Events rein — unterdruecken,
|
||||
// damit der Thinking-Indicator nicht wieder anspringt.
|
||||
const settled = lastChatFinalAt && (Date.now() - lastChatFinalAt) < SETTLED_WINDOW_MS;
|
||||
|
||||
// Tool-Nutzung erkennen und broadcasten
|
||||
if (stream === "tool_use" || data.type === "tool_use") {
|
||||
const toolName = data.name || data.tool || payload.tool || "";
|
||||
if (toolName) {
|
||||
if (toolName && !settled) {
|
||||
broadcast({ type: "agent_activity", activity: "tool", tool: toolName, data });
|
||||
log("info", "gateway", `Tool: ${toolName}`);
|
||||
}
|
||||
}
|
||||
|
||||
// Genereller Activity-Heartbeat (ARIA denkt)
|
||||
broadcast({ type: "agent_activity", activity: stream || "thinking" });
|
||||
if (!settled) {
|
||||
broadcast({ type: "agent_activity", activity: stream || "thinking" });
|
||||
}
|
||||
updateAgentActivity();
|
||||
return;
|
||||
}
|
||||
@@ -350,6 +392,7 @@ function handleGatewayMessage(msg) {
|
||||
if (runId && seenFinalRuns.has(runId)) return; // Duplikat
|
||||
if (runId) { seenFinalRuns.add(runId); setTimeout(() => seenFinalRuns.delete(runId), 60000); }
|
||||
log("info", "gateway", `ANTWORT: "${text.slice(0, 200)}"`);
|
||||
lastChatFinalAt = Date.now();
|
||||
if (pipelineActive) pipelineEnd(true, `"${text.slice(0, 120)}"`);
|
||||
broadcast({ type: "chat_final", text, payload });
|
||||
broadcast({ type: "agent_activity", activity: "idle" });
|
||||
@@ -372,6 +415,7 @@ function handleGatewayMessage(msg) {
|
||||
const error = payload.error || text || "Unbekannt";
|
||||
log("error", "gateway", `Chat-Fehler: ${error}`);
|
||||
if (pipelineActive) pipelineEnd(false, error);
|
||||
else broadcast({ type: "agent_activity", activity: "idle" });
|
||||
broadcast({ type: "chat_error", error, payload });
|
||||
return;
|
||||
}
|
||||
@@ -392,7 +436,9 @@ function handleGatewayMessage(msg) {
|
||||
if (runId) { seenFinalRuns.add(runId); setTimeout(() => seenFinalRuns.delete(runId), 60000); }
|
||||
const text = extractChatText(payload) || payload.text || "";
|
||||
log("info", "gateway", `ANTWORT: "${text.slice(0, 200)}"`);
|
||||
lastChatFinalAt = Date.now();
|
||||
if (pipelineActive) pipelineEnd(true, `"${text.slice(0, 120)}"`);
|
||||
else broadcast({ type: "agent_activity", activity: "idle" });
|
||||
broadcast({ type: "chat_final", text, payload });
|
||||
return;
|
||||
}
|
||||
@@ -400,6 +446,7 @@ function handleGatewayMessage(msg) {
|
||||
const error = payload.error || payload.message || "Unbekannt";
|
||||
log("error", "gateway", `Chat-Fehler: ${error}`);
|
||||
if (pipelineActive) pipelineEnd(false, error);
|
||||
else broadcast({ type: "agent_activity", activity: "idle" });
|
||||
broadcast({ type: "chat_error", error, payload });
|
||||
return;
|
||||
}
|
||||
@@ -1109,6 +1156,16 @@ const server = http.createServer((req, res) => {
|
||||
} else if (req.url === "/api/session") {
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ sessionKey: activeSessionKey }));
|
||||
} else if (req.url === "/api/cancel" && req.method === "POST") {
|
||||
log("warn", "server", "HTTP /api/cancel — Cancel-Request (von Bridge)");
|
||||
pendingMessageTime = 0;
|
||||
watchdogWarned = false;
|
||||
watchdogFixAttempted = false;
|
||||
if (pipelineActive) pipelineEnd(false, "Vom Benutzer abgebrochen (App)");
|
||||
else broadcast({ type: "agent_activity", activity: "idle" });
|
||||
dockerExec("aria-core", "openclaw doctor --fix 2>/dev/null || true").catch(() => {});
|
||||
res.writeHead(200, { "Content-Type": "application/json" });
|
||||
res.end(JSON.stringify({ ok: true }));
|
||||
} else if (req.url.startsWith("/shared/")) {
|
||||
// Dateien aus Shared Volume ausliefern (Bilder, Uploads)
|
||||
const filePath = decodeURIComponent(req.url);
|
||||
@@ -1181,6 +1238,14 @@ wss.on("connection", (ws) => {
|
||||
if (ws._sshSock) ws._sshSock.write(msg.data);
|
||||
} else if (msg.action === "live_ssh_close") {
|
||||
if (ws._sshSock) { ws._sshSock.end(); ws._sshSock = null; }
|
||||
} else if (msg.action === "send_file") {
|
||||
// Datei von Diagnostic an Bridge via RVS senden
|
||||
sendToRVS_raw({
|
||||
type: "file",
|
||||
payload: { name: msg.name, type: msg.type, size: msg.size, base64: msg.base64 },
|
||||
timestamp: Date.now(),
|
||||
});
|
||||
log("info", "server", `Datei gesendet: ${msg.name} (${msg.type})`);
|
||||
} else if (msg.action === "cancel_request") {
|
||||
// Laufende Anfrage abbrechen — doctor --fix beendet stuck runs
|
||||
log("warn", "server", "Anfrage abgebrochen — fuehre doctor --fix aus");
|
||||
@@ -1201,7 +1266,11 @@ wss.on("connection", (ws) => {
|
||||
handleGetVoiceConfig(ws);
|
||||
} else if (msg.action === "send_voice_config") {
|
||||
// Stimmen-Config persistent speichern + an Bridge via RVS senden
|
||||
// Bestehende Config lesen um Felder zu mergen die dieser Call nicht setzt
|
||||
let existing = {};
|
||||
try { existing = JSON.parse(fs.readFileSync("/shared/config/voice_config.json", "utf-8")); } catch {}
|
||||
const voiceConfig = {
|
||||
...existing,
|
||||
defaultVoice: msg.defaultVoice || "ramona",
|
||||
highlightVoice: msg.highlightVoice || "thorsten",
|
||||
ttsEnabled: msg.ttsEnabled !== false,
|
||||
@@ -1210,12 +1279,13 @@ wss.on("connection", (ws) => {
|
||||
speedRamona: msg.speedRamona || 1.0,
|
||||
speedThorsten: msg.speedThorsten || 1.0,
|
||||
};
|
||||
if (msg.whisperModel !== undefined) voiceConfig.whisperModel = msg.whisperModel;
|
||||
try {
|
||||
fs.mkdirSync("/shared/config", { recursive: true });
|
||||
fs.writeFileSync("/shared/config/voice_config.json", JSON.stringify(voiceConfig, null, 2));
|
||||
} catch {}
|
||||
sendToRVS_raw({ type: "config", payload: voiceConfig, timestamp: Date.now() });
|
||||
log("info", "server", `Voice-Config gespeichert+gesendet: default=${voiceConfig.defaultVoice}, highlight=${voiceConfig.highlightVoice}, tts=${voiceConfig.ttsEnabled}`);
|
||||
log("info", "server", `Voice-Config gespeichert+gesendet: default=${voiceConfig.defaultVoice}, whisper=${voiceConfig.whisperModel || "-"}`);
|
||||
} else if (msg.action === "get_triggers") {
|
||||
handleGetTriggers(ws);
|
||||
} else if (msg.action === "save_triggers") {
|
||||
@@ -1232,6 +1302,8 @@ wss.on("connection", (ws) => {
|
||||
handleListSessions(ws);
|
||||
} else if (msg.action === "read_session") {
|
||||
handleReadSession(ws, msg.sessionPath);
|
||||
} else if (msg.action === "export_session") {
|
||||
handleExportSession(ws, msg.sessionPath, msg.sessionKey);
|
||||
} else if (msg.action === "delete_session") {
|
||||
handleDeleteSession(ws, msg.sessionPath);
|
||||
} else if (msg.action === "set_active_session") {
|
||||
@@ -1614,6 +1686,68 @@ async function handleReadSession(clientWs, sessionPath) {
|
||||
}
|
||||
}
|
||||
|
||||
async function handleExportSession(clientWs, sessionPath, sessionKey) {
|
||||
if (!sessionPath || sessionPath.includes("..") || !sessionPath.startsWith(SESSIONS_DIR)) {
|
||||
clientWs.send(JSON.stringify({ type: "session_export", ok: false, error: "Ungueltiger Pfad" }));
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const safePath = sessionPath.replace(/'/g, "");
|
||||
const raw = await dockerExec("aria-core", `cat '${safePath}'`);
|
||||
const lines = raw.split("\n").filter(l => l.trim());
|
||||
|
||||
const blocks = [];
|
||||
for (const line of lines) {
|
||||
let obj;
|
||||
try { obj = JSON.parse(line); } catch { continue; }
|
||||
if (obj.type !== "message" || !obj.message) continue;
|
||||
const role = obj.message.role;
|
||||
if (role !== "user" && role !== "assistant") continue;
|
||||
|
||||
let text = "";
|
||||
const content = obj.message.content;
|
||||
if (typeof content === "string") text = content;
|
||||
else if (Array.isArray(content)) text = content.filter(c => c.type === "text").map(c => c.text || "").join("\n");
|
||||
if (!text) continue;
|
||||
|
||||
if (role === "user") {
|
||||
text = text.replace(/^Sender \(untrusted metadata\):[\s\S]*?```[\s\S]*?```\s*\n*/m, "").trim();
|
||||
text = text.replace(/^\[.*?\]\s*/, "").trim();
|
||||
} else {
|
||||
text = text.replace(/^\[\[reply_to_\w+\]\]\s*/g, "").trim();
|
||||
}
|
||||
if (!text) continue;
|
||||
|
||||
const ts = obj.message.timestamp || obj.timestamp || 0;
|
||||
const when = ts ? new Date(ts).toISOString().replace("T", " ").slice(0, 19) : "";
|
||||
const heading = role === "user" ? "## 🧑 User" : "## 🤖 ARIA";
|
||||
blocks.push(`${heading}${when ? ` — ${when}` : ""}\n\n${text}`);
|
||||
}
|
||||
|
||||
const exportedAt = new Date().toISOString().replace("T", " ").slice(0, 19);
|
||||
const title = sessionKey || sessionPath.split("/").pop().replace(".jsonl", "");
|
||||
const markdown = [
|
||||
`# Session: ${title}`,
|
||||
``,
|
||||
`Exportiert: ${exportedAt} `,
|
||||
`Quelle: ${sessionPath}`,
|
||||
``,
|
||||
`---`,
|
||||
``,
|
||||
blocks.join("\n\n---\n\n"),
|
||||
``,
|
||||
].join("\n");
|
||||
|
||||
const safeKey = (sessionKey || "session").replace(/[^a-zA-Z0-9_-]/g, "_");
|
||||
const filename = `${exportedAt.slice(0, 10)}_${safeKey}.md`;
|
||||
clientWs.send(JSON.stringify({ type: "session_export", ok: true, filename, markdown }));
|
||||
log("info", "server", `Session exportiert: ${filename} (${blocks.length} Nachrichten)`);
|
||||
} catch (err) {
|
||||
log("error", "server", `Session-Export fehlgeschlagen: ${err.message}`);
|
||||
clientWs.send(JSON.stringify({ type: "session_export", ok: false, error: err.message }));
|
||||
}
|
||||
}
|
||||
|
||||
async function handleDeleteSession(clientWs, sessionPath) {
|
||||
if (!sessionPath || sessionPath.includes("..") || !sessionPath.startsWith(SESSIONS_DIR)) {
|
||||
clientWs.send(JSON.stringify({ type: "session_deleted", ok: false, error: "Ungueltiger Pfad" }));
|
||||
@@ -1654,13 +1788,11 @@ async function handleDeleteSession(clientWs, sessionPath) {
|
||||
}
|
||||
|
||||
// ── Session-Aufloesung: letzte aktive Session finden ────
|
||||
// Wird nach Gateway-(Re-)Connect aufgerufen. Darf die explizit gewaehlte
|
||||
// Session NIE ueberschreiben — nur beim absoluten Erststart auto-picken.
|
||||
async function resolveActiveSession() {
|
||||
// Nur bei Fallback-Key "main" automatisch aufloesen — gespeicherte Wahl respektieren
|
||||
const hasSavedSession = (() => {
|
||||
try { return !!fs.readFileSync(SESSION_KEY_FILE, "utf-8").trim(); } catch { return false; }
|
||||
})();
|
||||
if (hasSavedSession && activeSessionKey !== "main") {
|
||||
log("info", "server", `Gespeicherte Session '${activeSessionKey}' wird beibehalten`);
|
||||
if (sessionFromFile) {
|
||||
log("info", "server", `Session '${activeSessionKey}' aus /data — keine Auto-Wahl`);
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -1679,10 +1811,19 @@ async function resolveActiveSession() {
|
||||
const keys = entries.map(e => (e.key || e.sessionKey || e.name || "?").replace(/^agent:main:/, ""));
|
||||
log("info", "server", `Verfuegbare Sessions: [${keys.join(", ")}]`);
|
||||
|
||||
// Neueste Session nehmen
|
||||
// Neueste Session nehmen — aber user-definierte bevorzugen.
|
||||
// aria-bridge / aria-diagnostic werden von den Services auto-erstellt;
|
||||
// bei erstem Start soll lieber eine "echte" Session gewaehlt werden,
|
||||
// falls vorhanden.
|
||||
const AUTO_KEYS = new Set(["aria-bridge", "aria-diagnostic"]);
|
||||
const normalise = (e) => (e.key || e.sessionKey || e.name || "").replace(/^agent:main:/, "");
|
||||
|
||||
const userEntries = entries.filter(e => !AUTO_KEYS.has(normalise(e)));
|
||||
const pool = userEntries.length > 0 ? userEntries : entries;
|
||||
|
||||
let newest = null;
|
||||
let newestTime = 0;
|
||||
for (const entry of entries) {
|
||||
for (const entry of pool) {
|
||||
const t = entry.updatedAt || entry.createdAt || 0;
|
||||
if (t >= newestTime) {
|
||||
newestTime = t;
|
||||
@@ -1691,12 +1832,11 @@ async function resolveActiveSession() {
|
||||
}
|
||||
|
||||
if (newest) {
|
||||
const rawKey = newest.key || newest.sessionKey || newest.name || "";
|
||||
const key = rawKey.replace(/^agent:main:/, "");
|
||||
const key = normalise(newest);
|
||||
if (key) {
|
||||
activeSessionKey = key;
|
||||
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
|
||||
log("info", "server", `Aktive Session auf neueste gewechselt: '${activeSessionKey}'`);
|
||||
persistActiveSession(activeSessionKey);
|
||||
log("info", "server", `Auto-Wahl Erststart: '${activeSessionKey}'`);
|
||||
for (const c of browserClients) {
|
||||
c.send(JSON.stringify({ type: "active_session", sessionKey: activeSessionKey }));
|
||||
}
|
||||
@@ -1785,8 +1925,11 @@ function handleSetActiveSession(clientWs, sessionKey) {
|
||||
return;
|
||||
}
|
||||
activeSessionKey = sessionKey;
|
||||
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
|
||||
log("info", "server", `Aktive Session: ${activeSessionKey}`);
|
||||
const ok = persistActiveSession(activeSessionKey);
|
||||
log("info", "server", `Aktive Session: ${activeSessionKey}${ok ? "" : " (WARN: nicht persistiert!)"}`);
|
||||
if (!ok) {
|
||||
clientWs.send(JSON.stringify({ type: "active_session", ok: false, sessionKey: activeSessionKey, error: "Persistierung fehlgeschlagen — /data Volume pruefen" }));
|
||||
}
|
||||
// Allen Clients mitteilen
|
||||
for (const c of browserClients) {
|
||||
c.send(JSON.stringify({ type: "active_session", sessionKey: activeSessionKey }));
|
||||
@@ -1802,7 +1945,7 @@ async function handleCreateSession(clientWs, sessionName) {
|
||||
try {
|
||||
// Session wird automatisch erstellt wenn man die erste Nachricht sendet
|
||||
activeSessionKey = sessionName;
|
||||
try { fs.writeFileSync(SESSION_KEY_FILE, activeSessionKey); } catch {}
|
||||
persistActiveSession(activeSessionKey);
|
||||
log("info", "server", `Neue Session erstellt und aktiviert: ${sessionName}`);
|
||||
// Allen Clients mitteilen
|
||||
for (const c of browserClients) {
|
||||
|
||||
@@ -6,9 +6,9 @@
|
||||
- [x] Sprachnachrichten werden als Text angezeigt (STT → Chat-Bubble)
|
||||
- [x] Cache leeren + Auto-Download von Anhaengen
|
||||
- [x] ARIA liest Nachrichten vor (TTS via Piper)
|
||||
- [x] Autoscroll zur letzten Nachricht
|
||||
- [x] Autoscroll zur letzten Nachricht (inverted FlatList)
|
||||
- [x] Bilder im Chat groesser + Vollbild-Vorschau
|
||||
- [x] Ohr-Button Absturz gefixt (LiveAudioStream entfernt, Phase 1 Placeholder)
|
||||
- [x] Ohr-Button → Gespraechsmodus (Auto-Aufnahme nach ARIA-Antwort)
|
||||
- [x] Play-Button in ARIA-Nachrichten fuer Sprachwiedergabe
|
||||
- [x] Chat-Suche in der App (Lupe in Statusleiste)
|
||||
- [x] Watchdog mit Container-Restart (2min Warnung → 5min doctor --fix → 8min Restart)
|
||||
@@ -22,27 +22,34 @@
|
||||
- [x] XTTS Voice Cloning (Audio-Samples hochladen, eigene Stimme)
|
||||
- [x] TTS Engine waehlbar (Piper/XTTS) in Diagnostic + App
|
||||
- [x] Auto-Update System (APK via RVS WebSocket)
|
||||
- [x] Auto-Update: APK-Installation via FileProvider
|
||||
- [x] Auto-Update: "Auf Updates pruefen" Button in App-Einstellungen
|
||||
- [x] Audio-Queue (sequentielle Wiedergabe, kein Ueberlappen)
|
||||
- [x] Textnachrichten werden von ARIA beantwortet (Bridge chat handler fix)
|
||||
- [x] Mehrere Anhaenge + Text vor dem Senden (Pending-Vorschau)
|
||||
- [x] Paste-Support fuer Bilder in Diagnostic Chat
|
||||
- [x] Markdown-Bereinigung fuer TTS (fett, kursiv, code, links, etc.)
|
||||
- [x] SSH Volume read-write fuer Proxy (kein -F Workaround mehr)
|
||||
- [x] Diagnostic: Sessions als Markdown exportieren (Download-Button)
|
||||
- [x] Speech Gate: Aufnahme wird verworfen wenn keine Sprache erkannt (verhindert dass Umgebungsgeraeusche an Whisper gehen)
|
||||
- [x] Session-Persistenz: Gewaehlte Session bleibt ueber Container-Restarts erhalten (sessionFromFile-Flag, atomic write)
|
||||
- [x] Diagnostic: "ARIA denkt..." bleibt nicht mehr stehen (pipelineEnd broadcastet immer idle, auch bei Timeout/Fehler/Disconnect)
|
||||
- [x] App: "ARIA denkt..." Indicator + Abbrechen-Button (Bridge spiegelt agent_activity via RVS)
|
||||
- [x] Whisper STT: Model-Auswahl in Diagnostic (tiny/base/small/medium/large-v3), Hot-Reload in Bridge, Default auf medium
|
||||
- [x] App: Audio-Aufnahme explizit 16kHz mono (spart Resample, optimal fuer Whisper)
|
||||
|
||||
## Offen
|
||||
|
||||
### Bugs (Prioritaet)
|
||||
- [ ] Session-Persistenz: Bei Container-Restart wird immer aria-bridge geladen statt die zuletzt gewaehlte Session. Wird nicht persistent gespeichert.
|
||||
- [ ] App: Textnachrichten, Bilder und Anhaenge werden von ARIA nicht beantwortet — nur Sprachnachrichten funktionieren.
|
||||
- [ ] App: Audioausgabe hoert ab und zu einfach auf (mitten im Satz oder zwischen Chunks)
|
||||
- [ ] Auto-Update: APK-Installation schlaegt fehl (file:// URI exposed beyond app — braucht FileProvider fuer content:// URI)
|
||||
- [ ] Auto-Update: "Auf Updates pruefen" Button in App-Einstellungen
|
||||
- [ ] App: Kein Auto-Scroll zur letzten Nachricht beim App-Start (soll direkt springen, nicht animiert scrollen)
|
||||
- [ ] App: Bei neuen Nachrichten soll automatisch zur letzten Nachricht gescrollt werden
|
||||
|
||||
### App Features
|
||||
- [ ] App: Zu Anhaengen noch Text/Sprache hinzufuegen koennen (z.B. Bild senden + "Was siehst du?")
|
||||
- [ ] Wake Word on-device (Porcupine "ARIA" Keyword, Phase 2)
|
||||
- [ ] Wake Word on-device (Porcupine "ARIA" Keyword, Phase 2 — passives Lauschen)
|
||||
- [ ] Chat-History zuverlaessiger laden (AsyncStorage Race Condition)
|
||||
- [ ] Background Audio Service (TTS auch bei minimierter App)
|
||||
|
||||
### TTS / Audio
|
||||
- [ ] XTTS Audio-Streaming verbessern (minimales Stottern bei Chunk-Uebergaengen)
|
||||
- [ ] XTTS Audio-Streaming (PCM-Stream statt WAV-Dateien, eliminiert Stottern komplett)
|
||||
- [ ] Audio-Normalisierung (Lautstaerke zwischen Chunks angleichen)
|
||||
- [ ] Piper Voices Download ueber Diagnostic (neue Sprachen/Stimmen)
|
||||
|
||||
@@ -50,4 +57,4 @@
|
||||
- [ ] Bilder: Claude Vision direkt nutzen (aktuell nur Dateipfad an ARIA)
|
||||
- [ ] Auto-Compacting und Memory/Brain Verwaltung (SQLite?)
|
||||
- [ ] Diagnostic: System-Info Tab (Container-Status, Disk, RAM, CPU)
|
||||
- [ ] RVS Zombie-Connections endgueltig loesen (WebRTC statt WebSocket?)
|
||||
- [ ] RVS Zombie-Connections endgueltig loesen
|
||||
|
||||
+4
-1
@@ -76,8 +76,11 @@ echo -e " ${GREEN}✓${NC} SettingsScreen → Version $VERSION"
|
||||
echo ""
|
||||
|
||||
# ── APK bauen ─────────────────────────────────
|
||||
echo -e "${GREEN}[2/5] APK bauen...${NC}"
|
||||
echo -e "${GREEN}[2/5] APK bauen (Cache leeren + Build)...${NC}"
|
||||
cd android
|
||||
# Metro + Gradle Cache leeren damit neue Version sauber eingebettet wird
|
||||
rm -rf node_modules/.cache 2>/dev/null
|
||||
cd android && ./gradlew clean 2>/dev/null; cd ..
|
||||
./build.sh release
|
||||
cd ..
|
||||
|
||||
|
||||
@@ -16,6 +16,7 @@ const ALLOWED_TYPES = new Set([
|
||||
"file_request", "file_response", "file_saved", "stt_result", "config", "tts_request",
|
||||
"xtts_request", "xtts_response", "xtts_list_voices", "xtts_voices_list", "voice_upload", "xtts_voice_saved",
|
||||
"update_check", "update_available", "update_download", "update_data",
|
||||
"agent_activity", "cancel_request",
|
||||
]);
|
||||
|
||||
// Token-Raum: token -> { clients: Set<ws> }
|
||||
|
||||
+16
-2
@@ -97,8 +97,22 @@ async function handleTTSRequest(payload) {
|
||||
const { text, voice, requestId, language } = payload;
|
||||
if (!text) return;
|
||||
|
||||
// Markdown entfernen
|
||||
const cleanText = text.replace(/\*\*([^*]+)\*\*/g, "$1").trim();
|
||||
// Markdown + Sonderzeichen entfernen fuer natuerliche Sprache
|
||||
let cleanText = text
|
||||
.replace(/\*\*([^*]+)\*\*/g, "$1") // **fett** → fett
|
||||
.replace(/\*([^*]+)\*/g, "$1") // *kursiv* → kursiv
|
||||
.replace(/`([^`]+)`/g, "$1") // `code` → code
|
||||
.replace(/```[\s\S]*?```/g, "") // Code-Bloecke entfernen
|
||||
.replace(/\[([^\]]+)\]\([^)]+\)/g, "$1") // [text](url) → text
|
||||
.replace(/#{1,6}\s*/g, "") // ### Ueberschriften → entfernen
|
||||
.replace(/>\s*/g, "") // > Zitate → entfernen
|
||||
.replace(/[-*]\s+/g, "") // - Listen → entfernen
|
||||
.replace(/\n{2,}/g, ". ") // Mehrere Newlines → Punkt
|
||||
.replace(/\n/g, ", ") // Einzelne Newlines → Komma
|
||||
.replace(/\s{2,}/g, " ") // Mehrfach-Leerzeichen
|
||||
.replace(/["""„]/g, "") // Anfuehrungszeichen entfernen
|
||||
.replace(/\(\)/g, "") // Leere Klammern
|
||||
.trim();
|
||||
|
||||
// Text in Saetze aufteilen, dann zu Chunks von 2-3 Saetzen zusammenfassen
|
||||
// (mehr Kontext = konsistentere Stimme/Lautstaerke, aber nicht zu lang fuer WebSocket)
|
||||
|
||||
Reference in New Issue
Block a user