feat(app): Wake-Word komplett on-device via openWakeWord (ONNX)
Picovoice/Porcupine raus — neuer Stack ist openWakeWord (Apache 2.0, on-device, ONNX Runtime). Kein API-Key, keine Lizenzgebuehren, Audio verlaesst das Geraet nicht. Eigene Wake-Words sind via openWakeWord- Notebook gratis trainierbar. Pipeline (alles im OpenWakeWordModule.kt): 1. AudioRecord 16kHz mono int16 in 1280-Sample-Chunks (80ms) 2. melspectrogram.onnx → 32-mel Frames (mel/10 + 2 wie in Python) 3. embedding_model.onnx, 76-Frame Sliding Window (stride 8) → 96-dim 4. hey_jarvis.onnx (oder anderes Keyword) auf letzten 16 Embeddings 5. Sigmoid-Score, threshold/patience/debounce-Filter 6. RN-Event "WakeWordDetected" raus Mitgelieferte Modelle in assets/openwakeword/: hey_jarvis (Default), alexa, hey_mycroft, hey_rhasspy. Externe Service-API (start/stop/ configure/onWakeWord/...) bleibt identisch — ChatScreen unveraendert. build.gradle: com.microsoft.onnxruntime:onnxruntime-android:1.17.1 package.json: @picovoice/porcupine-react-native + voice-processor raus SettingsScreen: AccessKey-Feld weg, neue Keyword-Liste mit Labels README: Wake-Word-Sektion komplett umgeschrieben (kein Picovoice mehr) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -41,9 +41,9 @@ import {
|
||||
TTS_SPEED_STORAGE_KEY,
|
||||
} from '../services/audio';
|
||||
import wakeWordService, {
|
||||
BUILTIN_KEYWORDS,
|
||||
WAKE_KEYWORDS,
|
||||
KEYWORD_LABELS,
|
||||
DEFAULT_KEYWORD,
|
||||
WAKE_ACCESS_KEY_STORAGE,
|
||||
WAKE_KEYWORD_STORAGE,
|
||||
} from '../services/wakeword';
|
||||
import ModeSelector from '../components/ModeSelector';
|
||||
@@ -103,8 +103,6 @@ const SettingsScreen: React.FC = () => {
|
||||
const [vadSilenceSec, setVadSilenceSec] = useState<number>(VAD_SILENCE_DEFAULT_SEC);
|
||||
const [convWindowSec, setConvWindowSec] = useState<number>(CONV_WINDOW_DEFAULT_SEC);
|
||||
const [ttsSpeed, setTtsSpeed] = useState<number>(TTS_SPEED_DEFAULT);
|
||||
const [wakeAccessKey, setWakeAccessKey] = useState<string>('');
|
||||
const [wakeAccessKeyVisible, setWakeAccessKeyVisible] = useState(false);
|
||||
const [wakeKeyword, setWakeKeyword] = useState<string>(DEFAULT_KEYWORD);
|
||||
const [wakeStatus, setWakeStatus] = useState<string>('');
|
||||
const [editingPath, setEditingPath] = useState(false);
|
||||
@@ -164,11 +162,8 @@ const SettingsScreen: React.FC = () => {
|
||||
if (isFinite(n) && n >= TTS_SPEED_MIN && n <= TTS_SPEED_MAX) setTtsSpeed(n);
|
||||
}
|
||||
});
|
||||
AsyncStorage.getItem(WAKE_ACCESS_KEY_STORAGE).then(saved => {
|
||||
if (saved) setWakeAccessKey(saved);
|
||||
});
|
||||
AsyncStorage.getItem(WAKE_KEYWORD_STORAGE).then(saved => {
|
||||
if (saved) setWakeKeyword(saved);
|
||||
if (saved && (WAKE_KEYWORDS as readonly string[]).includes(saved)) setWakeKeyword(saved);
|
||||
});
|
||||
AsyncStorage.getItem('aria_xtts_voice').then(saved => {
|
||||
if (saved) setXttsVoice(saved);
|
||||
@@ -678,44 +673,23 @@ const SettingsScreen: React.FC = () => {
|
||||
</View>
|
||||
</View>
|
||||
|
||||
{/* === Wake-Word (geraetelokal) === */}
|
||||
{/* === Wake-Word (komplett on-device, openWakeWord) === */}
|
||||
<Text style={styles.sectionTitle}>Wake-Word</Text>
|
||||
<View style={styles.card}>
|
||||
<Text style={styles.toggleHint}>
|
||||
Wenn ein Picovoice-Access-Key eingetragen ist, hoert die App passiv
|
||||
auf das gewaehlte Wake-Word — du kannst dich mit anderen unterhalten,
|
||||
Musik laufen lassen und mit "{wakeKeyword}" eine Konversation mit
|
||||
ARIA starten. Ohne Key oder bei Fehlschlag startet das Ohr direkt
|
||||
eine Konversation (klassischer Modus).
|
||||
Lokale Erkennung via openWakeWord (ONNX, on-device). Kein API-Key,
|
||||
kein Cloud-Roundtrip — Audio verlaesst das Geraet nicht. Wenn das Ohr
|
||||
aktiv ist, hoerst du normal mit; sagst du das Wake-Word, startet eine
|
||||
Konversation mit ARIA.
|
||||
</Text>
|
||||
|
||||
<Text style={[styles.toggleLabel, {marginTop: 16}]}>Picovoice Access Key</Text>
|
||||
<View style={{flexDirection: 'row', alignItems: 'center', gap: 8, marginTop: 6}}>
|
||||
<TextInput
|
||||
style={[styles.input, {flex: 1}]}
|
||||
value={wakeAccessKey}
|
||||
onChangeText={setWakeAccessKey}
|
||||
placeholder="kostenlos auf console.picovoice.ai"
|
||||
placeholderTextColor="#666680"
|
||||
secureTextEntry={!wakeAccessKeyVisible}
|
||||
autoCapitalize="none"
|
||||
autoCorrect={false}
|
||||
/>
|
||||
<TouchableOpacity
|
||||
onPress={() => setWakeAccessKeyVisible(v => !v)}
|
||||
style={{padding: 8}}
|
||||
>
|
||||
<Text style={{fontSize: 18}}>{wakeAccessKeyVisible ? '🙈' : '👁'}</Text>
|
||||
</TouchableOpacity>
|
||||
</View>
|
||||
|
||||
<Text style={[styles.toggleLabel, {marginTop: 16}]}>Wake-Word</Text>
|
||||
<Text style={styles.toggleHint}>
|
||||
Built-In: sofort verwendbar. "ARIA" als Custom-Keyword kommt spaeter
|
||||
ueber Diagnostic-Upload.
|
||||
Eigene Wake-Words via openWakeWord-Notebook trainierbar (gratis).
|
||||
Custom-Upload ueber Diagnostic kommt in einer spaeteren Version.
|
||||
</Text>
|
||||
<View style={{flexDirection: 'row', flexWrap: 'wrap', gap: 6, marginTop: 8}}>
|
||||
{BUILTIN_KEYWORDS.map(kw => (
|
||||
{WAKE_KEYWORDS.map(kw => (
|
||||
<TouchableOpacity
|
||||
key={kw}
|
||||
style={[
|
||||
@@ -728,7 +702,7 @@ const SettingsScreen: React.FC = () => {
|
||||
styles.keywordChipText,
|
||||
wakeKeyword === kw && styles.keywordChipTextActive,
|
||||
]}>
|
||||
{kw}
|
||||
{KEYWORD_LABELS[kw]}
|
||||
</Text>
|
||||
</TouchableOpacity>
|
||||
))}
|
||||
@@ -740,8 +714,8 @@ const SettingsScreen: React.FC = () => {
|
||||
onPress={async () => {
|
||||
setWakeStatus('Initialisiere...');
|
||||
try {
|
||||
const ok = await wakeWordService.configure(wakeAccessKey, wakeKeyword);
|
||||
setWakeStatus(ok ? `✅ "${wakeKeyword}" bereit` : '❌ Fehlgeschlagen — Access Key pruefen');
|
||||
const ok = await wakeWordService.configure(wakeKeyword);
|
||||
setWakeStatus(ok ? `✅ "${KEYWORD_LABELS[wakeKeyword as keyof typeof KEYWORD_LABELS]}" bereit` : '❌ Init-Fehler — Logs pruefen');
|
||||
} catch (err: any) {
|
||||
setWakeStatus('❌ ' + String(err?.message || err).slice(0, 80));
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user