Compare commits

...

7 Commits

1478 changed files with 85954 additions and 74 deletions
+72 -7
View File
@@ -271,9 +271,13 @@ Die Bridge verbindet die Android App mit ARIA und bietet lokale Sprachverarbeitu
**Nachrichtenfluss:**
```
App → RVS → Bridge → aria-core
aria-core → Bridge → RVS → App
→ Lautsprecher (TTS)
Text: App → RVS → Bridge → chat.send → aria-core
Audio: App → RVS → Bridge → FFmpeg → Whisper STT → chat.send → aria-core
Datei: App → RVS → Bridge → /shared/uploads/ → chat.send (mit Pfad) → aria-core
aria-core → Antwort → Gateway → Diagnostic → RVS → App
→ Bridge → Piper TTS → RVS → App (Audio)
→ Bridge → Lautsprecher (lokal)
```
### Features
@@ -335,9 +339,11 @@ API-Endpoint fuer andere Services: `GET http://localhost:3001/api/session`
- Text-Chat mit ARIA
- **Sprachaufnahme**: Push-to-Talk (halten) oder Tap-to-Talk (tippen, Auto-Stop bei Stille)
- **VAD (Voice Activity Detection)**: Erkennt 1.8s Stille und stoppt automatisch
- **Wake Word**: Toggle-Button aktiviert kontinuierliches Mikrofon-Monitoring
- **STT (Speech-to-Text)**: Audio wird in der Bridge per Whisper transkribiert, transkribierter Text erscheint im Chat
- **Wake Word**: Toggle-Button (Ohr-Symbol) aktiviert kontinuierliches Mikrofon-Monitoring
- **TTS-Wiedergabe**: ARIA antwortet per Lautsprecher (Ramona/Thorsten)
- Datei- und Kamera-Upload
- **Datei- und Bild-Upload**: Bilder inline im Chat, Dateien mit Icon + Name + Groesse
- **Anhaenge**: Bridge speichert Dateien in Shared Volume (`/shared/uploads/`), ARIA kann darauf zugreifen
- GPS-Position (optional)
- QR-Code Scanner fuer Token-Pairing
@@ -361,15 +367,67 @@ cd android
# APK liegt unter android/app/build/outputs/apk/release/
```
### Audio-Pipeline
### Release auf Gitea veroeffentlichen
```bash
./release.sh 1.2.0
```
Das Script macht alles in einem Schritt:
1. Fragt Gitea-Kennwort ab (wird nirgends gespeichert)
2. Baut die Release-APK
3. Erstellt Git Tag + pusht
4. Erstellt Gitea Release
5. Laedt APK als Asset hoch
Voraussetzung in `.env`:
```bash
GITEA_URL=https://gitea.hackersoft.de
GITEA_REPO=stefan/aria-agent
GITEA_USER=stefan
```
### Audio-Pipeline (Spracheingabe)
```
App (Mikrofon) → AAC/MP4 Aufnahme → Base64 → RVS → Bridge
Bridge: FFmpeg (16kHz PCM) → Whisper STT → Text → aria-core
Bridge: STT-Ergebnis → RVS → App (Placeholder wird durch transkribierten Text ersetzt)
aria-core → Antwort → Bridge → Piper TTS (WAV) → Base64 → RVS → App
App: Base64 → WAV → Lautsprecher
```
### Datei-Pipeline (Bilder & Anhaenge)
```
App (Kamera/Dateimanager) → Base64 → RVS → Bridge
Bridge: Speichert in /shared/uploads/ (Shared Volume, fuer aria-core sichtbar)
Bridge: chat.send → "Stefan hat ein Bild geschickt: foto.jpg — liegt unter /shared/uploads/..."
ARIA: Kann Datei per Bash/Read-Tool oeffnen und analysieren
```
**Unterstuetzte Formate:** Bilder (JPG, PNG), Dokumente (PDF, DOCX, TXT), beliebige Dateien.
Bilder werden in der App inline angezeigt, andere Dateien als Icon + Dateiname.
**Re-Download:** Wird der lokale Cache in der App geleert (Einstellungen → Anhang-Speicher → Cache leeren),
werden fehlende Anhaenge automatisch ueber RVS vom Server neu geladen. Der Speicherort
ist in den App-Einstellungen konfigurierbar.
> **Tipp Speicherplatz:** Das Docker Volume `aria-shared` liegt standardmaessig auf ARIAs VM-Disk.
> Bei vielen Uploads kann das den Speicher der VM belasten (dort laufen auch alle Container).
> Empfehlung: Das Volume auf ein Netzwerk-Filesystem mounten (CephFS, NFS, GlusterFS):
> ```yaml
> # docker-compose.yml
> volumes:
> aria-shared:
> driver: local
> driver_opts:
> type: nfs
> o: addr=nas.local,rw
> device: ":/exports/aria-uploads"
> ```
> So bleibt ARIAs VM-Disk sauber und die Uploads liegen auf dediziertem Storage.
---
## Datenverzeichnis — aria-data/
@@ -433,6 +491,8 @@ docker compose up -d
| `./aria-data/ssh` (bind) | `/root/.ssh`, `/home/node/.ssh` | SSH Keys |
| `./aria-data/brain` (bind) | `/home/node/.openclaw/workspace/memory` | Gedaechtnis |
| `./aria-data/skills` (bind) | `/home/node/.openclaw/workspace/skills` | Skills |
| `aria-shared` | `/shared` (Core + Bridge) | Datei-Austausch (Uploads von App) |
| `./aria-data/config/diag-state` (bind) | `/data` (Diagnostic) | Persistenter State (aktive Session) |
---
@@ -487,8 +547,13 @@ docker exec aria-core ssh aria-wohnung hostname
Dadurch ist ARIA langsamer als die direkte Claude CLI. Timeout ist auf 900s (15 Min).
- **Kein Streaming zur App**: Die App zeigt erst die fertige Antwort, keine Streaming-Tokens.
- **Wake Word nur auf VM**: Die Bridge hoert auf "ARIA" ueber das lokale Mikrofon der VM.
In der App gibt es Energy-basierte Erkennung (Phase 1).
In der App gibt es Energy-basierte Erkennung (Phase 1). On-device "ARIA"-Keyword (Porcupine) ist Phase 2.
- **Audio-Format**: App nimmt AAC/MP4 auf, Bridge konvertiert via FFmpeg zu 16kHz PCM.
- **Bildanalyse eingeschraenkt**: Bilder werden in `/shared/uploads/` gespeichert. ARIA kann
sie per Bash/Read-Tool oeffnen, aber Claude Vision (direkte Bildanalyse) ist ueber den
Proxy-Pfad (`claude --print`) noch nicht moeglich. ARIA sieht den Dateipfad, nicht das Bild.
- **Dateigroesse**: Grosse Dateien (>5MB) koennen WebSocket-Limits ueberschreiten.
Bilder werden in der App auf max 1920x1920px @ 80% Qualitaet komprimiert.
---
@@ -0,0 +1,11 @@
<?xml version="1.0" encoding="utf-8"?>
<resources>
<!-- Automatically generated file. DO NOT MODIFY -->
<!-- Value from default config. -->
<integer name="react_native_dev_server_port">8081</integer>
<!-- Value from default config. -->
<integer name="react_native_inspector_proxy_port">8081</integer>
</resources>
File diff suppressed because one or more lines are too long
@@ -0,0 +1,2 @@
#- File Locator -
listingFile=../../../outputs/apk/release/output-metadata.json
@@ -0,0 +1,2 @@
appMetadataVersion=1.1
androidGradlePluginVersion=8.1.1
@@ -0,0 +1,10 @@
{
"version": 3,
"artifactType": {
"type": "COMPATIBLE_SCREEN_MANIFEST",
"kind": "Directory"
},
"applicationId": "com.ariacockpit",
"variantName": "release",
"elements": []
}
@@ -0,0 +1,117 @@
# This is a configuration file for ProGuard.
# http://proguard.sourceforge.net/index.html#manual/usage.html
#
# Starting with version 2.2 of the Android plugin for Gradle, this file is distributed together with
# the plugin and unpacked at build-time. The files in $ANDROID_HOME are no longer maintained and
# will be ignored by new version of the Android plugin for Gradle.
# Optimizations: If you don't want to optimize, use the proguard-android.txt configuration file
# instead of this one, which turns off the optimization flags.
# Adding optimization introduces certain risks, since for example not all optimizations performed by
# ProGuard works on all versions of Dalvik. The following flags turn off various optimizations
# known to have issues, but the list may not be complete or up to date. (The "arithmetic"
# optimization can be used if you are only targeting Android 2.0 or later.) Make sure you test
# thoroughly if you go this route.
-optimizations !code/simplification/arithmetic,!code/simplification/cast,!field/*,!class/merging/*
-optimizationpasses 5
-allowaccessmodification
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-verbose
# Preserve some attributes that may be required for reflection.
-keepattributes AnnotationDefault,
EnclosingMethod,
InnerClasses,
RuntimeVisibleAnnotations,
RuntimeVisibleParameterAnnotations,
RuntimeVisibleTypeAnnotations,
Signature
-keep public class com.google.vending.licensing.ILicensingService
-keep public class com.android.vending.licensing.ILicensingService
-keep public class com.google.android.vending.licensing.ILicensingService
-dontnote com.android.vending.licensing.ILicensingService
-dontnote com.google.vending.licensing.ILicensingService
-dontnote com.google.android.vending.licensing.ILicensingService
# For native methods, see http://proguard.sourceforge.net/manual/examples.html#native
-keepclasseswithmembernames,includedescriptorclasses class * {
native <methods>;
}
# Keep setters in Views so that animations can still work.
-keepclassmembers public class * extends android.view.View {
void set*(***);
*** get*();
}
# We want to keep methods in Activity that could be used in the XML attribute onClick.
-keepclassmembers class * extends android.app.Activity {
public void *(android.view.View);
}
# For enumeration classes, see http://proguard.sourceforge.net/manual/examples.html#enumerations
-keepclassmembers enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-keepclassmembers class * implements android.os.Parcelable {
public static final ** CREATOR;
}
# Preserve annotated Javascript interface methods.
-keepclassmembers class * {
@android.webkit.JavascriptInterface <methods>;
}
# The support libraries contains references to newer platform versions.
# Don't warn about those in case this app is linking against an older
# platform version. We know about them, and they are safe.
-dontnote android.support.**
-dontnote androidx.**
-dontwarn android.support.**
-dontwarn androidx.**
# This class is deprecated, but remains for backward compatibility.
-dontwarn android.util.FloatMath
# Understand the @Keep support annotation.
-keep class android.support.annotation.Keep
-keep class androidx.annotation.Keep
-keep @android.support.annotation.Keep class * {*;}
-keep @androidx.annotation.Keep class * {*;}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <init>(...);
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <init>(...);
}
# These classes are duplicated between android.jar and org.apache.http.legacy.jar.
-dontnote org.apache.http.**
-dontnote android.net.http.**
# These classes are duplicated between android.jar and core-lambda-stubs.jar.
-dontnote java.lang.invoke.**
@@ -0,0 +1,116 @@
# This is a configuration file for ProGuard.
# http://proguard.sourceforge.net/index.html#manual/usage.html
#
# Starting with version 2.2 of the Android plugin for Gradle, this file is distributed together with
# the plugin and unpacked at build-time. The files in $ANDROID_HOME are no longer maintained and
# will be ignored by new version of the Android plugin for Gradle.
# Optimization is turned off by default. Dex does not like code run
# through the ProGuard optimize steps (and performs some
# of these optimizations on its own).
# Note that if you want to enable optimization, you cannot just
# include optimization flags in your own project configuration file;
# instead you will need to point to the
# "proguard-android-optimize.txt" file instead of this one from your
# project.properties file.
-dontoptimize
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-verbose
# Preserve some attributes that may be required for reflection.
-keepattributes AnnotationDefault,
EnclosingMethod,
InnerClasses,
RuntimeVisibleAnnotations,
RuntimeVisibleParameterAnnotations,
RuntimeVisibleTypeAnnotations,
Signature
-keep public class com.google.vending.licensing.ILicensingService
-keep public class com.android.vending.licensing.ILicensingService
-keep public class com.google.android.vending.licensing.ILicensingService
-dontnote com.android.vending.licensing.ILicensingService
-dontnote com.google.vending.licensing.ILicensingService
-dontnote com.google.android.vending.licensing.ILicensingService
# For native methods, see http://proguard.sourceforge.net/manual/examples.html#native
-keepclasseswithmembernames,includedescriptorclasses class * {
native <methods>;
}
# Keep setters in Views so that animations can still work.
-keepclassmembers public class * extends android.view.View {
void set*(***);
*** get*();
}
# We want to keep methods in Activity that could be used in the XML attribute onClick.
-keepclassmembers class * extends android.app.Activity {
public void *(android.view.View);
}
# For enumeration classes, see http://proguard.sourceforge.net/manual/examples.html#enumerations
-keepclassmembers enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-keepclassmembers class * implements android.os.Parcelable {
public static final ** CREATOR;
}
# Preserve annotated Javascript interface methods.
-keepclassmembers class * {
@android.webkit.JavascriptInterface <methods>;
}
# The support libraries contains references to newer platform versions.
# Don't warn about those in case this app is linking against an older
# platform version. We know about them, and they are safe.
-dontnote android.support.**
-dontnote androidx.**
-dontwarn android.support.**
-dontwarn androidx.**
# This class is deprecated, but remains for backward compatibility.
-dontwarn android.util.FloatMath
# Understand the @Keep support annotation.
-keep class android.support.annotation.Keep
-keep class androidx.annotation.Keep
-keep @android.support.annotation.Keep class * {*;}
-keep @androidx.annotation.Keep class * {*;}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <init>(...);
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <init>(...);
}
# These classes are duplicated between android.jar and org.apache.http.legacy.jar.
-dontnote org.apache.http.**
-dontnote android.net.http.**
# These classes are duplicated between android.jar and core-lambda-stubs.jar.
-dontnote java.lang.invoke.**
@@ -0,0 +1,117 @@
# This is a configuration file for ProGuard.
# http://proguard.sourceforge.net/index.html#manual/usage.html
#
# Starting with version 2.2 of the Android plugin for Gradle, this file is distributed together with
# the plugin and unpacked at build-time. The files in $ANDROID_HOME are no longer maintained and
# will be ignored by new version of the Android plugin for Gradle.
# Optimizations can be turned on and off in the 'postProcessing' DSL block.
# The configuration below is applied if optimizations are enabled.
# Adding optimization introduces certain risks, since for example not all optimizations performed by
# ProGuard works on all versions of Dalvik. The following flags turn off various optimizations
# known to have issues, but the list may not be complete or up to date. (The "arithmetic"
# optimization can be used if you are only targeting Android 2.0 or later.) Make sure you test
# thoroughly if you go this route.
-optimizations !code/simplification/arithmetic,!code/simplification/cast,!field/*,!class/merging/*
-optimizationpasses 5
-allowaccessmodification
-dontusemixedcaseclassnames
-dontskipnonpubliclibraryclasses
-verbose
# Preserve some attributes that may be required for reflection.
-keepattributes AnnotationDefault,
EnclosingMethod,
InnerClasses,
RuntimeVisibleAnnotations,
RuntimeVisibleParameterAnnotations,
RuntimeVisibleTypeAnnotations,
Signature
-keep public class com.google.vending.licensing.ILicensingService
-keep public class com.android.vending.licensing.ILicensingService
-keep public class com.google.android.vending.licensing.ILicensingService
-dontnote com.android.vending.licensing.ILicensingService
-dontnote com.google.vending.licensing.ILicensingService
-dontnote com.google.android.vending.licensing.ILicensingService
# For native methods, see http://proguard.sourceforge.net/manual/examples.html#native
-keepclasseswithmembernames,includedescriptorclasses class * {
native <methods>;
}
# Keep setters in Views so that animations can still work.
-keepclassmembers public class * extends android.view.View {
void set*(***);
*** get*();
}
# We want to keep methods in Activity that could be used in the XML attribute onClick.
-keepclassmembers class * extends android.app.Activity {
public void *(android.view.View);
}
# For enumeration classes, see http://proguard.sourceforge.net/manual/examples.html#enumerations
-keepclassmembers enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-keepclassmembers class * implements android.os.Parcelable {
public static final ** CREATOR;
}
# Preserve annotated Javascript interface methods.
-keepclassmembers class * {
@android.webkit.JavascriptInterface <methods>;
}
# The support libraries contains references to newer platform versions.
# Don't warn about those in case this app is linking against an older
# platform version. We know about them, and they are safe.
-dontnote android.support.**
-dontnote androidx.**
-dontwarn android.support.**
-dontwarn androidx.**
# This class is deprecated, but remains for backward compatibility.
-dontwarn android.util.FloatMath
# Understand the @Keep support annotation.
-keep class android.support.annotation.Keep
-keep class androidx.annotation.Keep
-keep @android.support.annotation.Keep class * {*;}
-keep @androidx.annotation.Keep class * {*;}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <methods>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <fields>;
}
-keepclasseswithmembers class * {
@android.support.annotation.Keep <init>(...);
}
-keepclasseswithmembers class * {
@androidx.annotation.Keep <init>(...);
}
# These classes are duplicated between android.jar and org.apache.http.legacy.jar.
-dontnote org.apache.http.**
-dontnote android.net.http.**
# These classes are duplicated between android.jar and core-lambda-stubs.jar.
-dontnote java.lang.invoke.**

Some files were not shown because too many files have changed in this diff Show More