Hacker-Software/ARIA-AGENT
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 236 Wiki Activity

feat(vnc): Fn-Steuertasten-Leiste + Enter-Fix fuer VM-Bedienung

Browse Source 
Die Software-Tastatur liefert weder Enter (Haken) noch Esc/Pfeile/F-Tasten/
Strg/Alt. Neu:
- Haken der Tastatur → onSubmitEditing sendet jetzt Enter (returnKeyType=send).
- Fn-Leiste (Toggle in der ctlBar): Esc, Tab, Pfeile, Pos1/Ende/Bild, Einfg/
  Entf, Enter, F1–F12, Strg+Alt+Entf — direkt an rfb.sendKey.
- Sticky-Modifier Strg/Alt/Shift (one-shot): kombinieren mit der naechsten
  Taste ODER dem naechsten getippten Zeichen → Strg+C, Strg+Alt+Entf, Alt+F …
- noVNC: window.ariaVncKey.combo(mods,ks) haelt Modifier, tappt, laesst los.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
duffyduck
2026-07-21 01:17:56 +02:00
co-authored by Claude Opus 4.8
parent 1c7157327c
commit 2d3ba024a4
2 changed files with 111 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files
android/src/workspace/assets/novncHtml.ts
+14 -4
View File
@@ -101,12 +101,22 @@ export const NOVNC_HTML = `<!doctype html><html><head><meta charset="utf-8">
// Down+Up einer Taste an die VM schicken. // Down+Up einer Taste an die VM schicken.
function tap(keysym, code){ try{ rfb.sendKey(keysym, code||null, true); rfb.sendKey(keysym, code||null, false); }catch(_){} } function tap(keysym, code){ try{ rfb.sendKey(keysym, code||null, true); rfb.sendKey(keysym, code||null, false); }catch(_){} }
// Empfaenger-API: die App (RN-<TextInput>) ruft das per injectJavaScript. // Empfaenger-API: die App (RN-<TextInput> + Sondertasten-Leiste) ruft das
// char(cp) druckbares Zeichen (Codepoint) // per injectJavaScript.
// keysym(ks) Sondertaste als fertiges X11-Keysym (Backspace/Enter/…) // char(cp) druckbares Zeichen (Codepoint)
// keysym(ks) Sondertaste als fertiges X11-Keysym (Enter/Esc/F1/…)
// combo(mods,ks) Modifier(-Keysyms) halten → Taste → wieder loslassen
// (Strg+C, Strg+Alt+Entf, …). mods = Array von Keysyms.
window.ariaVncKey = { window.ariaVncKey = {
char: function(cp){ tap(cpToKeysym(cp)); }, char: function(cp){ tap(cpToKeysym(cp)); },
keysym: function(ks){ tap(ks); } keysym: function(ks){ tap(ks); },
combo: function(mods, ks){
try{
for(var i=0;i<mods.length;i++) rfb.sendKey(mods[i], null, true);
rfb.sendKey(ks, null, true); rfb.sendKey(ks, null, false);
for(var j=mods.length-1;j>=0;j--) rfb.sendKey(mods[j], null, false);
}catch(_){}
}
}; };
// Steuerungs-API fuer die App (per injectJavaScript). // Steuerungs-API fuer die App (per injectJavaScript).
android/src/workspace/tiles/VncTile.tsx
+97 -16
View File
@@ -2,13 +2,16 @@
* VncTile — Live-Desktop der QEMU-VM (noVNC in einer WebView, RFB durch RVS). * VncTile — Live-Desktop der QEMU-VM (noVNC in einer WebView, RFB durch RVS).
* *
* Nur aktiv, wenn das Desktop-Panel offen ist (focused): dann WebView mounten, * Nur aktiv, wenn das Desktop-Panel offen ist (focused): dann WebView mounten,
* bei 'ready' den RVS-VNC-Tunnel oeffnen. Eine kleine Steuerungs-Leiste macht * bei 'ready' den RVS-VNC-Tunnel oeffnen. Zwei Bedien-Leisten machen die VM auf
* die VM auf dem Handy bedienbar: Tastatur einblenden (tippt in die VM), * dem Handy voll bedienbar:
* Strg-Alt-Entf, und Fit ↔ 1:1 umschalten. * - ctlBar (oben rechts): Fn-Leiste ein/aus, Software-Tastatur, Fit ↔ 1:1.
* - keyBar (oben, Fn): echte Steuertasten, die keine Software-Tastatur
* liefert — Esc, Tab, Pfeile, Pos1/Ende/Bild, Einfg/Entf, Enter, F1–F12 und
* Sticky-Modifier Strg/Alt/Shift (fuer Strg+C, Strg+Alt+Entf, …).
*/ */
import React, { useCallback, useEffect, useRef, useState } from 'react'; import React, { useCallback, useEffect, useRef, useState } from 'react';
import { NativeSyntheticEvent, StyleSheet, Text, TextInput, TextInputChangeEventData, TextInputKeyPressEventData, TouchableOpacity, View } from 'react-native'; import { NativeSyntheticEvent, ScrollView, StyleSheet, Text, TextInput, TextInputChangeEventData, TextInputKeyPressEventData, TouchableOpacity, View } from 'react-native';
import { WebView, WebViewMessageEvent } from 'react-native-webview'; import { WebView, WebViewMessageEvent } from 'react-native-webview';
import desktop from '../../services/desktop'; import desktop from '../../services/desktop';
import { NOVNC_HTML } from '../assets/novncHtml'; import { NOVNC_HTML } from '../assets/novncHtml';
@@ -21,6 +24,18 @@ interface Props {
// X11-Keysyms fuer Sondertasten, die kein druckbares Zeichen liefern. // X11-Keysyms fuer Sondertasten, die kein druckbares Zeichen liefern.
const KEYSYM = { Backspace: 0xff08, Enter: 0xff0d, Tab: 0xff09 }; const KEYSYM = { Backspace: 0xff08, Enter: 0xff0d, Tab: 0xff09 };
const MOD = { ctrl: 0xffe3, alt: 0xffe9, shift: 0xffe1 };
const cpToKeysym = (cp: number) => (cp < 0x100 ? cp : 0x01000000 + cp);
// Sondertasten fuer die Fn-Leiste (Label → Keysym).
const NAV_KEYS: { label: string; ks: number }[] = [
{ label: 'Esc', ks: 0xff1b }, { label: 'Tab', ks: 0xff09 },
{ label: '←', ks: 0xff51 }, { label: '↑', ks: 0xff52 }, { label: '↓', ks: 0xff54 }, { label: '→', ks: 0xff53 },
{ label: 'Pos1', ks: 0xff50 }, { label: 'Ende', ks: 0xff57 },
{ label: 'Bild↑', ks: 0xff55 }, { label: 'Bild↓', ks: 0xff56 },
{ label: 'Einfg', ks: 0xff63 }, { label: 'Entf', ks: 0xffff }, { label: '⏎', ks: 0xff0d },
];
const F_KEYS: { label: string; ks: number }[] = Array.from({ length: 12 }, (_, i) => ({ label: 'F' + (i + 1), ks: 0xffbe + i }));
const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => { const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
const webRef = useRef<WebView>(null); const webRef = useRef<WebView>(null);
@@ -28,6 +43,9 @@ const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
const bufRef = useRef(''); // Spiegel des TextInput-Textes const bufRef = useRef(''); // Spiegel des TextInput-Textes
const [status, setStatus] = useState<'idle' | 'connecting' | 'connected' | 'disconnected'>('idle'); const [status, setStatus] = useState<'idle' | 'connecting' | 'connected' | 'disconnected'>('idle');
const [kbdOn, setKbdOn] = useState(false); const [kbdOn, setKbdOn] = useState(false);
const [keyBar, setKeyBar] = useState(false); // Fn-Leiste sichtbar?
const [mods, setMods] = useState({ ctrl: false, alt: false, shift: false });
const modRef = useRef({ ctrl: false, alt: false, shift: false }); // Spiegel fuer Closures
const unsubDataRef = useRef<null | (() => void)>(null); const unsubDataRef = useRef<null | (() => void)>(null);
const teardown = useCallback(() => { const teardown = useCallback(() => {
@@ -44,12 +62,42 @@ const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
webRef.current?.injectJavaScript(`window.ariaVncCtl && window.ariaVncCtl.${fn}(); true;`); webRef.current?.injectJavaScript(`window.ariaVncCtl && window.ariaVncCtl.${fn}(); true;`);
}, []); }, []);
const sendChar = useCallback((cp: number) => {
webRef.current?.injectJavaScript(`window.ariaVncKey && window.ariaVncKey.char(${cp}); true;`);
}, []);
const sendKeysym = useCallback((ks: number) => { const sendKeysym = useCallback((ks: number) => {
webRef.current?.injectJavaScript(`window.ariaVncKey && window.ariaVncKey.keysym(${ks}); true;`); webRef.current?.injectJavaScript(`window.ariaVncKey && window.ariaVncKey.keysym(${ks}); true;`);
}, []); }, []);
const sendCombo = useCallback((modKeysyms: number[], ks: number) => {
webRef.current?.injectJavaScript(`window.ariaVncKey && window.ariaVncKey.combo(${JSON.stringify(modKeysyms)}, ${ks}); true;`);
}, []);
// Aktive Sticky-Modifier als Keysym-Liste; nach dem Anwenden one-shot zuruecksetzen.
const activeMods = useCallback(() => {
const m = modRef.current; const a: number[] = [];
if (m.ctrl) a.push(MOD.ctrl); if (m.alt) a.push(MOD.alt); if (m.shift) a.push(MOD.shift);
return a;
}, []);
const clearMods = useCallback(() => {
if (modRef.current.ctrl || modRef.current.alt || modRef.current.shift) {
modRef.current = { ctrl: false, alt: false, shift: false };
setMods(modRef.current);
}
}, []);
const toggleMod = useCallback((k: 'ctrl' | 'alt' | 'shift') => {
modRef.current = { ...modRef.current, [k]: !modRef.current[k] };
setMods(modRef.current);
}, []);
// Eine Taste (fertiges Keysym) senden — mit ggf. aktiven Modifiern.
const pressKey = useCallback((ks: number) => {
const m = activeMods();
if (m.length) { sendCombo(m, ks); clearMods(); } else sendKeysym(ks);
}, [activeMods, sendCombo, clearMods, sendKeysym]);
// Ein druckbares Zeichen senden — mit ggf. aktiven Modifiern (Strg+C etc.).
const pressChar = useCallback((cp: number) => {
const m = activeMods();
if (m.length) { sendCombo(m, cpToKeysym(cp)); clearMods(); }
else webRef.current?.injectJavaScript(`window.ariaVncKey && window.ariaVncKey.char(${cp}); true;`);
}, [activeMods, sendCombo, clearMods]);
// Tastatur ein-/ausblenden: echtes RN-<TextInput> fokussieren → Android oeffnet // Tastatur ein-/ausblenden: echtes RN-<TextInput> fokussieren → Android oeffnet
// die Software-Tastatur zuverlaessig (anders als ein verstecktes WebView-Feld). // die Software-Tastatur zuverlaessig (anders als ein verstecktes WebView-Feld).
@@ -66,17 +114,18 @@ const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
let i = 0; let i = 0;
const min = Math.min(prev.length, text.length); const min = Math.min(prev.length, text.length);
while (i < min && prev.charCodeAt(i) === text.charCodeAt(i)) i++; while (i < min && prev.charCodeAt(i) === text.charCodeAt(i)) i++;
for (const ch of text.slice(i)) { const cp = ch.codePointAt(0); if (cp) sendChar(cp); } for (const ch of text.slice(i)) { const cp = ch.codePointAt(0); if (cp) pressChar(cp); }
bufRef.current = text; bufRef.current = text;
if (text.length > 200) { bufRef.current = ''; kbdRef.current?.setNativeProps({ text: '' }); } if (text.length > 200) { bufRef.current = ''; kbdRef.current?.setNativeProps({ text: '' }); }
}, [sendChar]); }, [pressChar]);
// Sondertasten: Backspace/Enter feuern auf Android zuverlaessig als keyPress. // Sondertasten der Software-Tastatur: Backspace feuert auf Android zuverlaessig
// als keyPress; die Return-Taste (Haken) kommt als onSubmitEditing (s.u.).
const onKbdKeyPress = useCallback((e: NativeSyntheticEvent<TextInputKeyPressEventData>) => { const onKbdKeyPress = useCallback((e: NativeSyntheticEvent<TextInputKeyPressEventData>) => {
const k = e.nativeEvent.key; const k = e.nativeEvent.key;
if (k === 'Backspace') sendKeysym(KEYSYM.Backspace); if (k === 'Backspace') pressKey(KEYSYM.Backspace);
else if (k === 'Enter') sendKeysym(KEYSYM.Enter); else if (k === 'Enter') pressKey(KEYSYM.Enter);
}, [sendKeysym]); }, [pressKey]);
const onMessage = useCallback((e: WebViewMessageEvent) => { const onMessage = useCallback((e: WebViewMessageEvent) => {
let m: any; let m: any;
@@ -132,8 +181,10 @@ const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
style={styles.hiddenInput} style={styles.hiddenInput}
onChange={onKbdChange} onChange={onKbdChange}
onKeyPress={onKbdKeyPress} onKeyPress={onKbdKeyPress}
onSubmitEditing={() => pressKey(KEYSYM.Enter)}
onBlur={() => setKbdOn(false)} onBlur={() => setKbdOn(false)}
keyboardType="visible-password" keyboardType="visible-password"
returnKeyType="send"
autoCapitalize="none" autoCapitalize="none"
autoCorrect={false} autoCorrect={false}
spellCheck={false} spellCheck={false}
@@ -146,18 +197,38 @@ const VncTile: React.FC<Props> = ({ projectId, focused, port = 5901 }) => {
{/* Steuerungs-Leiste — nur wenn verbunden */} {/* Steuerungs-Leiste — nur wenn verbunden */}
{connected && ( {connected && (
<View style={styles.ctlBar}> <View style={styles.ctlBar}>
<TouchableOpacity style={[styles.ctlBtn, keyBar && styles.ctlBtnOn]} onPress={() => setKeyBar(v => !v)} activeOpacity={0.7}>
<Text style={styles.ctlText}>Fn</Text>
</TouchableOpacity>
<TouchableOpacity style={[styles.ctlBtn, kbdOn && styles.ctlBtnOn]} onPress={toggleKbd} activeOpacity={0.7}> <TouchableOpacity style={[styles.ctlBtn, kbdOn && styles.ctlBtnOn]} onPress={toggleKbd} activeOpacity={0.7}>
<Text style={styles.ctlText}>⌨</Text> <Text style={styles.ctlText}>⌨</Text>
</TouchableOpacity> </TouchableOpacity>
<TouchableOpacity style={styles.ctlBtn} onPress={() => ctl('cad')} activeOpacity={0.7}>
<Text style={styles.ctlTextSmall}>Strg+Alt+Entf</Text>
</TouchableOpacity>
<TouchableOpacity style={styles.ctlBtn} onPress={() => ctl('toggleFit')} activeOpacity={0.7}> <TouchableOpacity style={styles.ctlBtn} onPress={() => ctl('toggleFit')} activeOpacity={0.7}>
<Text style={styles.ctlText}>⤢</Text> <Text style={styles.ctlText}>⤢</Text>
</TouchableOpacity> </TouchableOpacity>
</View> </View>
)} )}
{/* Fn-Leiste — echte Steuertasten (oben, ueber der Software-Tastatur). */}
{connected && keyBar && (
<View style={styles.keyBar} pointerEvents="box-none">
<ScrollView horizontal showsHorizontalScrollIndicator={false} keyboardShouldPersistTaps="always" contentContainerStyle={styles.keyRow}>
<TouchableOpacity style={[styles.key, mods.ctrl && styles.keyOn]} onPress={() => toggleMod('ctrl')} activeOpacity={0.7}><Text style={styles.keyText}>Strg</Text></TouchableOpacity>
<TouchableOpacity style={[styles.key, mods.alt && styles.keyOn]} onPress={() => toggleMod('alt')} activeOpacity={0.7}><Text style={styles.keyText}>Alt</Text></TouchableOpacity>
<TouchableOpacity style={[styles.key, mods.shift && styles.keyOn]} onPress={() => toggleMod('shift')} activeOpacity={0.7}><Text style={styles.keyText}>Shift</Text></TouchableOpacity>
{NAV_KEYS.map(k => (
<TouchableOpacity key={k.label} style={styles.key} onPress={() => pressKey(k.ks)} activeOpacity={0.7}><Text style={styles.keyText}>{k.label}</Text></TouchableOpacity>
))}
</ScrollView>
<ScrollView horizontal showsHorizontalScrollIndicator={false} keyboardShouldPersistTaps="always" contentContainerStyle={styles.keyRow}>
{F_KEYS.map(k => (
<TouchableOpacity key={k.label} style={styles.key} onPress={() => pressKey(k.ks)} activeOpacity={0.7}><Text style={styles.keyText}>{k.label}</Text></TouchableOpacity>
))}
<TouchableOpacity style={styles.key} onPress={() => ctl('cad')} activeOpacity={0.7}><Text style={styles.keyTextSm}>Strg+Alt+Entf</Text></TouchableOpacity>
</ScrollView>
</View>
)}
{!connected && ( {!connected && (
<View style={styles.overlay} pointerEvents="none"> <View style={styles.overlay} pointerEvents="none">
<Text style={styles.overlayText}> <Text style={styles.overlayText}>
@@ -199,6 +270,16 @@ const styles = StyleSheet.create({
ctlTextSmall: { color: '#E0E0F0', fontSize: 11, fontWeight: '700' }, ctlTextSmall: { color: '#E0E0F0', fontSize: 11, fontWeight: '700' },
// Fokussierbar (nicht display:none), aber aus dem Sichtfeld geschoben. // Fokussierbar (nicht display:none), aber aus dem Sichtfeld geschoben.
hiddenInput: { position: 'absolute', width: 1, height: 1, top: -100, left: -100, opacity: 0, padding: 0 }, hiddenInput: { position: 'absolute', width: 1, height: 1, top: -100, left: -100, opacity: 0, padding: 0 },
keyBar: { position: 'absolute', top: 74, left: 0, right: 0, gap: 5 },
keyRow: { paddingHorizontal: 6, gap: 5, alignItems: 'center' },
key: {
backgroundColor: 'rgba(18,18,42,0.92)', borderColor: '#2A2A3E', borderWidth: 1,
borderRadius: 8, paddingHorizontal: 9, paddingVertical: 7, minWidth: 34,
alignItems: 'center', justifyContent: 'center',
},
keyOn: { backgroundColor: 'rgba(0,150,255,0.85)', borderColor: '#0096FF' },
keyText: { color: '#E0E0F0', fontSize: 13, fontWeight: '700' },
keyTextSm: { color: '#E0E0F0', fontSize: 10, fontWeight: '700' },
overlay: { position: 'absolute', top: 12, left: 0, right: 0, alignItems: 'center' }, overlay: { position: 'absolute', top: 12, left: 0, right: 0, alignItems: 'center' },
overlayText: { color: '#9090B0', fontSize: 12, backgroundColor: 'rgba(0,0,0,0.6)', paddingHorizontal: 10, paddingVertical: 4, borderRadius: 10, overflow: 'hidden' }, overlayText: { color: '#9090B0', fontSize: 12, backgroundColor: 'rgba(0,0,0,0.6)', paddingHorizontal: 10, paddingVertical: 4, borderRadius: 10, overflow: 'hidden' },
}); });