Fix HID transfers, harden the tunnel, add E2E crypto and direct peers

The HID failure came down to the endpoint type map being indexed by
endpoint number without the direction bit. A composite device can have
endpoint 1 as both interrupt IN (0x81) and bulk OUT (0x01); the last one
read won, so interrupt URBs were submitted as bulk and the kernel rejected
them. The device attached and stayed silent.

Endpoint data now comes from the raw descriptors read from /dev/bus/usb
rather than sysfs, which only ever exposes the active alternate setting —
a webcam's isochronous endpoints are invisible there because they only
exist after SET_INTERFACE. Two sysfs parsing bugs fell out of that too:
the numeric endpoint attributes are hex without a prefix (wMaxPacketSize
"0040" was read as 40, not 64), and bInterval was never read at all.

Reliability: three places could freeze the whole process. The share path
fed io.Pipe from the WebSocket read loop, so one slow USB transfer stalled
every tunnel and the keepalives with them. The relay wrote to client
sockets while holding the hub lock, so one peer that stopped reading
blocked routing and registration for everyone. Control transfers ran
inline in the protocol loop behind a 5s timeout. Also fixed: a use-after-
free where a discarded URB's memory could be collected while the kernel
still owned it, a reap loop that spun at 100% CPU on ioctl errors, a
missing attach timeout, a double close(done) panic, and Hash[:8] in the
relay's log line, which let a client with a short hash take the server
down.

Adds mode "both", so one client can offer and consume devices at once.
The tunnel and client-left callbacks became multicast for it: as plain
fields the second manager to register silently unhooked the first.

Tunnel traffic is now AES-256-GCM end to end, on the relay path as well
as directly. The key is derived from the three tokens, not from the group
hash — the relay is told the hash, so a key derived from it would protect
nothing from the one party in the middle. Group IDs are unchanged, so
existing setups keep working; only clients configured without the tokens
drop to unencrypted, relay-only operation.

Peers now try to connect directly, with the relay supplying the public
address neither side can determine for itself. Candidates are raced
because an unreachable address hangs until timeout rather than refusing.
Falling back to the relay is not an error.

Platform reach: cross-compiled targets for ARM, MIPS and RISC-V (the
Linux client needed no code changes — usbdevfs is not architecture
specific), multi-arch Docker images, an Android bridge that accepts
devices over SCM_RIGHTS because apps cannot open /dev/bus/usb, and macOS
builds via system_profiler enumeration.

Adds a Windows KMDF filter driver under driver/windows with its Go side.
UNTESTED: it has never been compiled or run, needs the WDK to build and
an EV certificate to distribute. Treat it as a starting point.

Adds "usb-client diag": says per machine whether sharing and using are
possible, what stands in the way, and what fixes it. Reports can be
uploaded to a relay to get them off machines that are awkward to copy
from.

96 tests, all green under -race. Builds for linux, windows and darwin on
amd64 and arm64.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-11 22:02:04 +02:00
co-authored by Claude Opus 5
parent 54178dce75
commit 9ed473a965
95 changed files with 12181 additions and 892 deletions
+358 -72
View File
@@ -10,29 +10,81 @@ import (
"time"
"github.com/duffy/usb-server/internal/config"
"github.com/duffy/usb-server/internal/crypto"
"github.com/duffy/usb-server/internal/protocol"
"github.com/google/uuid"
"github.com/gorilla/websocket"
)
const (
// readTimeout is how long we tolerate silence from the relay. The relay
// pings every 20s, and gorilla answers pings automatically, so exceeding
// this means the connection is genuinely dead — including the case where
// a NAT or proxy dropped it without sending a TCP reset.
readTimeout = 60 * time.Second
// pingInterval is how often we ping the relay ourselves, so that an idle
// tunnel keeps NAT mappings alive from both directions.
pingInterval = 20 * time.Second
// writeTimeout bounds a single frame write.
writeTimeout = 20 * time.Second
// sendQueueDepth bounds outgoing backlog before we consider the link stuck.
sendQueueDepth = 256
// reconnectMin/reconnectMax bound the exponential backoff between
// reconnect attempts, so a relay outage does not turn into a hot loop.
reconnectMin = 1 * time.Second
reconnectMax = 30 * time.Second
)
// outMsg is one queued outgoing WebSocket frame.
type outMsg struct {
typ int
data []byte
}
// Client manages the connection to the relay server
type Client struct {
cfg *config.Config
clientID string
conn *websocket.Conn
mu sync.Mutex
// Event callbacks
OnDeviceList func(msg *protocol.DeviceList)
OnDeviceGranted func(msg *protocol.DeviceGranted)
OnDeviceDenied func(msg *protocol.DeviceDenied)
OnDeviceReleased func(msg *protocol.DeviceReleased)
OnClientJoined func(msg *protocol.ClientJoined)
OnClientLeft func(msg *protocol.ClientLeft)
OnRequestDevice func(targetClient, fromClient, busID, requestID string)
OnReleaseDevice func(busID, fromClient string)
OnForceRelease func(targetClient, fromClient, busID string)
OnTunnelData func(tunnelID string, data []byte)
mu sync.Mutex
conn *websocket.Conn
send chan outMsg
dead chan struct{}
// Callbacks for messages that only one manager can own.
// In "both" mode the share manager takes the share-side ones and the use
// manager the use-side ones, so they never collide.
OnDeviceList func(msg *protocol.DeviceList) // use side
OnDeviceGranted func(msg *protocol.DeviceGranted) // use side
OnDeviceDenied func(msg *protocol.DeviceDenied) // use side
OnDeviceReleased func(msg *protocol.DeviceReleased) // use side
OnClientJoined func(msg *protocol.ClientJoined)
OnRequestDevice func(targetClient, fromClient, busID, requestID string) // share side
OnReleaseDevice func(busID, fromClient string) // share side
OnForceRelease func(targetClient, fromClient, busID string) // share side
// Multicast callbacks. Both managers care about these, so they are lists
// rather than single fields: in "both" mode a plain field would mean the
// second manager to register silently unhooked the first.
tunnelHandlers []func(tunnelID string, data []byte)
clientLeftHandlers []func(msg *protocol.ClientLeft)
disconnectHandlers []func()
handlerMu sync.RWMutex
// OnConnect fires once a registration has been sent successfully.
OnConnect func()
// secret derives per-tunnel keys and peer tokens. Nil when the config
// carries only a group hash, in which case tunnels stay unencrypted and
// direct connections are unavailable.
secret *crypto.TunnelSecret
// directPort is advertised to the relay so peers learn where to reach us.
directPort int
ctx context.Context
cancel context.CancelFunc
@@ -41,12 +93,38 @@ type Client struct {
// NewClient creates a new client instance
func NewClient(cfg *config.Config) *Client {
ctx, cancel := context.WithCancel(context.Background())
return &Client{
c := &Client{
cfg: cfg,
clientID: uuid.New().String(),
ctx: ctx,
cancel: cancel,
}
if cfg.HasTokens() {
secret, err := crypto.DeriveTunnelSecret(cfg.Token1, cfg.Token2, cfg.Token3)
if err != nil {
log.Printf("[client] tunnel encryption unavailable: %v", err)
} else {
c.secret = secret
}
} else {
log.Printf("[client] no tokens configured, only a group hash: " +
"tunnels will not be encrypted and direct connections are unavailable")
}
return c
}
// TunnelSecret returns the group secret, or nil if it could not be derived.
func (c *Client) TunnelSecret() *crypto.TunnelSecret { return c.secret }
// SetDirectPort records the port peers should use to reach this client
// directly. It is announced with the next registration.
func (c *Client) SetDirectPort(port int) {
c.mu.Lock()
c.directPort = port
c.mu.Unlock()
}
// ID returns the client ID
@@ -59,14 +137,18 @@ func (c *Client) Config() *config.Config {
return c.cfg
}
// Connect establishes connection to the relay server
func (c *Client) Connect() error {
// Context returns the client's lifetime context.
func (c *Client) Context() context.Context {
return c.ctx
}
// relayURL normalises the configured relay address into a WebSocket URL.
func (c *Client) relayURL() (string, error) {
u, err := url.Parse(c.cfg.RelayAddr)
if err != nil {
return fmt.Errorf("invalid relay address: %w", err)
return "", fmt.Errorf("invalid relay address: %w", err)
}
// Ensure WebSocket scheme
switch u.Scheme {
case "ws", "wss":
// ok
@@ -78,57 +160,145 @@ func (c *Client) Connect() error {
u.Scheme = "ws"
}
if u.Path == "" {
if u.Path == "" || u.Path == "/" {
u.Path = "/ws"
}
log.Printf("[client] connecting to %s", u.String())
return u.String(), nil
}
conn, _, err := websocket.DefaultDialer.Dial(u.String(), nil)
// Connect establishes connection to the relay server
func (c *Client) Connect() error {
target, err := c.relayURL()
if err != nil {
return err
}
log.Printf("[client] connecting to %s", target)
dialer := websocket.Dialer{
HandshakeTimeout: 15 * time.Second,
ReadBufferSize: 64 * 1024,
WriteBufferSize: 64 * 1024,
}
conn, _, err := dialer.DialContext(c.ctx, target, nil)
if err != nil {
return fmt.Errorf("connecting to relay: %w", err)
}
conn.SetReadLimit(maxMessageSize)
conn.SetReadDeadline(time.Now().Add(readTimeout))
conn.SetPongHandler(func(string) error {
conn.SetReadDeadline(time.Now().Add(readTimeout))
return nil
})
c.mu.Lock()
c.conn = conn
directPort := c.directPort
c.mu.Unlock()
// Send registration
reg := &protocol.Register{
Type: protocol.MsgRegister,
Hash: c.cfg.Hash,
Mode: c.cfg.Mode,
ClientID: c.clientID,
Name: c.cfg.Name,
Type: protocol.MsgRegister,
Hash: c.cfg.Hash,
Mode: c.cfg.Mode,
ClientID: c.clientID,
Name: c.cfg.Name,
DirectPort: directPort,
LocalEndpoints: localEndpoints(directPort),
}
regData, err := json.Marshal(reg)
if err != nil {
conn.Close()
return fmt.Errorf("encoding registration: %w", err)
}
if err := conn.WriteJSON(reg); err != nil {
// The registration is written directly because the write pump is not
// running yet; every later write goes through the pump.
conn.SetWriteDeadline(time.Now().Add(writeTimeout))
if err := conn.WriteMessage(websocket.TextMessage, regData); err != nil {
conn.Close()
return fmt.Errorf("sending registration: %w", err)
}
log.Printf("[client] registered as %s (mode=%s, name=%s)", c.clientID, c.cfg.Mode, c.cfg.Name)
c.mu.Lock()
c.conn = conn
c.send = make(chan outMsg, sendQueueDepth)
c.dead = make(chan struct{})
sendCh, deadCh := c.send, c.dead
c.mu.Unlock()
go c.writePump(conn, sendCh, deadCh)
log.Printf("[client] registered as %s (mode=%s, name=%s)",
protocol.ShortID(c.clientID), c.cfg.Mode, c.cfg.Name)
if c.OnConnect != nil {
c.OnConnect()
}
return nil
}
// RunReadLoop reads messages from the relay and dispatches them
func (c *Client) RunReadLoop() error {
// maxMessageSize must match the relay's limit.
const maxMessageSize = 1024 * 1024
// writePump serialises all writes to the relay socket and sends keepalives.
func (c *Client) writePump(conn *websocket.Conn, send <-chan outMsg, dead <-chan struct{}) {
ticker := time.NewTicker(pingInterval)
defer ticker.Stop()
defer conn.Close() // unblocks the read loop if we give up first
for {
select {
case <-c.ctx.Done():
return nil
default:
}
case msg := <-send:
conn.SetWriteDeadline(time.Now().Add(writeTimeout))
if err := conn.WriteMessage(msg.typ, msg.data); err != nil {
log.Printf("[client] write error: %v", err)
return
}
msgType, data, err := c.conn.ReadMessage()
case <-ticker.C:
conn.SetWriteDeadline(time.Now().Add(writeTimeout))
if err := conn.WriteMessage(websocket.PingMessage, nil); err != nil {
return
}
case <-dead:
return
case <-c.ctx.Done():
return
}
}
}
// RunReadLoop reads messages from the relay and dispatches them
func (c *Client) RunReadLoop() error {
c.mu.Lock()
conn := c.conn
c.mu.Unlock()
if conn == nil {
return fmt.Errorf("not connected")
}
for {
msgType, data, err := conn.ReadMessage()
if err != nil {
select {
case <-c.ctx.Done():
return nil
default:
}
if websocket.IsUnexpectedCloseError(err, websocket.CloseGoingAway, websocket.CloseNormalClosure) {
return fmt.Errorf("read error: %w", err)
}
return nil
return err
}
conn.SetReadDeadline(time.Now().Add(readTimeout))
switch msgType {
case websocket.TextMessage:
c.handleTextMessage(data)
@@ -140,67 +310,176 @@ func (c *Client) RunReadLoop() error {
// Run connects and runs the main loop with auto-reconnect
func (c *Client) Run() error {
backoff := reconnectMin
for {
select {
case <-c.ctx.Done():
return nil
default:
}
if err := c.Connect(); err != nil {
log.Printf("[client] connection failed: %v, retrying in 5s...", err)
select {
case <-time.After(5 * time.Second):
continue
case <-c.ctx.Done():
log.Printf("[client] connection failed: %v, retrying in %s", err, backoff)
if !c.sleep(backoff) {
return nil
}
backoff = nextBackoff(backoff)
continue
}
// Connected: reset the backoff so a later blip retries promptly.
backoff = reconnectMin
err := c.RunReadLoop()
if err != nil {
log.Printf("[client] disconnected: %v, reconnecting in 5s...", err)
log.Printf("[client] disconnected: %v", err)
} else {
log.Printf("[client] disconnected, reconnecting in 5s...")
log.Printf("[client] disconnected")
}
c.mu.Lock()
if c.conn != nil {
c.conn.Close()
c.conn = nil
}
c.mu.Unlock()
c.teardown()
// The relay dropped every tunnel involving us; local state that
// still references one has to go too.
c.fireDisconnect()
select {
case <-time.After(5 * time.Second):
case <-c.ctx.Done():
return nil
default:
}
log.Printf("[client] reconnecting in %s", backoff)
if !c.sleep(backoff) {
return nil
}
backoff = nextBackoff(backoff)
}
}
// nextBackoff doubles the delay up to reconnectMax.
func nextBackoff(d time.Duration) time.Duration {
d *= 2
if d > reconnectMax {
return reconnectMax
}
return d
}
// sleep waits for d, returning false if the client is shutting down.
func (c *Client) sleep(d time.Duration) bool {
timer := time.NewTimer(d)
defer timer.Stop()
select {
case <-timer.C:
return true
case <-c.ctx.Done():
return false
}
}
// teardown closes the current connection and stops its write pump.
func (c *Client) teardown() {
c.mu.Lock()
if c.dead != nil {
close(c.dead)
c.dead = nil
}
if c.conn != nil {
c.conn.Close()
c.conn = nil
}
c.send = nil
c.mu.Unlock()
}
// Close shuts down the client
func (c *Client) Close() {
c.cancel()
c.mu.Lock()
if c.conn != nil {
c.conn.Close()
c.teardown()
}
// AddTunnelHandler registers a handler for incoming tunnel frames.
// Handlers receive every frame and must ignore tunnel IDs they do not own.
func (c *Client) AddTunnelHandler(fn func(tunnelID string, data []byte)) {
c.handlerMu.Lock()
defer c.handlerMu.Unlock()
c.tunnelHandlers = append(c.tunnelHandlers, fn)
}
// AddClientLeftHandler registers a handler for peer disconnects.
func (c *Client) AddClientLeftHandler(fn func(msg *protocol.ClientLeft)) {
c.handlerMu.Lock()
defer c.handlerMu.Unlock()
c.clientLeftHandlers = append(c.clientLeftHandlers, fn)
}
// AddDisconnectHandler registers a handler that runs after the relay
// connection drops and before reconnecting.
//
// The relay forgets every tunnel when a client disconnects, so anything still
// attached locally now points at a tunnel that no longer exists. Handlers use
// this to tear that state down instead of leaving devices wedged until the
// process restarts.
func (c *Client) AddDisconnectHandler(fn func()) {
c.handlerMu.Lock()
defer c.handlerMu.Unlock()
c.disconnectHandlers = append(c.disconnectHandlers, fn)
}
func (c *Client) fireDisconnect() {
c.handlerMu.RLock()
handlers := append([]func(){}, c.disconnectHandlers...)
c.handlerMu.RUnlock()
for _, fn := range handlers {
fn()
}
}
// Connected reports whether the client currently has a live relay connection.
func (c *Client) Connected() bool {
c.mu.Lock()
defer c.mu.Unlock()
return c.conn != nil
}
// enqueue queues an outgoing frame. It never blocks on the socket; a full
// queue means the relay link is stuck, which is reported as an error so the
// caller can tear down whatever it was trying to send.
func (c *Client) enqueue(typ int, data []byte) error {
c.mu.Lock()
send, dead := c.send, c.dead
c.mu.Unlock()
if send == nil {
return fmt.Errorf("not connected")
}
select {
case send <- outMsg{typ: typ, data: data}:
return nil
case <-dead:
return fmt.Errorf("connection closed")
case <-c.ctx.Done():
return fmt.Errorf("client shutting down")
default:
return fmt.Errorf("send queue full, relay link stalled")
}
}
// SendJSON sends a JSON message to the relay
func (c *Client) SendJSON(v interface{}) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn == nil {
return fmt.Errorf("not connected")
data, err := json.Marshal(v)
if err != nil {
return fmt.Errorf("encoding message: %w", err)
}
return c.conn.WriteJSON(v)
return c.enqueue(websocket.TextMessage, data)
}
// SendBinary sends a binary message to the relay
func (c *Client) SendBinary(data []byte) error {
c.mu.Lock()
defer c.mu.Unlock()
if c.conn == nil {
return fmt.Errorf("not connected")
}
return c.conn.WriteMessage(websocket.BinaryMessage, data)
return c.enqueue(websocket.BinaryMessage, data)
}
// SendTunnelData sends tunnel data with the tunnel ID prefix
@@ -296,10 +575,13 @@ func (c *Client) handleTextMessage(data []byte) {
}
case protocol.MsgClientLeft:
if c.OnClientLeft != nil {
var msg protocol.ClientLeft
if json.Unmarshal(data, &msg) == nil {
c.OnClientLeft(&msg)
var msg protocol.ClientLeft
if json.Unmarshal(data, &msg) == nil {
c.handlerMu.RLock()
handlers := append([]func(*protocol.ClientLeft){}, c.clientLeftHandlers...)
c.handlerMu.RUnlock()
for _, fn := range handlers {
fn(&msg)
}
}
@@ -322,7 +604,11 @@ func (c *Client) handleBinaryMessage(data []byte) {
tunnelID := string(data[:protocol.TunnelHeaderSize])
payload := data[protocol.TunnelHeaderSize:]
if c.OnTunnelData != nil {
c.OnTunnelData(tunnelID, payload)
c.handlerMu.RLock()
handlers := c.tunnelHandlers
c.handlerMu.RUnlock()
for _, fn := range handlers {
fn(tunnelID, payload)
}
}
+13
View File
@@ -0,0 +1,13 @@
package client
import "crypto/subtle"
// constantTimeEqual compares two strings without leaking their contents
// through timing. Used for peer tokens, where a byte-by-byte comparison would
// let an attacker recover the expected value one byte at a time.
func constantTimeEqual(a, b string) bool {
if len(a) != len(b) {
return false
}
return subtle.ConstantTimeCompare([]byte(a), []byte(b)) == 1
}
+305
View File
@@ -0,0 +1,305 @@
package client
import (
"encoding/binary"
"encoding/hex"
"fmt"
"io"
"net"
"sync"
"time"
"github.com/duffy/usb-server/internal/protocol"
)
// Direct tunnel wire format.
//
// Handshake, sent by the connecting (use) side:
//
// [4] magic "USBD" [1] version [16] tunnel ID [32] peer token
//
// Reply, sent by the listening (share) side:
//
// [4] magic "USBD" [1] version [1] status (0 = accepted)
//
// Everything after that is length-prefixed encrypted frames:
//
// [4] length (big endian) [length bytes] sealed frame
const (
directMagic = "USBD"
directVersion = 1
handshakeSize = 4 + 1 + protocol.TunnelHeaderSize + 32
handshakeReplySize = 4 + 1 + 1
// directHandshakeTimeout bounds the handshake. A peer that reaches the
// port but does not speak this protocol must not hold the slot.
directHandshakeTimeout = 5 * time.Second
// directDialTimeout bounds one connection attempt. Candidate addresses
// are tried in parallel, so this is also how long the whole attempt takes
// before falling back to the relay.
directDialTimeout = 3 * time.Second
// maxDirectFrame caps a single frame, so a hostile or corrupt length
// prefix cannot make us allocate arbitrarily.
maxDirectFrame = 2 << 20
)
// Handshake status codes.
const (
directAccepted = 0
directUnknownTun = 1
directBadToken = 2
directWrongVerson = 3
)
// directConn carries length-prefixed frames over a plain TCP connection.
//
// It deliberately does no encryption of its own: tunnel frames are sealed one
// level up, by the tunnel's codec, so that relayed and direct tunnels get the
// same protection. Putting it here instead would leave the relay path in
// cleartext — the one path where a third party is actually in the middle.
type directConn struct {
conn net.Conn
writeMu sync.Mutex
}
func newDirectConn(conn net.Conn) *directConn {
return &directConn{conn: conn}
}
// WriteFrame sends one length-prefixed frame.
func (d *directConn) WriteFrame(payload []byte) error {
if len(payload) > maxDirectFrame {
return fmt.Errorf("frame of %d bytes exceeds the %d byte limit", len(payload), maxDirectFrame)
}
buf := make([]byte, 4+len(payload))
binary.BigEndian.PutUint32(buf, uint32(len(payload)))
copy(buf[4:], payload)
// TCP writes from several goroutines would interleave and corrupt the
// framing, so sends are serialised.
d.writeMu.Lock()
defer d.writeMu.Unlock()
if _, err := d.conn.Write(buf); err != nil {
return fmt.Errorf("writing frame: %w", err)
}
return nil
}
// ReadFrame reads one length-prefixed frame.
func (d *directConn) ReadFrame() ([]byte, error) {
var lenBuf [4]byte
if _, err := io.ReadFull(d.conn, lenBuf[:]); err != nil {
return nil, err
}
length := binary.BigEndian.Uint32(lenBuf[:])
if length == 0 || length > maxDirectFrame {
return nil, fmt.Errorf("frame length %d is out of range", length)
}
payload := make([]byte, length)
if _, err := io.ReadFull(d.conn, payload); err != nil {
return nil, err
}
return payload, nil
}
// RemoteAddr reports the peer address, for logging.
func (d *directConn) RemoteAddr() string { return d.conn.RemoteAddr().String() }
// Close closes the underlying connection.
func (d *directConn) Close() error { return d.conn.Close() }
// buildHandshake assembles the greeting the connecting side sends.
func buildHandshake(tunnelID, peerToken string) ([]byte, error) {
tokenBytes, err := hex.DecodeString(peerToken)
if err != nil || len(tokenBytes) != 32 {
return nil, fmt.Errorf("invalid peer token")
}
if len(tunnelID) != protocol.TunnelHeaderSize {
return nil, fmt.Errorf("tunnel ID is %d bytes, want %d", len(tunnelID), protocol.TunnelHeaderSize)
}
buf := make([]byte, 0, handshakeSize)
buf = append(buf, directMagic...)
buf = append(buf, directVersion)
buf = append(buf, tunnelID...)
buf = append(buf, tokenBytes...)
return buf, nil
}
// parseHandshake validates the greeting and returns the requested tunnel ID
// and the presented token in hex form.
func parseHandshake(data []byte) (tunnelID, peerToken string, err error) {
if len(data) != handshakeSize {
return "", "", fmt.Errorf("handshake is %d bytes, want %d", len(data), handshakeSize)
}
if string(data[:4]) != directMagic {
return "", "", fmt.Errorf("bad magic")
}
if data[4] != directVersion {
return "", "", fmt.Errorf("unsupported version %d", data[4])
}
tunnelID = string(data[5 : 5+protocol.TunnelHeaderSize])
peerToken = hex.EncodeToString(data[5+protocol.TunnelHeaderSize:])
return tunnelID, peerToken, nil
}
func buildHandshakeReply(status byte) []byte {
buf := make([]byte, 0, handshakeReplySize)
buf = append(buf, directMagic...)
buf = append(buf, directVersion)
buf = append(buf, status)
return buf
}
func parseHandshakeReply(data []byte) error {
if len(data) != handshakeReplySize {
return fmt.Errorf("reply is %d bytes, want %d", len(data), handshakeReplySize)
}
if string(data[:4]) != directMagic {
return fmt.Errorf("bad magic in reply")
}
if data[4] != directVersion {
return fmt.Errorf("peer speaks version %d, we speak %d", data[4], directVersion)
}
switch data[5] {
case directAccepted:
return nil
case directUnknownTun:
return fmt.Errorf("peer does not know this tunnel")
case directBadToken:
return fmt.Errorf("peer rejected our token")
case directWrongVerson:
return fmt.Errorf("peer rejected our version")
default:
return fmt.Errorf("peer rejected the connection (status %d)", data[5])
}
}
// localEndpoints lists host:port addresses on this machine's own interfaces.
//
// Loopback is skipped — a peer on another machine cannot use it — but every
// other usable unicast address is offered, because which one is reachable
// depends on the network and only the attempt can tell.
func localEndpoints(port int) []string {
if port == 0 {
return nil
}
addrs, err := net.InterfaceAddrs()
if err != nil {
return nil
}
var endpoints []string
for _, addr := range addrs {
ipNet, ok := addr.(*net.IPNet)
if !ok {
continue
}
ip := ipNet.IP
if ip.IsLoopback() || ip.IsUnspecified() || !ip.IsGlobalUnicast() {
continue
}
// Link-local IPv6 needs a zone to be dialable and rarely helps here.
if ip.To4() == nil && ip.IsLinkLocalUnicast() {
continue
}
endpoints = append(endpoints, net.JoinHostPort(ip.String(), fmt.Sprint(port)))
}
return endpoints
}
// dialDirect races the candidate addresses and returns the first connection
// that completes the handshake.
//
// Racing rather than trying in sequence matters: an unreachable address on a
// different subnet typically does not refuse the connection, it hangs until
// the timeout, and trying those one after another would take longer than the
// relay fallback it is meant to avoid.
func dialDirect(endpoints []string, tunnelID, peerToken string) (*directConn, string, error) {
if len(endpoints) == 0 {
return nil, "", fmt.Errorf("no candidate addresses")
}
greeting, err := buildHandshake(tunnelID, peerToken)
if err != nil {
return nil, "", err
}
type result struct {
conn *directConn
addr string
err error
}
results := make(chan result, len(endpoints))
for _, endpoint := range endpoints {
go func(addr string) {
conn, err := attemptDirect(addr, greeting)
results <- result{conn: conn, addr: addr, err: err}
}(endpoint)
}
var lastErr error
var winner *directConn
var winnerAddr string
// Collect every result so that a connection completing after we already
// have a winner still gets closed instead of leaking.
for range endpoints {
r := <-results
switch {
case r.err != nil:
lastErr = r.err
case winner == nil:
winner, winnerAddr = r.conn, r.addr
default:
r.conn.Close()
}
}
if winner == nil {
return nil, "", fmt.Errorf("no address reachable: %w", lastErr)
}
return winner, winnerAddr, nil
}
// attemptDirect performs one dial plus handshake.
func attemptDirect(addr string, greeting []byte) (*directConn, error) {
conn, err := net.DialTimeout("tcp", addr, directDialTimeout)
if err != nil {
return nil, err
}
conn.SetDeadline(time.Now().Add(directHandshakeTimeout))
if _, err := conn.Write(greeting); err != nil {
conn.Close()
return nil, fmt.Errorf("sending handshake to %s: %w", addr, err)
}
reply := make([]byte, handshakeReplySize)
if _, err := io.ReadFull(conn, reply); err != nil {
conn.Close()
return nil, fmt.Errorf("reading handshake reply from %s: %w", addr, err)
}
if err := parseHandshakeReply(reply); err != nil {
conn.Close()
return nil, fmt.Errorf("handshake with %s: %w", addr, err)
}
// Clear the handshake deadline; tunnel traffic has no fixed timing.
conn.SetDeadline(time.Time{})
return newDirectConn(conn), nil
}
+479
View File
@@ -0,0 +1,479 @@
package client
import (
"bytes"
"net"
"strconv"
"testing"
"time"
"github.com/duffy/usb-server/internal/crypto"
"github.com/duffy/usb-server/internal/protocol"
)
const (
testTok1 = "111111111111111111111111111111111111111111="
testTok2 = "222222222222222222222222222222222222222222="
testTok3 = "333333333333333333333333333333333333333333="
)
func testSecret(t *testing.T) *crypto.TunnelSecret {
t.Helper()
s, err := crypto.DeriveTunnelSecret(testTok1, testTok2, testTok3)
if err != nil {
t.Fatalf("DeriveTunnelSecret: %v", err)
}
return s
}
const testTunnelID = "0123456789abcdef" // exactly TunnelHeaderSize
func TestHandshakeRoundTrip(t *testing.T) {
s := testSecret(t)
token := s.PeerToken(testTunnelID)
greeting, err := buildHandshake(testTunnelID, token)
if err != nil {
t.Fatalf("buildHandshake: %v", err)
}
if len(greeting) != handshakeSize {
t.Fatalf("greeting is %d bytes, want %d", len(greeting), handshakeSize)
}
gotID, gotToken, err := parseHandshake(greeting)
if err != nil {
t.Fatalf("parseHandshake: %v", err)
}
if gotID != testTunnelID {
t.Errorf("tunnel ID = %q, want %q", gotID, testTunnelID)
}
if gotToken != token {
t.Errorf("token = %q, want %q", gotToken, token)
}
}
func TestParseHandshakeRejectsMalformed(t *testing.T) {
s := testSecret(t)
valid, _ := buildHandshake(testTunnelID, s.PeerToken(testTunnelID))
tests := []struct {
name string
data []byte
}{
{"empty", nil},
{"truncated", valid[:handshakeSize-1]},
{"too long", append(append([]byte{}, valid...), 0x00)},
{"bad magic", func() []byte {
b := append([]byte{}, valid...)
b[0] = 'X'
return b
}()},
{"unsupported version", func() []byte {
b := append([]byte{}, valid...)
b[4] = 99
return b
}()},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if _, _, err := parseHandshake(tt.data); err == nil {
t.Error("malformed handshake was accepted")
}
})
}
}
func TestHandshakeReplyStatuses(t *testing.T) {
if err := parseHandshakeReply(buildHandshakeReply(directAccepted)); err != nil {
t.Errorf("accepted reply reported an error: %v", err)
}
for _, status := range []byte{directUnknownTun, directBadToken, directWrongVerson, 99} {
if err := parseHandshakeReply(buildHandshakeReply(status)); err == nil {
t.Errorf("status %d was treated as success", status)
}
}
}
func TestBuildHandshakeRejectsBadInput(t *testing.T) {
s := testSecret(t)
good := s.PeerToken(testTunnelID)
if _, err := buildHandshake("short", good); err == nil {
t.Error("a wrong-length tunnel ID was accepted")
}
if _, err := buildHandshake(testTunnelID, "not-hex"); err == nil {
t.Error("a non-hex token was accepted")
}
if _, err := buildHandshake(testTunnelID, "abcd"); err == nil {
t.Error("a short token was accepted")
}
}
// A listener must only hand over connections whose peer proves group
// membership. The relay knows tunnel IDs, so the token is what stops it — or
// anyone else who reaches the port — from taking a device over.
func TestListenerAcceptsOnlyValidToken(t *testing.T) {
s := testSecret(t)
dl, err := newDirectListener(0, s)
if err != nil {
t.Fatalf("newDirectListener: %v", err)
}
defer dl.Close()
accepted, err := dl.Expect(testTunnelID)
if err != nil {
t.Fatalf("Expect: %v", err)
}
addr := net.JoinHostPort("127.0.0.1", strconv.Itoa(dl.Port()))
t.Run("wrong token is rejected", func(t *testing.T) {
other, _ := crypto.DeriveTunnelSecret(testTok1, testTok2, "different")
greeting, _ := buildHandshake(testTunnelID, other.PeerToken(testTunnelID))
conn, err := net.DialTimeout("tcp", addr, time.Second)
if err != nil {
t.Fatalf("dial: %v", err)
}
defer conn.Close()
conn.Write(greeting)
reply := make([]byte, handshakeReplySize)
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
if _, err := readFull(conn, reply); err != nil {
t.Fatalf("reading reply: %v", err)
}
if err := parseHandshakeReply(reply); err == nil {
t.Fatal("listener accepted a connection with the wrong token")
}
})
t.Run("unknown tunnel is rejected", func(t *testing.T) {
greeting, _ := buildHandshake("fedcba9876543210", s.PeerToken("fedcba9876543210"))
conn, err := net.DialTimeout("tcp", addr, time.Second)
if err != nil {
t.Fatalf("dial: %v", err)
}
defer conn.Close()
conn.Write(greeting)
reply := make([]byte, handshakeReplySize)
conn.SetReadDeadline(time.Now().Add(2 * time.Second))
readFull(conn, reply)
if err := parseHandshakeReply(reply); err == nil {
t.Fatal("listener accepted a connection for an unregistered tunnel")
}
})
t.Run("valid token is accepted", func(t *testing.T) {
conn, _, err := dialDirect([]string{addr}, testTunnelID, s.PeerToken(testTunnelID))
if err != nil {
t.Fatalf("dialDirect: %v", err)
}
defer conn.Close()
select {
case got := <-accepted:
if got == nil {
t.Fatal("listener delivered a nil connection")
}
got.Close()
case <-time.After(2 * time.Second):
t.Fatal("listener never delivered the accepted connection")
}
})
}
// End-to-end over a real socket pair: the two ends must agree on framing and
// on which direction each encrypts in.
func TestDirectTunnelCarriesTrafficBothWays(t *testing.T) {
s := testSecret(t)
dl, err := newDirectListener(0, s)
if err != nil {
t.Fatalf("newDirectListener: %v", err)
}
defer dl.Close()
accepted, err := dl.Expect(testTunnelID)
if err != nil {
t.Fatalf("Expect: %v", err)
}
addr := net.JoinHostPort("127.0.0.1", strconv.Itoa(dl.Port()))
useConn, _, err := dialDirect([]string{addr}, testTunnelID, s.PeerToken(testTunnelID))
if err != nil {
t.Fatalf("dialDirect: %v", err)
}
defer useConn.Close()
var shareConn *directConn
select {
case shareConn = <-accepted:
case <-time.After(2 * time.Second):
t.Fatal("listener never delivered the connection")
}
defer shareConn.Close()
shareCodec, err := newTunnelCodec(s, testTunnelID, crypto.DirShareToUse)
if err != nil {
t.Fatalf("share codec: %v", err)
}
useCodec, err := newTunnelCodec(s, testTunnelID, crypto.DirUseToShare)
if err != nil {
t.Fatalf("use codec: %v", err)
}
// use -> share
want := []byte("USBIP CMD_SUBMIT payload")
if err := send(useCodec, directSender(useConn), want); err != nil {
t.Fatalf("sending use->share: %v", err)
}
frame, err := shareConn.ReadFrame()
if err != nil {
t.Fatalf("share reading frame: %v", err)
}
got, err := shareCodec.decode(frame)
if err != nil {
t.Fatalf("share decoding frame: %v", err)
}
if !bytes.Equal(got, want) {
t.Errorf("share received %q, want %q", got, want)
}
// share -> use
want2 := []byte("USBIP RET_SUBMIT payload")
if err := send(shareCodec, directSender(shareConn), want2); err != nil {
t.Fatalf("sending share->use: %v", err)
}
frame2, err := useConn.ReadFrame()
if err != nil {
t.Fatalf("use reading frame: %v", err)
}
got2, err := useCodec.decode(frame2)
if err != nil {
t.Fatalf("use decoding frame: %v", err)
}
if !bytes.Equal(got2, want2) {
t.Errorf("use received %q, want %q", got2, want2)
}
}
func TestDirectConnFramingPreservesBoundaries(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
sender := newDirectConn(client)
receiver := newDirectConn(server)
payloads := [][]byte{
[]byte("a"),
bytes.Repeat([]byte("x"), 1000),
[]byte("last one"),
}
go func() {
for _, p := range payloads {
if err := sender.WriteFrame(p); err != nil {
return
}
}
}()
for i, want := range payloads {
got, err := receiver.ReadFrame()
if err != nil {
t.Fatalf("frame %d: %v", i, err)
}
if !bytes.Equal(got, want) {
t.Errorf("frame %d is %d bytes, want %d", i, len(got), len(want))
}
}
}
func TestDirectConnRejectsOversizedLength(t *testing.T) {
client, server := net.Pipe()
defer client.Close()
defer server.Close()
go func() {
// A length prefix claiming far more than the cap must be refused
// before anything is allocated.
client.Write([]byte{0xFF, 0xFF, 0xFF, 0xFF})
}()
receiver := newDirectConn(server)
if _, err := receiver.ReadFrame(); err == nil {
t.Error("an oversized frame length was accepted")
}
}
func TestDialDirectFailsWithoutReachableAddress(t *testing.T) {
s := testSecret(t)
// Port 1 on loopback refuses immediately, so this stays fast.
_, _, err := dialDirect([]string{"127.0.0.1:1"}, testTunnelID, s.PeerToken(testTunnelID))
if err == nil {
t.Fatal("dialDirect succeeded against a closed port")
}
if _, _, err := dialDirect(nil, testTunnelID, s.PeerToken(testTunnelID)); err == nil {
t.Error("dialDirect succeeded with no candidate addresses")
}
}
// The dialer races candidates; an unreachable one alongside a good one must
// not stop the good one from winning.
func TestDialDirectPicksTheReachableAddress(t *testing.T) {
s := testSecret(t)
dl, err := newDirectListener(0, s)
if err != nil {
t.Fatalf("newDirectListener: %v", err)
}
defer dl.Close()
accepted, _ := dl.Expect(testTunnelID)
good := net.JoinHostPort("127.0.0.1", strconv.Itoa(dl.Port()))
conn, addr, err := dialDirect(
[]string{"127.0.0.1:1", good, "127.0.0.1:2"},
testTunnelID, s.PeerToken(testTunnelID))
if err != nil {
t.Fatalf("dialDirect: %v", err)
}
defer conn.Close()
if addr != good {
t.Errorf("connected to %s, want %s", addr, good)
}
select {
case c := <-accepted:
c.Close()
case <-time.After(2 * time.Second):
t.Error("listener never saw the connection")
}
}
func TestLocalEndpointsExcludeLoopback(t *testing.T) {
if got := localEndpoints(0); got != nil {
t.Errorf("localEndpoints(0) = %v, want nil — port 0 means no listener", got)
}
for _, ep := range localEndpoints(9000) {
host, port, err := net.SplitHostPort(ep)
if err != nil {
t.Errorf("endpoint %q is not host:port: %v", ep, err)
continue
}
if port != "9000" {
t.Errorf("endpoint %q has port %q, want 9000", ep, port)
}
ip := net.ParseIP(host)
if ip == nil {
t.Errorf("endpoint %q has an unparseable host", ep)
continue
}
if ip.IsLoopback() {
t.Errorf("endpoint %q is loopback; a peer cannot reach that", ep)
}
}
}
func TestTunnelCodecNilPassesThrough(t *testing.T) {
var codec *tunnelCodec
if codec.encrypted() {
t.Error("a nil codec reported itself as encrypted")
}
payload := []byte("cleartext")
encoded, err := codec.encode(payload)
if err != nil {
t.Fatalf("encode: %v", err)
}
if !bytes.Equal(encoded, payload) {
t.Error("a nil codec altered the payload")
}
decoded, err := codec.decode(encoded)
if err != nil {
t.Fatalf("decode: %v", err)
}
if !bytes.Equal(decoded, payload) {
t.Error("round trip through a nil codec changed the payload")
}
}
func TestTunnelCodecEncryptsWhenSecretPresent(t *testing.T) {
s := testSecret(t)
codec, err := newTunnelCodec(s, testTunnelID, crypto.DirShareToUse)
if err != nil {
t.Fatalf("newTunnelCodec: %v", err)
}
if !codec.encrypted() {
t.Fatal("codec with a secret reported itself as unencrypted")
}
payload := []byte("this must not appear on the wire")
encoded, err := codec.encode(payload)
if err != nil {
t.Fatalf("encode: %v", err)
}
if bytes.Contains(encoded, payload) {
t.Error("the encoded frame contains its plaintext")
}
peer, _ := newTunnelCodec(s, testTunnelID, crypto.DirUseToShare)
decoded, err := peer.decode(encoded)
if err != nil {
t.Fatalf("peer decode: %v", err)
}
if !bytes.Equal(decoded, payload) {
t.Errorf("peer decoded %q, want %q", decoded, payload)
}
}
func TestConstantTimeEqual(t *testing.T) {
if !constantTimeEqual("abc", "abc") {
t.Error("equal strings compared unequal")
}
if constantTimeEqual("abc", "abd") {
t.Error("different strings compared equal")
}
if constantTimeEqual("abc", "abcd") {
t.Error("strings of different length compared equal")
}
if !constantTimeEqual("", "") {
t.Error("empty strings compared unequal")
}
}
// Guards the assumption baked into the wire format.
func TestTunnelIDFitsHandshake(t *testing.T) {
if protocol.TunnelHeaderSize != 16 {
t.Fatalf("TunnelHeaderSize is %d; the handshake layout assumes 16", protocol.TunnelHeaderSize)
}
if len(testTunnelID) != protocol.TunnelHeaderSize {
t.Fatalf("test tunnel ID is %d bytes, want %d", len(testTunnelID), protocol.TunnelHeaderSize)
}
}
// --- helpers ---
func readFull(conn net.Conn, buf []byte) (int, error) {
total := 0
for total < len(buf) {
n, err := conn.Read(buf[total:])
total += n
if err != nil {
return total, err
}
}
return total, nil
}
+187
View File
@@ -0,0 +1,187 @@
package client
import (
"fmt"
"io"
"log"
"net"
"sync"
"time"
"github.com/duffy/usb-server/internal/crypto"
)
// directListener accepts incoming direct tunnel connections.
//
// Only the share side listens: the use side is the one that knows a tunnel
// has been granted, so it makes the call. Tunnels are registered here as they
// are granted, and an incoming connection is matched against them.
type directListener struct {
listener net.Listener
secret *crypto.TunnelSecret
mu sync.Mutex
expected map[string]*expectedTunnel // tunnel ID -> pending acceptance
closed bool
}
// expectedTunnel is a granted tunnel waiting for its peer to connect.
type expectedTunnel struct {
token string
accepted chan *directConn
}
// newDirectListener starts listening on the given port.
// Port 0 picks a free one, which is the sensible default: the actual port is
// advertised to peers, so it does not need to be predictable.
func newDirectListener(port int, secret *crypto.TunnelSecret) (*directListener, error) {
if secret == nil {
return nil, fmt.Errorf("direct connections require the tunnel secret")
}
ln, err := net.Listen("tcp", fmt.Sprintf(":%d", port))
if err != nil {
return nil, fmt.Errorf("listening for direct connections: %w", err)
}
dl := &directListener{
listener: ln,
secret: secret,
expected: make(map[string]*expectedTunnel),
}
go dl.acceptLoop()
log.Printf("[direct] listening on %s", ln.Addr())
return dl, nil
}
// Port returns the port actually bound.
func (dl *directListener) Port() int {
if addr, ok := dl.listener.Addr().(*net.TCPAddr); ok {
return addr.Port
}
return 0
}
// Expect registers a granted tunnel and returns a channel that receives the
// connection once a peer completes the handshake for it.
func (dl *directListener) Expect(tunnelID string) (<-chan *directConn, error) {
accepted := make(chan *directConn, 1)
dl.mu.Lock()
defer dl.mu.Unlock()
if dl.closed {
return nil, fmt.Errorf("listener is closed")
}
dl.expected[tunnelID] = &expectedTunnel{
token: dl.secret.PeerToken(tunnelID),
accepted: accepted,
}
return accepted, nil
}
// Forget drops a tunnel, whether it was taken over directly or fell back to
// the relay. Leaving entries behind would let a peer connect to a tunnel that
// is no longer live.
func (dl *directListener) Forget(tunnelID string) {
dl.mu.Lock()
defer dl.mu.Unlock()
delete(dl.expected, tunnelID)
}
// Close stops accepting connections.
func (dl *directListener) Close() error {
dl.mu.Lock()
dl.closed = true
dl.expected = make(map[string]*expectedTunnel)
dl.mu.Unlock()
return dl.listener.Close()
}
func (dl *directListener) acceptLoop() {
for {
conn, err := dl.listener.Accept()
if err != nil {
dl.mu.Lock()
closed := dl.closed
dl.mu.Unlock()
if closed {
return
}
// A transient accept error should not kill the listener, but it
// must not spin either.
log.Printf("[direct] accept error: %v", err)
time.Sleep(100 * time.Millisecond)
continue
}
go dl.handleIncoming(conn)
}
}
// handleIncoming validates one incoming connection's handshake.
func (dl *directListener) handleIncoming(conn net.Conn) {
conn.SetDeadline(time.Now().Add(directHandshakeTimeout))
greeting := make([]byte, handshakeSize)
if _, err := io.ReadFull(conn, greeting); err != nil {
conn.Close()
return
}
tunnelID, presented, err := parseHandshake(greeting)
if err != nil {
log.Printf("[direct] rejecting %s: %v", conn.RemoteAddr(), err)
conn.Write(buildHandshakeReply(directWrongVerson))
conn.Close()
return
}
dl.mu.Lock()
tunnel, known := dl.expected[tunnelID]
dl.mu.Unlock()
if !known {
conn.Write(buildHandshakeReply(directUnknownTun))
conn.Close()
return
}
// The token proves group membership. The relay knows the tunnel ID — it
// routed the grant — but cannot derive this, so it cannot impersonate a
// peer, and neither can anything else that merely reaches the port.
if !constantTimeEqual(presented, tunnel.token) {
log.Printf("[direct] rejecting %s: bad token for tunnel", conn.RemoteAddr())
conn.Write(buildHandshakeReply(directBadToken))
conn.Close()
return
}
if _, err := conn.Write(buildHandshakeReply(directAccepted)); err != nil {
conn.Close()
return
}
conn.SetDeadline(time.Time{})
direct := newDirectConn(conn)
// Claim the tunnel: whoever handshakes first wins, and a second connection
// for the same tunnel is dropped rather than replacing a live one.
dl.mu.Lock()
current, still := dl.expected[tunnelID]
if still && current == tunnel {
delete(dl.expected, tunnelID)
} else {
still = false
}
dl.mu.Unlock()
if !still {
direct.Close()
return
}
log.Printf("[direct] accepted connection from %s for tunnel %s", conn.RemoteAddr(), tunnelID)
tunnel.accepted <- direct
}
+264 -31
View File
@@ -8,6 +8,7 @@ import (
"time"
"github.com/duffy/usb-server/internal/config"
"github.com/duffy/usb-server/internal/crypto"
"github.com/duffy/usb-server/internal/protocol"
"github.com/duffy/usb-server/internal/usb"
"github.com/duffy/usb-server/internal/usbip"
@@ -22,21 +23,81 @@ type ShareManager struct {
devices []usb.Device
active map[string]*activeShare // busID -> active share
tunnels map[string]*shareTunnel // tunnelID -> tunnel
// secret derives per-tunnel keys. Nil when the client is configured with
// only a group hash, in which case tunnels stay unencrypted.
secret *crypto.TunnelSecret
// listener accepts direct connections from peers. Nil when direct
// connections are disabled or could not be set up, which just means every
// tunnel goes through the relay.
listener *directListener
}
type activeShare struct {
device *usb.Device
server *usbip.Server
usedBy string // client ID using this device
tunnelID string
device *usb.Device
server *usbip.Server
usedBy string // client ID using this device
tunnelID string
}
type shareTunnel struct {
id string
busID string
inPipe *io.PipeWriter
outPipe *io.PipeReader
done chan struct{}
id string
busID string
// in carries peer -> USB/IP server bytes. It buffers instead of blocking
// so that feeding it from the WebSocket read loop cannot stall the client.
in *streamBuffer
// out carries USB/IP server -> peer bytes. This direction stays a pipe:
// blocking there is real backpressure onto the USB reap loop, which is
// what we want when the network cannot keep up.
outPipe *io.PipeReader
done chan struct{}
// codec seals outgoing and opens incoming payloads, whichever transport
// carries them.
codec *tunnelCodec
// sendMu guards swapping the transport when a direct connection takes
// over from the relay mid-tunnel.
sendMu sync.Mutex
send tunnelSender
direct *directConn
}
// setTransport switches this tunnel to a new sender, closing the old direct
// connection if there was one.
func (t *shareTunnel) setTransport(sender tunnelSender, conn *directConn) {
t.sendMu.Lock()
defer t.sendMu.Unlock()
if t.direct != nil && t.direct != conn {
t.direct.Close()
}
t.send = sender
t.direct = conn
}
// deliver encodes and transmits one payload over the current transport.
func (t *shareTunnel) deliver(payload []byte) error {
t.sendMu.Lock()
sender := t.send
t.sendMu.Unlock()
if sender == nil {
return fmt.Errorf("tunnel %s has no transport", t.id)
}
return send(t.codec, sender, payload)
}
// closeDirect tears down any direct connection this tunnel holds.
func (t *shareTunnel) closeDirect() {
t.sendMu.Lock()
defer t.sendMu.Unlock()
if t.direct != nil {
t.direct.Close()
t.direct = nil
}
}
// NewShareManager creates a share manager
@@ -46,18 +107,118 @@ func NewShareManager(client *Client, cfg *config.Config) *ShareManager {
cfg: cfg,
active: make(map[string]*activeShare),
tunnels: make(map[string]*shareTunnel),
secret: client.TunnelSecret(),
}
// Set up callbacks
sm.startDirectListener()
// Share-side messages only this manager handles.
client.OnRequestDevice = sm.handleRequestDevice
client.OnReleaseDevice = sm.handleReleaseDevice
client.OnTunnelData = sm.handleTunnelData
client.OnClientLeft = sm.handleClientLeft
client.OnForceRelease = sm.handleForceRelease
// Shared with the use manager in "both" mode, hence multicast.
client.AddTunnelHandler(sm.handleTunnelData)
client.AddClientLeftHandler(sm.handleClientLeft)
client.AddDisconnectHandler(sm.handleRelayDisconnect)
return sm
}
// startDirectListener opens the port peers connect to for direct tunnels.
//
// Failure is never fatal: without a listener every tunnel simply goes through
// the relay, which is exactly how the system worked before.
func (sm *ShareManager) startDirectListener() {
if sm.cfg.DisableDirect {
log.Printf("[share] direct connections disabled by configuration")
return
}
if sm.secret == nil {
log.Printf("[share] direct connections unavailable: no tokens configured, only a group hash")
return
}
listener, err := newDirectListener(sm.cfg.DirectPort, sm.secret)
if err != nil {
log.Printf("[share] direct connections unavailable: %v (falling back to relay)", err)
return
}
sm.listener = listener
sm.client.SetDirectPort(listener.Port())
}
// DirectPort reports the port peers can reach for direct tunnels, 0 if none.
func (sm *ShareManager) DirectPort() int {
if sm.listener == nil {
return 0
}
return sm.listener.Port()
}
// awaitDirect waits for the peer to connect directly and, when it does, moves
// the tunnel off the relay.
//
// The switch is safe at any moment because USB/IP is a stream of complete
// messages and each tunnel frame carries one chunk of it: frames sent before
// the switch travel via the relay, frames after it travel directly, and both
// arrive in order at the same reader. Nothing is in flight in pieces.
func (sm *ShareManager) awaitDirect(tunnel *shareTunnel, accepted <-chan *directConn) {
select {
case conn := <-accepted:
if conn == nil {
return
}
select {
case <-tunnel.done:
conn.Close()
return
default:
}
log.Printf("[share] tunnel %s now direct with %s, bypassing the relay",
tunnel.id, conn.RemoteAddr())
tunnel.setTransport(directSender(conn), conn)
// Incoming frames now arrive on this connection instead of the relay.
receiveLoop(conn, tunnel.codec, func(payload []byte) error {
_, err := tunnel.in.Write(payload)
return err
}, tunnel.done, "share/"+tunnel.busID)
// The direct connection ended. The USB/IP stream cannot resume on the
// relay mid-conversation — the peer's VHCI has torn down its side —
// so release the device and let it be requested again.
select {
case <-tunnel.done:
default:
log.Printf("[share] direct connection for %s ended, releasing device", tunnel.busID)
go sm.handleReleaseDevice(tunnel.busID, "")
}
case <-tunnel.done:
}
}
// handleRelayDisconnect releases every active share after the relay link
// drops. The relay discarded those tunnels, so the remote side is gone and
// the local device would otherwise stay claimed and unusable.
func (sm *ShareManager) handleRelayDisconnect() {
sm.mu.RLock()
busIDs := make([]string, 0, len(sm.active))
for busID := range sm.active {
busIDs = append(busIDs, busID)
}
sm.mu.RUnlock()
for _, busID := range busIDs {
log.Printf("[share] releasing %s (relay connection lost)", busID)
sm.handleReleaseDevice(busID, "")
}
}
// Run starts the share manager: periodic device enumeration + event handling
func (sm *ShareManager) Run() error {
// Initial enumeration
@@ -89,6 +250,14 @@ func (sm *ShareManager) GetDevices() []usb.Device {
return result
}
// RefreshNow re-enumerates and announces immediately, rather than waiting for
// the next poll. Used when devices appear through the bridge, where the
// change is known the instant it happens.
func (sm *ShareManager) RefreshNow() {
sm.refreshDevices()
sm.broadcastDeviceList()
}
func (sm *ShareManager) refreshDevices() {
devices, err := usb.Enumerate()
if err != nil {
@@ -202,15 +371,44 @@ func (sm *ShareManager) handleRequestDevice(targetClient, fromClient, busID, req
tunnelID += "0"
}
inReader, inWriter := io.Pipe()
codec, err := newTunnelCodec(sm.secret, tunnelID, crypto.DirShareToUse)
if err != nil {
sm.mu.Unlock()
server.Detach()
log.Printf("[share] failed to set up tunnel encryption for %s: %v", busID, err)
sm.client.SendJSON(map[string]interface{}{
"type": protocol.MsgDeviceDenied,
"bus_id": busID,
"request_id": requestID,
"reason": fmt.Sprintf("tunnel setup failed: %v", err),
"target_client": fromClient,
})
return
}
in := newStreamBuffer()
outReader, outWriter := io.Pipe()
tunnel := &shareTunnel{
id: tunnelID,
busID: busID,
inPipe: inWriter,
in: in,
outPipe: outReader,
done: make(chan struct{}),
codec: codec,
}
// Start on the relay. If the peer reaches us directly, the transport is
// swapped underneath without the USB/IP layer noticing.
tunnel.setTransport(relaySender(sm.client, tunnelID), nil)
// Register the tunnel before announcing it, so a peer that connects
// immediately after receiving the grant is not rejected as unknown.
var accepted <-chan *directConn
if sm.listener != nil {
var err error
if accepted, err = sm.listener.Expect(tunnelID); err != nil {
log.Printf("[share] cannot expect a direct connection for %s: %v", busID, err)
}
}
share := &activeShare{
@@ -224,23 +422,26 @@ func (sm *ShareManager) handleRequestDevice(targetClient, fromClient, busID, req
sm.tunnels[tunnelID] = tunnel
sm.mu.Unlock()
// Start USB/IP protocol handler in background
// Start USB/IP protocol handler in background.
// The tunnel carries the USB/IP transfer phase directly: on Linux the
// use side hands the socket straight to VHCI, and on Windows usbip.exe's
// management phase is answered locally, so there is no import request here.
go func() {
defer func() {
close(tunnel.done)
inWriter.Close()
in.Close()
outWriter.Close()
outReader.Close()
}()
// First handle the management phase (import request from client)
// The USB/IP client will send OP_REQ_IMPORT, we respond, then enter transfer phase
err := server.HandleConnection(inReader, outWriter)
err := server.HandleConnection(in, outWriter)
if err != nil {
log.Printf("[share] USB/IP connection error for %s: %v", busID, err)
}
}()
// Forward outgoing data from USB/IP server to tunnel
// Forward outgoing data from the USB/IP server over whichever transport
// the tunnel currently uses.
go func() {
buf := make([]byte, 65536)
for {
@@ -248,13 +449,20 @@ func (sm *ShareManager) handleRequestDevice(targetClient, fromClient, busID, req
if err != nil {
return
}
if err := sm.client.SendTunnelData(tunnelID, buf[:n]); err != nil {
if err := tunnel.deliver(buf[:n]); err != nil {
log.Printf("[share] tunnel %s send failed: %v", tunnelID, err)
return
}
}
}()
// Send grant message
// Wait in the background for the peer to connect directly.
if accepted != nil {
go sm.awaitDirect(tunnel, accepted)
}
// Send grant message, including where we can be reached directly. The
// relay adds the public address it sees before passing this on.
sm.client.SendJSON(map[string]interface{}{
"type": protocol.MsgDeviceGranted,
"bus_id": busID,
@@ -263,9 +471,12 @@ func (sm *ShareManager) handleRequestDevice(targetClient, fromClient, busID, req
"dev_id": dev.DevID(),
"speed": dev.Speed,
"target_client": fromClient,
"endpoints": localEndpoints(sm.DirectPort()),
"encrypted": codec.encrypted(),
})
log.Printf("[share] device %s granted to %s (tunnel=%s)", busID, fromClient, tunnelID)
log.Printf("[share] device %s granted to %s (tunnel=%s, encrypted=%v)",
busID, fromClient, tunnelID, codec.encrypted())
// Broadcast updated device list
sm.refreshDevices()
@@ -282,13 +493,17 @@ func (sm *ShareManager) handleReleaseDevice(busID, fromClient string) {
return
}
// Close the tunnel pipe to signal HandleConnection to stop reading
// Close the tunnel input to signal HandleConnection to stop reading
var tunnelDone <-chan struct{}
if tunnel, ok := sm.tunnels[share.tunnelID]; ok {
tunnel.inPipe.Close()
tunnel.in.Close()
tunnel.closeDirect()
tunnelDone = tunnel.done
delete(sm.tunnels, share.tunnelID)
}
if sm.listener != nil {
sm.listener.Forget(share.tunnelID)
}
server := share.server
delete(sm.active, busID)
@@ -332,7 +547,7 @@ func (sm *ShareManager) handleForceRelease(targetClient, fromClient, busID strin
return
}
log.Printf("[share] force-releasing %s (requested by %s, was used by %s)", busID, fromClient[:8], share.usedBy[:8])
log.Printf("[share] force-releasing %s (requested by %s, was used by %s)", busID, protocol.ShortID(fromClient), protocol.ShortID(share.usedBy))
sm.handleReleaseDevice(busID, share.usedBy)
}
@@ -347,7 +562,7 @@ func (sm *ShareManager) handleClientLeft(msg *protocol.ClientLeft) {
sm.mu.RUnlock()
for _, busID := range toRelease {
log.Printf("[share] auto-releasing %s (client %s left)", busID, msg.ClientID[:8])
log.Printf("[share] auto-releasing %s (client %s left)", busID, protocol.ShortID(msg.ClientID))
sm.handleReleaseDevice(busID, msg.ClientID)
}
}
@@ -358,11 +573,24 @@ func (sm *ShareManager) handleTunnelData(tunnelID string, data []byte) {
sm.mu.RUnlock()
if !exists {
// In "both" mode the use manager owns the other tunnels and sees the
// same frames, so an unknown ID here is normal.
return
}
// Write incoming data to the USB/IP server's input pipe
tunnel.inPipe.Write(data)
payload, err := tunnel.codec.decode(data)
if err != nil {
log.Printf("[share] tunnel %s: rejecting relayed frame: %v", tunnelID, err)
go sm.handleReleaseDevice(tunnel.busID, "")
return
}
// Buffered, non-blocking: this runs on the WebSocket read loop, which
// must never stall on the USB side.
if _, err := tunnel.in.Write(payload); err != nil {
log.Printf("[share] tunnel %s input failed: %v — releasing %s", tunnelID, err, tunnel.busID)
go sm.handleReleaseDevice(tunnel.busID, "")
}
}
func (sm *ShareManager) cleanup() {
@@ -371,7 +599,8 @@ func (sm *ShareManager) cleanup() {
for busID, share := range sm.active {
if tunnel, ok := sm.tunnels[share.tunnelID]; ok {
tunnel.inPipe.Close()
tunnel.in.Close()
tunnel.closeDirect()
}
share.server.Detach()
log.Printf("[share] cleaned up device %s", busID)
@@ -379,6 +608,11 @@ func (sm *ShareManager) cleanup() {
sm.active = make(map[string]*activeShare)
sm.tunnels = make(map[string]*shareTunnel)
if sm.listener != nil {
sm.listener.Close()
sm.listener = nil
}
}
// DeviceListForAPI returns device info formatted for the web API
@@ -407,4 +641,3 @@ func (sm *ShareManager) DeviceListForAPI() []map[string]interface{} {
}
return result
}
+31
View File
@@ -0,0 +1,31 @@
//go:build darwin
package client
import (
"context"
"fmt"
"net"
"os"
"github.com/duffy/usb-server/internal/protocol"
)
// Attaching a remote device needs a virtual USB host controller, which macOS
// does not provide — see internal/usbip/vhci_darwin.go.
func createVHCIAttachment(_ context.Context, _ *protocol.DeviceGranted, _ *RemoteDevice) (net.Conn, int, error) {
return nil, -1, fmt.Errorf("receiving USB devices is not supported on macOS")
}
func createSocketPair() ([2]int, error) {
return [2]int{}, fmt.Errorf("not used on macOS")
}
func closeFDs(fds [2]int) {}
func fdToFile(fd int, name string) *os.File { return nil }
func logVHCIDeviceStatus(port int) {}
func fixVHCIDevicePermissions(port int) {}
+98
View File
@@ -76,6 +76,104 @@ func fdToFile(fd int, name string) *os.File {
return os.NewFile(uintptr(fd), name)
}
// logVHCIDeviceStatus reads the VHCI sysfs tree to check what happened
// with a newly attached device. Logs driver binding, device class, etc.
//
// This is diagnostics only, so it stays behind USBSRV_DEBUG: it waits three
// seconds and then walks the whole sysfs tree on every attach.
func logVHCIDeviceStatus(port int) {
if !protocol.Debug {
return
}
time.Sleep(3 * time.Second) // wait for enumeration
basePath := "/sys/devices/platform/vhci_hcd.0"
entries, err := os.ReadDir(basePath)
if err != nil {
log.Printf("[use-diag] cannot read VHCI sysfs: %v", err)
return
}
// Find the USB device for this port (usbN/N-M pattern)
for _, entry := range entries {
if !strings.HasPrefix(entry.Name(), "usb") {
continue
}
usbPath := filepath.Join(basePath, entry.Name())
devEntries, err := os.ReadDir(usbPath)
if err != nil {
continue
}
for _, devEntry := range devEntries {
devName := devEntry.Name()
// Device dirs look like "3-1", not "3-1:1.0"
if !strings.Contains(devName, "-") || strings.Contains(devName, ":") {
continue
}
devPath := filepath.Join(usbPath, devName)
// Read device info
readAttr := func(name string) string {
data, err := os.ReadFile(filepath.Join(devPath, name))
if err != nil {
return ""
}
return strings.TrimSpace(string(data))
}
vid := readAttr("idVendor")
pid := readAttr("idProduct")
product := readAttr("product")
manufacturer := readAttr("manufacturer")
speed := readAttr("speed")
devClass := readAttr("bDeviceClass")
if vid == "" {
continue // not a real device
}
log.Printf("[use-diag] VHCI device: %s %s:%s speed=%s class=%s %s %s",
devName, vid, pid, speed, devClass, manufacturer, product)
// Check interfaces and their drivers
ifEntries, _ := os.ReadDir(devPath)
for _, ifEntry := range ifEntries {
ifName := ifEntry.Name()
if !strings.Contains(ifName, ":") {
continue
}
ifPath := filepath.Join(devPath, ifName)
ifClass, _ := os.ReadFile(filepath.Join(ifPath, "bInterfaceClass"))
ifProto, _ := os.ReadFile(filepath.Join(ifPath, "bInterfaceProtocol"))
driverLink, err := os.Readlink(filepath.Join(ifPath, "driver"))
driver := "(no driver)"
if err == nil {
driver = filepath.Base(driverLink)
}
log.Printf("[use-diag] interface %s: class=%s proto=%s driver=%s",
ifName, strings.TrimSpace(string(ifClass)), strings.TrimSpace(string(ifProto)), driver)
// Check for input devices under this interface
filepath.WalkDir(ifPath, func(path string, d os.DirEntry, err error) error {
if err != nil {
return nil
}
if strings.HasPrefix(d.Name(), "event") && strings.Contains(path, "/input/input") {
log.Printf("[use-diag] → /dev/input/%s", d.Name())
}
if strings.HasPrefix(d.Name(), "hidraw") && filepath.Base(filepath.Dir(path)) == "hidraw" {
log.Printf("[use-diag] → /dev/%s", d.Name())
}
return nil
})
}
}
}
}
// fixVHCIDevicePermissions waits for the VHCI-attached device to create
// device nodes (e.g. /dev/video*, /dev/input/event*, /dev/hidraw*) and sets
// them to world-accessible. VHCI-created devices don't get normal udev
+3
View File
@@ -199,5 +199,8 @@ func fdToFile(fd int, name string) *os.File {
return nil
}
// logVHCIDeviceStatus is Linux-only (sysfs).
func logVHCIDeviceStatus(port int) {}
// fixVHCIDevicePermissions is not needed on Windows.
func fixVHCIDevicePermissions(port int) {}
+106
View File
@@ -0,0 +1,106 @@
package client
import (
"bytes"
"errors"
"fmt"
"io"
"sync"
)
// ErrStreamOverflow is returned by streamBuffer.Read once the buffer has
// exceeded its limit. The tunnel is unusable at that point and must be torn
// down; the alternative would be growing without bound.
var ErrStreamOverflow = errors.New("tunnel buffer overflow")
// defaultStreamLimit caps how much unread tunnel data we hold.
//
// USB/IP traffic is request/response, so the consumer normally keeps up. A
// backlog this large means the USB side has stalled, and 8 MB is far more
// than any legitimate burst of in-flight URBs.
const defaultStreamLimit = 8 << 20
// streamBuffer is an unbounded-write, blocking-read byte pipe.
//
// It replaces io.Pipe on the path from the WebSocket read loop into the
// USB/IP server. io.Pipe is synchronous: a Write blocks until a Reader has
// consumed the bytes, so feeding it from the WebSocket read loop meant one
// slow USB transfer froze the entire client — no control messages, no
// keepalives, no other tunnel. Writes here never block.
type streamBuffer struct {
mu sync.Mutex
cond *sync.Cond
buf bytes.Buffer
limit int
closed bool
err error
}
func newStreamBuffer() *streamBuffer {
return newStreamBufferLimit(defaultStreamLimit)
}
func newStreamBufferLimit(limit int) *streamBuffer {
s := &streamBuffer{limit: limit}
s.cond = sync.NewCond(&s.mu)
return s
}
// Write appends data to the buffer and never blocks.
// Once the limit is exceeded the stream is failed: further reads drain what
// is already buffered and then return ErrStreamOverflow.
func (s *streamBuffer) Write(p []byte) (int, error) {
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return 0, io.ErrClosedPipe
}
if s.err != nil {
return 0, s.err
}
if s.buf.Len()+len(p) > s.limit {
s.err = fmt.Errorf("%w: %d bytes buffered, limit %d", ErrStreamOverflow, s.buf.Len(), s.limit)
s.cond.Broadcast()
return 0, s.err
}
n, err := s.buf.Write(p)
s.cond.Broadcast()
return n, err
}
// Read blocks until data is available, the stream is closed, or it failed.
func (s *streamBuffer) Read(p []byte) (int, error) {
s.mu.Lock()
defer s.mu.Unlock()
for s.buf.Len() == 0 {
if s.err != nil {
return 0, s.err
}
if s.closed {
return 0, io.EOF
}
s.cond.Wait()
}
return s.buf.Read(p)
}
// Close makes pending and future reads return EOF once the buffer is drained.
func (s *streamBuffer) Close() error {
s.mu.Lock()
defer s.mu.Unlock()
s.closed = true
s.cond.Broadcast()
return nil
}
// Buffered reports how many bytes are waiting to be read.
func (s *streamBuffer) Buffered() int {
s.mu.Lock()
defer s.mu.Unlock()
return s.buf.Len()
}
+212
View File
@@ -0,0 +1,212 @@
package client
import (
"bytes"
"errors"
"io"
"sync"
"testing"
"time"
)
func TestStreamBufferRoundTrip(t *testing.T) {
s := newStreamBuffer()
want := []byte("usbip frame")
if _, err := s.Write(want); err != nil {
t.Fatalf("Write: %v", err)
}
got := make([]byte, len(want))
if _, err := io.ReadFull(s, got); err != nil {
t.Fatalf("ReadFull: %v", err)
}
if !bytes.Equal(got, want) {
t.Errorf("read %q, want %q", got, want)
}
}
// The whole point of replacing io.Pipe: a write must return immediately even
// when nobody is reading, because it happens on the WebSocket read loop.
func TestStreamBufferWriteNeverBlocks(t *testing.T) {
s := newStreamBuffer()
done := make(chan struct{})
go func() {
defer close(done)
for i := 0; i < 100; i++ {
if _, err := s.Write(make([]byte, 1024)); err != nil {
t.Errorf("Write %d: %v", i, err)
return
}
}
}()
select {
case <-done:
case <-time.After(2 * time.Second):
t.Fatal("writes blocked with no reader — this is what froze the client")
}
if got := s.Buffered(); got != 100*1024 {
t.Errorf("buffered %d bytes, want %d", got, 100*1024)
}
}
func TestStreamBufferReadBlocksUntilData(t *testing.T) {
s := newStreamBuffer()
read := make(chan []byte, 1)
go func() {
buf := make([]byte, 4)
n, err := s.Read(buf)
if err != nil {
t.Errorf("Read: %v", err)
read <- nil
return
}
read <- buf[:n]
}()
// Give the reader time to park in Read before any data exists.
time.Sleep(50 * time.Millisecond)
select {
case <-read:
t.Fatal("Read returned before data was written")
default:
}
s.Write([]byte("ping"))
select {
case got := <-read:
if string(got) != "ping" {
t.Errorf("read %q, want %q", got, "ping")
}
case <-time.After(time.Second):
t.Fatal("Read did not wake up after Write")
}
}
func TestStreamBufferCloseGivesEOFAfterDraining(t *testing.T) {
s := newStreamBuffer()
s.Write([]byte("tail"))
s.Close()
// Buffered data must still be readable after Close.
got := make([]byte, 4)
if _, err := io.ReadFull(s, got); err != nil {
t.Fatalf("reading buffered data after Close: %v", err)
}
if string(got) != "tail" {
t.Errorf("read %q, want %q", got, "tail")
}
if _, err := s.Read(make([]byte, 4)); err != io.EOF {
t.Errorf("Read after drain = %v, want io.EOF", err)
}
}
func TestStreamBufferCloseWakesBlockedReader(t *testing.T) {
s := newStreamBuffer()
errCh := make(chan error, 1)
go func() {
_, err := s.Read(make([]byte, 4))
errCh <- err
}()
time.Sleep(50 * time.Millisecond)
s.Close()
select {
case err := <-errCh:
if err != io.EOF {
t.Errorf("blocked Read woke with %v, want io.EOF", err)
}
case <-time.After(time.Second):
t.Fatal("Close did not wake the blocked reader")
}
}
func TestStreamBufferOverflowFailsInsteadOfGrowing(t *testing.T) {
s := newStreamBufferLimit(1024)
if _, err := s.Write(make([]byte, 1000)); err != nil {
t.Fatalf("first write: %v", err)
}
if _, err := s.Write(make([]byte, 100)); !errors.Is(err, ErrStreamOverflow) {
t.Fatalf("overflowing write = %v, want ErrStreamOverflow", err)
}
// Further writes keep failing rather than silently resuming.
if _, err := s.Write([]byte("x")); !errors.Is(err, ErrStreamOverflow) {
t.Errorf("write after overflow = %v, want ErrStreamOverflow", err)
}
// Buffered data is still drainable, then the error surfaces.
if _, err := io.ReadFull(s, make([]byte, 1000)); err != nil {
t.Fatalf("draining after overflow: %v", err)
}
if _, err := s.Read(make([]byte, 4)); !errors.Is(err, ErrStreamOverflow) {
t.Errorf("Read after drain = %v, want ErrStreamOverflow", err)
}
}
func TestStreamBufferWriteAfterClose(t *testing.T) {
s := newStreamBuffer()
s.Close()
if _, err := s.Write([]byte("late")); err != io.ErrClosedPipe {
t.Errorf("Write after Close = %v, want io.ErrClosedPipe", err)
}
}
// Concurrent writers and one reader, the shape the share path actually has.
func TestStreamBufferConcurrent(t *testing.T) {
s := newStreamBuffer()
const writers = 8
const perWriter = 200
const chunk = 64
var wg sync.WaitGroup
wg.Add(writers)
for i := 0; i < writers; i++ {
go func() {
defer wg.Done()
for j := 0; j < perWriter; j++ {
if _, err := s.Write(make([]byte, chunk)); err != nil {
t.Errorf("Write: %v", err)
return
}
}
}()
}
total := writers * perWriter * chunk
readDone := make(chan int, 1)
go func() {
got := 0
buf := make([]byte, 128)
for got < total {
n, err := s.Read(buf)
if err != nil {
break
}
got += n
}
readDone <- got
}()
wg.Wait()
select {
case got := <-readDone:
if got != total {
t.Errorf("read %d bytes, want %d", got, total)
}
case <-time.After(5 * time.Second):
t.Fatal("concurrent read/write did not finish")
}
}
+130
View File
@@ -0,0 +1,130 @@
package client
import (
"fmt"
"log"
"github.com/duffy/usb-server/internal/crypto"
)
// tunnelCodec seals and opens tunnel payloads.
//
// It sits above the transport so that a tunnel is protected the same way
// whether its frames travel directly or through the relay. A nil codec passes
// data through unchanged, which is what a client configured with only a group
// hash — and therefore unable to derive the key — falls back to.
type tunnelCodec struct {
sealer *crypto.Sealer
opener *crypto.Opener
}
// newTunnelCodec builds a codec for one end of a tunnel.
// send is the direction this end transmits in; it receives on the other.
func newTunnelCodec(secret *crypto.TunnelSecret, tunnelID string, send crypto.Direction) (*tunnelCodec, error) {
if secret == nil {
return nil, nil // unencrypted, by configuration
}
key, err := secret.TunnelKey(tunnelID)
if err != nil {
return nil, err
}
recv := crypto.DirShareToUse
if send == crypto.DirShareToUse {
recv = crypto.DirUseToShare
}
sealer, err := crypto.NewSealer(key, send)
if err != nil {
return nil, err
}
opener, err := crypto.NewOpener(key, recv)
if err != nil {
return nil, err
}
return &tunnelCodec{sealer: sealer, opener: opener}, nil
}
// encode prepares a payload for transmission.
func (c *tunnelCodec) encode(payload []byte) ([]byte, error) {
if c == nil {
return payload, nil
}
return c.sealer.Seal(payload)
}
// decode recovers a received payload.
func (c *tunnelCodec) decode(frame []byte) ([]byte, error) {
if c == nil {
return frame, nil
}
return c.opener.Open(frame)
}
// encrypted reports whether this codec actually protects anything.
func (c *tunnelCodec) encrypted() bool { return c != nil }
// tunnelSender delivers one encoded frame to the peer.
type tunnelSender func(frame []byte) error
// relaySender routes frames through the relay, tagged with the tunnel ID.
func relaySender(c *Client, tunnelID string) tunnelSender {
return func(frame []byte) error {
return c.SendTunnelData(tunnelID, frame)
}
}
// directSender routes frames over an established direct connection.
func directSender(conn *directConn) tunnelSender {
return func(frame []byte) error {
return conn.WriteFrame(frame)
}
}
// send encodes a payload and hands it to the transport.
func send(codec *tunnelCodec, sender tunnelSender, payload []byte) error {
frame, err := codec.encode(payload)
if err != nil {
return fmt.Errorf("encoding tunnel frame: %w", err)
}
return sender(frame)
}
// receiveLoop reads frames from a direct connection, decodes them and hands
// each payload to deliver. It returns when the connection ends, when the
// tunnel is torn down, or on the first frame that fails to authenticate.
func receiveLoop(conn *directConn, codec *tunnelCodec, deliver func([]byte) error, done <-chan struct{}, label string) {
for {
select {
case <-done:
return
default:
}
frame, err := conn.ReadFrame()
if err != nil {
select {
case <-done:
default:
log.Printf("[direct] %s: read ended: %v", label, err)
}
return
}
payload, err := codec.decode(frame)
if err != nil {
// A frame that fails to authenticate means the stream is either
// corrupt or being tampered with. Either way this tunnel cannot
// be trusted to carry USB traffic any further.
log.Printf("[direct] %s: dropping connection: %v", label, err)
return
}
if err := deliver(payload); err != nil {
log.Printf("[direct] %s: delivery failed: %v", label, err)
return
}
}
}
+283 -91
View File
@@ -6,13 +6,19 @@ import (
"net"
"strings"
"sync"
"time"
"github.com/duffy/usb-server/internal/config"
"github.com/duffy/usb-server/internal/crypto"
"github.com/duffy/usb-server/internal/protocol"
"github.com/duffy/usb-server/internal/usbip"
"github.com/google/uuid"
)
// attachRequestTimeout bounds how long we wait for a share client to answer a
// device request before giving up.
const attachRequestTimeout = 30 * time.Second
// RemoteDevice represents a USB device available from a share client
type RemoteDevice struct {
protocol.USBDevice
@@ -29,15 +35,25 @@ type AttachedDevice struct {
// UseManager handles receiving/using remote USB devices
type UseManager struct {
client *Client
cfg *config.Config
cfgPath string
mu sync.RWMutex
available map[string][]RemoteDevice // clientID -> devices
attached map[string]*AttachedDevice // busID@clientID -> attached info
tunnels map[string]*useTunnel // tunnelID -> tunnel
pending map[string]chan *protocol.DeviceGranted // requestID -> response channel
forceDetachable map[string]bool // clientID -> allow_force_detach
client *Client
cfg *config.Config
cfgPath string
mu sync.RWMutex
available map[string][]RemoteDevice // clientID -> devices
attached map[string]*AttachedDevice // busID@clientID -> attached info
tunnels map[string]*useTunnel // tunnelID -> tunnel
pending map[string]*pendingRequest // requestID -> in-flight attach
forceDetachable map[string]bool // clientID -> allow_force_detach
}
// pendingRequest tracks an attach request waiting for the share client's reply.
// It carries the target so that a reply arriving after the caller gave up can
// still be undone — otherwise the share side would hold the device open for a
// user who is no longer waiting for it.
type pendingRequest struct {
clientID string
busID string
resp chan *protocol.DeviceGranted
}
type useTunnel struct {
@@ -45,32 +61,114 @@ type useTunnel struct {
busID string
clientID string
conn net.Conn // our end of the socketpair
done chan struct{}
// codec seals outgoing and opens incoming payloads, whichever transport
// carries them.
codec *tunnelCodec
// send delivers an encoded frame; direct is non-nil when this tunnel
// bypasses the relay.
send tunnelSender
direct *directConn
// done is closed exactly once. Several paths can tear down the same
// tunnel — an explicit detach, a release from the share side, the peer
// leaving, a relay disconnect — and closing it twice would panic.
done chan struct{}
closeOnce sync.Once
}
// tryDirect attempts a direct connection to the granting peer, returning nil
// if none can be established.
//
// Every failure here is benign: the tunnel simply runs through the relay, the
// way it always did. Only the latency improves when this succeeds.
func (um *UseManager) tryDirect(granted *protocol.DeviceGranted, secret *crypto.TunnelSecret) *directConn {
if um.cfg.DisableDirect || secret == nil || len(granted.Endpoints) == 0 {
return nil
}
conn, addr, err := dialDirect(granted.Endpoints, granted.TunnelID, secret.PeerToken(granted.TunnelID))
if err != nil {
log.Printf("[use] no direct route to %s (%v), using the relay", granted.BusID, err)
return nil
}
log.Printf("[use] direct connection to %s established for %s", addr, granted.BusID)
return conn
}
// NewUseManager creates a use manager
func NewUseManager(client *Client, cfg *config.Config, cfgPath string) *UseManager {
um := &UseManager{
client: client,
cfg: cfg,
cfgPath: cfgPath,
client: client,
cfg: cfg,
cfgPath: cfgPath,
available: make(map[string][]RemoteDevice),
attached: make(map[string]*AttachedDevice),
tunnels: make(map[string]*useTunnel),
pending: make(map[string]chan *protocol.DeviceGranted),
pending: make(map[string]*pendingRequest),
forceDetachable: make(map[string]bool),
}
// Use-side messages only this manager handles.
client.OnDeviceList = um.handleDeviceList
client.OnDeviceGranted = um.handleDeviceGranted
client.OnDeviceDenied = um.handleDeviceDenied
client.OnDeviceReleased = um.handleDeviceReleased
client.OnTunnelData = um.handleTunnelData
client.OnClientLeft = um.handleClientLeft
// Shared with the share manager in "both" mode, hence multicast.
client.AddTunnelHandler(um.handleTunnelData)
client.AddClientLeftHandler(um.handleClientLeft)
client.AddDisconnectHandler(um.handleRelayDisconnect)
return um
}
// handleRelayDisconnect detaches everything after the relay link drops.
// The relay discarded those tunnels, so the devices are dead: without this
// they would stay listed as attached while no traffic could reach them.
func (um *UseManager) handleRelayDisconnect() {
um.mu.Lock()
defer um.mu.Unlock()
if n := len(um.attached); n > 0 {
log.Printf("[use] detaching %d device(s) (relay connection lost)", n)
for key, dev := range um.attached {
um.closeAttachedLocked(key, dev)
}
}
// The peers that advertised these are unreachable, and their device lists
// are re-sent on reconnect. Keeping stale entries would show devices the
// UI cannot actually attach.
um.available = make(map[string][]RemoteDevice)
um.forceDetachable = make(map[string]bool)
}
// closeAttachedLocked tears down one attached device: its tunnel, its VHCI
// port and its bookkeeping. Callers must hold um.mu.
func (um *UseManager) closeAttachedLocked(key string, dev *AttachedDevice) {
if tunnel, ok := um.tunnels[dev.TunnelID]; ok {
tunnel.closeOnce.Do(func() { close(tunnel.done) })
if tunnel.conn != nil {
tunnel.conn.Close()
}
if tunnel.direct != nil {
tunnel.direct.Close()
}
delete(um.tunnels, dev.TunnelID)
}
if dev.VHCIPort >= 0 {
if err := usbip.DetachDevice(dev.VHCIPort); err != nil {
log.Printf("[use] warning: VHCI detach error for %s: %v", key, err)
}
}
delete(um.attached, key)
}
// GetAvailableDevices returns all available remote devices
func (um *UseManager) GetAvailableDevices() []RemoteDevice {
um.mu.RLock()
@@ -115,7 +213,7 @@ func (um *UseManager) AttachDevice(clientID, busID string) error {
respChan := make(chan *protocol.DeviceGranted, 1)
um.mu.Lock()
um.pending[requestID] = respChan
um.pending[requestID] = &pendingRequest{clientID: clientID, busID: busID, resp: respChan}
um.mu.Unlock()
defer func() {
@@ -137,13 +235,22 @@ func (um *UseManager) AttachDevice(clientID, busID string) error {
log.Printf("[use] requesting device %s from %s", busID, clientID)
// Wait for response (with timeout via context)
// Wait for a grant or denial. Without the timeout a share client that
// never answers — because it crashed, or the relay dropped the message —
// would leave this call blocked forever, and with it the HTTP request or
// auto-connect goroutine that made it.
timer := time.NewTimer(attachRequestTimeout)
defer timer.Stop()
select {
case granted, ok := <-respChan:
if !ok || granted == nil {
return fmt.Errorf("device request denied")
}
return um.setupVHCI(clientID, busID, granted)
case <-timer.C:
return fmt.Errorf("no response from %s for device %s after %s",
protocol.ShortID(clientID), busID, attachRequestTimeout)
case <-um.client.ctx.Done():
return fmt.Errorf("client shutting down")
}
@@ -159,24 +266,7 @@ func (um *UseManager) DetachDevice(clientID, busID string) error {
um.mu.Unlock()
return fmt.Errorf("device %s not attached", key)
}
// Clean up tunnel
if tunnel, ok := um.tunnels[dev.TunnelID]; ok {
close(tunnel.done)
if tunnel.conn != nil {
tunnel.conn.Close()
}
delete(um.tunnels, dev.TunnelID)
}
// Detach from VHCI
if dev.VHCIPort >= 0 {
if err := usbip.DetachDevice(dev.VHCIPort); err != nil {
log.Printf("[use] warning: VHCI detach error: %v", err)
}
}
delete(um.attached, key)
um.closeAttachedLocked(key, dev)
um.mu.Unlock()
// Notify share client
@@ -203,9 +293,37 @@ func (um *UseManager) setupVHCI(clientID, busID string, granted *protocol.Device
}
um.mu.RUnlock()
// The granting side tells us whether it encrypts. Both ends must agree:
// a mismatch would turn ciphertext into garbage USB traffic.
secret := um.client.TunnelSecret()
if granted.Encrypted && secret == nil {
return fmt.Errorf("%s encrypts its tunnels but this client has no tokens configured, "+
"only a group hash — copy the three tokens over to connect", protocol.ShortID(clientID))
}
if !granted.Encrypted {
if secret != nil {
log.Printf("[use] warning: %s does not encrypt tunnel traffic for %s",
protocol.ShortID(clientID), busID)
}
secret = nil
}
codec, err := newTunnelCodec(secret, granted.TunnelID, crypto.DirUseToShare)
if err != nil {
return fmt.Errorf("setting up tunnel encryption: %w", err)
}
// Try to reach the peer directly before falling back to the relay. This
// is where the latency win comes from: two machines on the same network
// otherwise send every USB transfer out to the relay and back.
direct := um.tryDirect(granted, secret)
// Platform-specific VHCI attachment (Linux: socketpair+sysfs, Windows: TCP proxy+usbip.exe)
tunnelConn, vhciPort, err := createVHCIAttachment(um.client.ctx, granted, devInfo)
if err != nil {
if direct != nil {
direct.Close()
}
return fmt.Errorf("VHCI attachment: %w", err)
}
@@ -215,6 +333,13 @@ func (um *UseManager) setupVHCI(clientID, busID string, granted *protocol.Device
clientID: clientID,
conn: tunnelConn,
done: make(chan struct{}),
codec: codec,
direct: direct,
}
if direct != nil {
tunnel.send = directSender(direct)
} else {
tunnel.send = relaySender(um.client, granted.TunnelID)
}
key := busID + "@" + clientID
@@ -235,14 +360,42 @@ func (um *UseManager) setupVHCI(clientID, busID string, granted *protocol.Device
}
um.mu.Unlock()
// Start reading from the tunnel socket (VHCI -> relay)
// Start reading from the tunnel socket (VHCI -> peer)
go um.tunnelReadLoop(tunnel)
log.Printf("[use] device %s attached on VHCI port %d", key, vhciPort)
// On a direct connection, incoming frames arrive here instead of through
// the relay's tunnel-data callback.
if direct != nil {
go func() {
receiveLoop(direct, tunnel.codec, func(payload []byte) error {
_, err := tunnel.conn.Write(payload)
return err
}, tunnel.done, "use/"+busID)
// Fix permissions on newly created device nodes (e.g. /dev/video*)
// Losing the direct connection ends the tunnel: the USB/IP stream
// cannot be resumed on the relay mid-conversation.
select {
case <-tunnel.done:
default:
log.Printf("[use] direct connection for %s ended, detaching", key)
um.DetachDevice(clientID, busID)
}
}()
}
transport := "relay"
if direct != nil {
transport = "direct " + direct.RemoteAddr()
}
log.Printf("[use] device %s attached on VHCI port %d (devID=0x%08x speed=%d, %s, encrypted=%v)",
key, vhciPort, granted.DevID, granted.Speed, transport, codec.encrypted())
// Check device status and fix permissions on newly created device nodes
// VHCI-created devices don't get normal udev permissions
go fixVHCIDevicePermissions(vhciPort)
go func() {
logVHCIDeviceStatus(vhciPort)
fixVHCIDevicePermissions(vhciPort)
}()
return nil
}
@@ -268,7 +421,11 @@ func (um *UseManager) tunnelReadLoop(tunnel *useTunnel) {
}
}
if err := um.client.SendTunnelData(tunnel.id, buf[:n]); err != nil {
if protocol.Debug {
usbip.TraceRequest("use-tunnel", buf[:n])
}
if err := send(tunnel.codec, tunnel.send, buf[:n]); err != nil {
log.Printf("[use] tunnel send error: %v", err)
return
}
@@ -305,7 +462,7 @@ func (um *UseManager) handleDeviceList(msg *protocol.DeviceList) {
um.mu.Unlock()
log.Printf("[use] received device list from %s (%s): %d devices",
msg.ClientName, msg.ClientID[:8], len(msg.Devices))
msg.ClientName, protocol.ShortID(msg.ClientID), len(msg.Devices))
// Auto-connect matching devices (outside lock, each in its own goroutine)
for _, dev := range toAutoConnect {
@@ -401,25 +558,71 @@ func (um *UseManager) ForceDetachDevice(clientID, busID string) error {
})
}
// resolvePending hands a response to the waiting AttachDevice call and removes
// the request, so that a duplicate or late reply cannot reach the channel
// twice — a grant arriving after a denial closed it would panic.
func (um *UseManager) resolvePending(requestID string) (*pendingRequest, bool) {
um.mu.Lock()
defer um.mu.Unlock()
req, exists := um.pending[requestID]
if exists {
delete(um.pending, requestID)
}
return req, exists
}
func (um *UseManager) handleDeviceGranted(msg *protocol.DeviceGranted) {
req, exists := um.resolvePending(msg.RequestID)
if !exists {
// Nobody is waiting any more — the request timed out, or the caller
// gave up. The share client has already claimed the device for us, so
// hand it back instead of leaving it stuck in "in use".
log.Printf("[use] late grant for %s, releasing it again", msg.BusID)
um.releaseOrphanedGrant(msg)
return
}
// The channel is buffered with capacity 1 and we are the only sender for
// this request ID, so this never blocks.
req.resp <- msg
}
// releaseOrphanedGrant tells the share client to take back a device that was
// granted to a request nobody is waiting for.
func (um *UseManager) releaseOrphanedGrant(msg *protocol.DeviceGranted) {
// Find who owns this bus ID; the grant message does not name the sender.
um.mu.RLock()
ch, exists := um.pending[msg.RequestID]
var owner string
for clientID, devs := range um.available {
for _, d := range devs {
if d.BusID == msg.BusID {
owner = clientID
break
}
}
if owner != "" {
break
}
}
um.mu.RUnlock()
if exists {
ch <- msg
if owner == "" {
return
}
um.client.SendJSON(&protocol.ReleaseDevice{
Type: protocol.MsgReleaseDevice,
TargetClient: owner,
BusID: msg.BusID,
})
}
func (um *UseManager) handleDeviceDenied(msg *protocol.DeviceDenied) {
log.Printf("[use] device request denied: %s - %s", msg.BusID, msg.Reason)
um.mu.RLock()
ch, exists := um.pending[msg.RequestID]
um.mu.RUnlock()
if exists {
close(ch) // signal denial by closing channel
if req, exists := um.resolvePending(msg.RequestID); exists {
close(req.resp) // a closed channel reads as a denial
}
}
@@ -436,24 +639,8 @@ func (um *UseManager) handleDeviceReleased(msg *protocol.DeviceReleased) {
continue
}
// Clean up tunnel
if tunnel, ok := um.tunnels[dev.TunnelID]; ok {
close(tunnel.done)
if tunnel.conn != nil {
tunnel.conn.Close()
}
delete(um.tunnels, dev.TunnelID)
}
// Detach from VHCI
if dev.VHCIPort >= 0 {
if err := usbip.DetachDevice(dev.VHCIPort); err != nil {
log.Printf("[use] warning: VHCI detach error for force-released device: %v", err)
}
}
delete(um.attached, key)
log.Printf("[use] device %s cleaned up (force-released by share client)", key)
um.closeAttachedLocked(key, dev)
log.Printf("[use] device %s cleaned up (released by share client)", key)
break
}
um.mu.Unlock()
@@ -465,16 +652,35 @@ func (um *UseManager) handleTunnelData(tunnelID string, data []byte) {
um.mu.RUnlock()
if !exists {
log.Printf("[use] tunnel data for unknown tunnel %s (%d bytes)", tunnelID[:8], len(data))
// In "both" mode the share manager sees the same frames and owns the
// other tunnels, so an unknown ID here is normal, not an error.
return
}
// Write to the tunnel socket (relay -> VHCI)
n, err := tunnel.conn.Write(data)
// A tunnel running directly gets its frames from that connection; anything
// arriving via the relay for it is stale or spoofed.
if tunnel.direct != nil {
return
}
payload, err := tunnel.codec.decode(data)
if err != nil {
log.Printf("[use] tunnel write error: %v", err)
} else if n != len(data) {
log.Printf("[use] tunnel short write: %d/%d", n, len(data))
log.Printf("[use] tunnel %s: rejecting relayed frame: %v", protocol.ShortID(tunnelID), err)
tunnel.closeOnce.Do(func() { close(tunnel.done) })
tunnel.conn.Close()
return
}
if protocol.Debug {
usbip.TraceResponse("use-tunnel", payload)
}
// Write to the tunnel socket (peer -> VHCI). A failed write would desync
// the USB/IP stream permanently, so treat it as fatal for this tunnel.
if _, err := tunnel.conn.Write(payload); err != nil {
log.Printf("[use] tunnel %s write error: %v", protocol.ShortID(tunnelID), err)
tunnel.closeOnce.Do(func() { close(tunnel.done) })
tunnel.conn.Close()
}
}
@@ -486,15 +692,7 @@ func (um *UseManager) handleClientLeft(msg *protocol.ClientLeft) {
// Detach any devices from this client
for key, dev := range um.attached {
if dev.ClientID == msg.ClientID {
if tunnel, ok := um.tunnels[dev.TunnelID]; ok {
close(tunnel.done)
tunnel.conn.Close()
delete(um.tunnels, dev.TunnelID)
}
if dev.VHCIPort >= 0 {
usbip.DetachDevice(dev.VHCIPort)
}
delete(um.attached, key)
um.closeAttachedLocked(key, dev)
log.Printf("[use] device %s auto-detached (client left)", key)
}
}
@@ -507,13 +705,7 @@ func (um *UseManager) Cleanup() {
defer um.mu.Unlock()
for key, dev := range um.attached {
if tunnel, ok := um.tunnels[dev.TunnelID]; ok {
close(tunnel.done)
tunnel.conn.Close()
}
if dev.VHCIPort >= 0 {
usbip.DetachDevice(dev.VHCIPort)
}
um.closeAttachedLocked(key, dev)
log.Printf("[use] cleaned up device %s", key)
}