Files
Jane Alesi 9cb6bfb714 feat(ui): add UIAutomator-based ui subcommand for AI remote control
Add 'fs5 ui' subcommand with 4 operations:
- ui dump [path]      — dump UI hierarchy XML via uiautomator
- ui elements [--json] — list all interactive elements with coords
- ui find <text>      — find element by text/content-desc, return x y
- ui tap <text>       — find element and tap it (ui find + tap)

lib/input.sh:
- adb_ui_dump()     — uiautomator dump → pull XML → cleanup
- adb_ui_find()     — Python XML parse, case-insensitive search
- adb_ui_elements() — full element inventory as JSON

tests/test_ui.bats: 22 new tests (fixture-based, no live device needed)

Total: 85/85 BATS tests passing (was 64/64)

Research findings (AppAgent pattern, ADB+vision):
- UIAutomator dump: best no-root UI inspection on Android 9 SDK 28
- AppAgent (6.6k★): screenshot + GPT-4V + ADB tap/swipe pipeline
- RustDesk InputService already enabled → accessibility layer available
- scrcpy 3.3.4 + tesseract OCR pipeline already working
2026-03-02 18:13:44 +01:00
..
2026-03-02 15:55:09 +01:00
2026-03-02 15:55:09 +01:00