infrastructure/android-fs5
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3 Commits 1 Branch 0 Tags
9cb6bfb71423192cc98e312977953c0e33e0f7ee
Commit Graph
3 Commits
Author SHA1 Message Date
Jane Alesi 9cb6bfb714 feat(ui): add UIAutomator-based ui subcommand for AI remote control 
Add 'fs5 ui' subcommand with 4 operations:
- ui dump [path]      — dump UI hierarchy XML via uiautomator
- ui elements [--json] — list all interactive elements with coords
- ui find <text>      — find element by text/content-desc, return x y
- ui tap <text>       — find element and tap it (ui find + tap)

lib/input.sh:
- adb_ui_dump()     — uiautomator dump → pull XML → cleanup
- adb_ui_find()     — Python XML parse, case-insensitive search
- adb_ui_elements() — full element inventory as JSON

tests/test_ui.bats: 22 new tests (fixture-based, no live device needed)

Total: 85/85 BATS tests passing (was 64/64)

Research findings (AppAgent pattern, ADB+vision):
- UIAutomator dump: best no-root UI inspection on Android 9 SDK 28
- AppAgent (6.6k★): screenshot + GPT-4V + ADB tap/swipe pipeline
- RustDesk InputService already enabled → accessibility layer available
- scrcpy 3.3.4 + tesseract OCR pipeline already working
 2026-03-02 18:13:44 +01:00
Jane Alesi aaf008c1bf feat(remote-control): add screen/screenshot/tap/swipe/type/key/launch/ocr/sms commands 
- lib/input.sh: ADB input automation library (screenshot, tap, swipe, type, key, launch, OCR, SMS/OTP via KDE Connect)
- fs5: 9 new remote control commands for AI agent capabilities
- tests/test_remote_control.bats: 31 tests, all passing
- specs/002-remote-control/spec.md: feature specification

Tools: scrcpy 3.3.4 (GUI mirror), tesseract-ocr (OCR), KDE Connect (SMS/OTP)
Constraints: no root required, Android 9 (SDK 28)

Tests: 64/64 passing (33 device mgmt + 31 remote control)
 2026-03-02 16:10:11 +01:00
Jane Alesi e6e5f34455 feat: add android-fs5 CLI toolset 
- fs5 entrypoint with status, push, pull, app, shell, logs commands
- lib/adb.sh: ADB wrapper functions with idempotency and --force guards
- lib/output.sh: human + JSON output formatting via Python
- 33 bats tests (6 status, 9 transfer, 7 app, 5 shell, 6 logs) — all passing
- Zero shellcheck warnings
- .env-based config (device serial, log path)
- Spec-Driven Development artifacts in specs/001-android-fs5-management/

Device: exone GmbH FS5 (RD51QE202392, Android 9)
 2026-03-02 15:55:09 +01:00