From ad8c73816036ccaa5c68da8fe9845f0d0555bf0b Mon Sep 17 00:00:00 2001 From: Jane Alesi Date: Mon, 2 Mar 2026 18:38:35 +0100 Subject: [PATCH] feat(spec): add 003-autonomous-agent-control specification MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec for full AI agent mobile control capability: - Voice call control (dial/answer/hangup/status/list) - App lifecycle management (install --url, update, permissions, grant, info) - Device backup & restore (create/list/restore/app/verify, AES-256 encrypt) - SMS send/delete + wait-reply via KDE Connect - Audio control (volume/mute/route) - Contacts management (list/add/export VCF) 56 tasks across 7 phases. TDD: fixtures → RED tests → GREEN impl. Target: ≥130 BATS tests on completion. Closes: specs/003-autonomous-agent-control/{spec,plan,tasks}.md --- NEXT_STEPS.md | 11 + specs/003-autonomous-agent-control/plan.md | 202 ++++++++++++++++++ specs/003-autonomous-agent-control/spec.md | 216 ++++++++++++++++++++ specs/003-autonomous-agent-control/tasks.md | 110 ++++++++++ 4 files changed, 539 insertions(+) create mode 100644 specs/003-autonomous-agent-control/plan.md create mode 100644 specs/003-autonomous-agent-control/spec.md create mode 100644 specs/003-autonomous-agent-control/tasks.md diff --git a/NEXT_STEPS.md b/NEXT_STEPS.md index 4f8d445..37f96f7 100644 --- a/NEXT_STEPS.md +++ b/NEXT_STEPS.md @@ -6,6 +6,17 @@ --- +## Priority 0 — Spec 003 Implementation (Next Feature) + +- [ ] Start with T001–T004: capture device fixtures (`dumpsys telephony`, `dumpsys package`, content providers) +- [ ] T005–T008: create `lib/call.sh`, `lib/backup.sh`, `lib/contacts.sh`, extend `lib/adb.sh` +- [ ] T009–T015: write BATS tests (RED phase) for all 6 user stories +- [ ] T016–T051: implement P1 (call/app/backup), P2 (sms-send/audio), P3 (contacts) +- [ ] T052–T056: shellcheck, README update, integration auth flow test +- See full task list: `specs/003-autonomous-agent-control/tasks.md` + +--- + ## Priority 1 — Spec 002 Completion - [ ] Create `specs/002-remote-control/plan.md` — architecture decisions for remote control layer diff --git a/specs/003-autonomous-agent-control/plan.md b/specs/003-autonomous-agent-control/plan.md new file mode 100644 index 0000000..8d10461 --- /dev/null +++ b/specs/003-autonomous-agent-control/plan.md @@ -0,0 +1,202 @@ +# Implementation Plan: Autonomous AI Agent Mobile Control + +**Feature**: 003-autonomous-agent-control +**Created**: 2026-03-02 +**Status**: Draft +**Spec**: `specs/003-autonomous-agent-control/spec.md` + +--- + +## Technical Context + +| Component | Technology | Rationale | +|-----------|-----------|-----------| +| Call control | `adb shell am start -a android.intent.action.CALL` + UIAutomator | ADB intent for dial; UIAutomator for answer/hangup (no root needed) | +| Call state | `adb shell dumpsys telephony.registry` | Exposes call state without root | +| App install (URL) | `curl` download → `adb install` | Reuses existing `adb install` path | +| App update | `adb install -r` (replace) | Standard ADB replace-install | +| App permissions | `adb shell dumpsys package ` | Full permission dump | +| App grant | `adb shell pm grant ` | Runtime permission grant | +| Backup | `adb backup -apk -shared -all` + `adb pull` for SMS/contacts | Android 9 supports `adb backup` | +| SMS send | `kdeconnect-cli --send-sms` | Already paired, used in existing sms commands | +| SMS wait-reply | Poll `kdeconnect-cli --list-notifications` with timeout | KDE Connect exposes incoming SMS as notifications | +| Audio control | `adb shell media volume` + `adb shell service call audio` | Media volume API | +| Contacts | `adb shell content query --uri content://contacts/phones` | Content provider query | + +--- + +## Constitution Check + +- [x] **Library-First**: New commands added to `lib/call.sh`, `lib/backup.sh`, `lib/contacts.sh` +- [x] **CLI Interface**: All new commands exposed via `fs5` entrypoint with `--help` +- [x] **Test-First**: BATS tests written before implementation (RED → GREEN) +- [x] **Simplicity**: Extends existing `fs5` CLI — no new entrypoints +- [x] **Anti-Abstraction**: Direct ADB/KDE Connect calls, no wrapper frameworks +- [x] **Integration-First**: Tests run against real device where possible; fixtures for offline + +--- + +## Architecture + +### New Library Modules + +```text +lib/ +├── adb.sh (existing — adb_check_device, adb_cmd, etc.) +├── output.sh (existing — out_human, out_json, out_error) +├── input.sh (existing — tap, swipe, key, type) +├── call.sh (NEW — call_dial, call_answer, call_hangup, call_status, call_list) +├── backup.sh (NEW — backup_create, backup_list, backup_restore, backup_app, backup_verify) +└── contacts.sh (NEW — contacts_list, contacts_add, contacts_export) +``` + +### fs5 Entrypoint Extensions + +New subcommands added to `fs5`: + +| Subcommand | Library | P1/P2/P3 | +|------------|---------|----------| +| `call dial/answer/hangup/status/list` | `lib/call.sh` | P1 | +| `app install --url` | `lib/adb.sh` (extend) | P1 | +| `app update [--all]` | `lib/adb.sh` (extend) | P1 | +| `app permissions/grant/info` | `lib/adb.sh` (extend) | P1 | +| `backup create/list/restore/app/verify` | `lib/backup.sh` | P1 | +| `sms send [--wait-reply]` | `lib/adb.sh` (extend) | P2 | +| `sms delete` | `lib/adb.sh` (extend) | P2 | +| `audio volume/mute/route` | `lib/call.sh` (extend) | P2 | +| `contacts list/add/export` | `lib/contacts.sh` | P3 | + +--- + +## Data Model + +### Call State JSON + +```json +{ + "state": "active", + "number": "+49123456789", + "direction": "outgoing", + "duration_seconds": 42, + "timestamp": "2026-03-02T18:30:00Z" +} +``` + +### Backup Manifest JSON + +```json +{ + "id": "backup-20260302-183000", + "timestamp": "2026-03-02T18:30:00Z", + "path": "/backups/backup-20260302-183000.tar.gz", + "size_bytes": 524288000, + "checksum": "sha256:abc123...", + "contents": ["apks", "appdata", "sms", "contacts", "calllog"], + "device_serial": "RD51QE202392", + "android_version": "9" +} +``` + +### AppInfo JSON + +```json +{ + "package": "com.example.app", + "version_name": "1.2.3", + "version_code": 123, + "install_date": "2026-01-15T10:00:00Z", + "size_bytes": 10485760, + "is_system": false, + "permissions": { + "granted": ["android.permission.CAMERA"], + "denied": ["android.permission.READ_CONTACTS"] + } +} +``` + +--- + +## Key Technical Decisions + +### ADR-001: Call Control via ADB Intent + UIAutomator + +**Decision**: Use `adb shell am start -a android.intent.action.CALL tel:` for +dialing. Use UIAutomator to tap answer/hangup buttons (no root required on Android 9). + +**Alternative rejected**: `adb shell service call phone` — requires root on Android 9. + +**Consequence**: `call answer` requires UIAutomator XML dump to find button coordinates. +Add fixture for offline testing. + +### ADR-002: Backup via `adb backup` + Content Providers + +**Decision**: Use `adb backup -apk -shared -all -f backup.ab` for app data. +Use content provider queries for SMS (`content://sms`) and contacts (`content://contacts`). + +**Alternative rejected**: Third-party backup apps — requires installation and UI automation. + +**Consequence**: `adb backup` is deprecated in Android 12+ but fully functional on +Android 9 (FS5 target). Document this limitation. + +### ADR-003: SMS Send via KDE Connect + +**Decision**: Use `kdeconnect-cli --send-sms "" --destination -d ` + +**Alternative rejected**: `adb shell service call isms` — requires root. + +**Consequence**: KDE Connect must remain paired. Add pairing check to `sms send`. + +--- + +## Test Strategy + +### Offline Tests (no device required) + +- `backup verify` — checksum validation against fixture +- `backup list` — parse fixture backup directory +- `app info` — parse fixture `dumpsys package` output +- `call status` — parse fixture `dumpsys telephony.registry` output +- `contacts list` — parse fixture content provider output + +### Online Tests (device required, marked `@requires_device`) + +- `call dial` — initiates call (verify via `call status`) +- `backup create` — creates archive (verify size > 0) +- `app install --url` — downloads and installs test APK +- `sms send` — sends SMS (verify via `sms list`) + +### Fixture Files + +```text +tests/fixtures/ +├── ui_dump.xml (existing) +├── dumpsys_telephony.txt (NEW — call state output) +├── dumpsys_package.txt (NEW — app info output) +├── content_sms.txt (NEW — SMS content provider output) +└── content_contacts.txt (NEW — contacts content provider output) +``` + +--- + +## Dependencies + +| Dependency | Status | Notes | +|------------|--------|-------| +| `adb` | ✅ Available | Used throughout existing commands | +| `kdeconnect-cli` | ✅ Paired | Used in existing `sms` commands | +| `curl` | ✅ Available | For APK URL download | +| `sha256sum` | ✅ Available | For backup verification | +| `openssl` | ✅ Available | For AES-256 backup encryption | +| `bats` 1.13.0 | ✅ Available | Test runner | +| UIAutomator | ✅ Available | Used in existing `ui` commands | + +--- + +## Risks + +| Risk | Likelihood | Mitigation | +|------|-----------|------------| +| `adb backup` blocked by device policy | Low | Android 9 allows it; document if blocked | +| Call answer via UIAutomator fails (button layout changes) | Medium | Use multiple selector strategies (text + content-desc) | +| KDE Connect pairing lost | Low | Add `sms send` pairing check with re-pair instructions | +| APK URL download fails (network) | Medium | Retry logic + clear error message | diff --git a/specs/003-autonomous-agent-control/spec.md b/specs/003-autonomous-agent-control/spec.md new file mode 100644 index 0000000..265608d --- /dev/null +++ b/specs/003-autonomous-agent-control/spec.md @@ -0,0 +1,216 @@ +# Feature Specification: Autonomous AI Agent Mobile Control + +**Feature Branch**: `003-autonomous-agent-control` +**Created**: 2026-03-02 +**Status**: Draft +**Depends on**: 001-android-fs5-management, 002-remote-control + +--- + +## Overview + +Enable a fully autonomous AI agent to operate a physical Android device as a +**communication endpoint and tool platform**: make/receive calls, send/receive SMS, +manage applications, and maintain device state through backup/restore — without +human intervention. + +**Key insight**: The FS5 device with its mobile number is the agent's identity in +the physical world. Full control means the agent can authenticate via SMS/voice OTP, +install tools it needs, and recover from failures autonomously. + +--- + +## User Scenarios & Testing + +### User Story 1 — Voice Call Control (Priority: P1) + +**Why**: Voice is the primary human communication channel. An AI agent must be able +to initiate and receive calls to interact with services and humans. + +**Independent Test**: Can be tested with a second phone or SIP endpoint. + +**Acceptance Scenarios**: + +1. **Given** the device is connected, **When** I run `./fs5 call dial `, + **Then** the device initiates a phone call to that number. + +2. **Given** an incoming call is ringing, **When** I run `./fs5 call answer`, + **Then** the call is answered. + +3. **Given** a call is active, **When** I run `./fs5 call hangup`, + **Then** the call is terminated. + +4. **Given** the device is connected, **When** I run `./fs5 call status`, + **Then** the current call state (idle/ringing/active) is returned as JSON. + +5. **When** I run `./fs5 call list`, + **Then** recent call log entries are returned with number, direction, duration. + +--- + +### User Story 2 — Application Lifecycle Management (Priority: P1) + +**Why**: The agent must install tools it needs (authenticators, communication apps) +and remove them when done. Update management prevents security vulnerabilities. + +**Independent Test**: Requires an APK source (local file or URL). + +**Acceptance Scenarios**: + +1. **When** I run `./fs5 app install --url `, + **Then** the APK is downloaded, verified, and installed on the device. + +2. **When** I run `./fs5 app update `, + **Then** the installed package is updated to the latest available version. + +3. **When** I run `./fs5 app update --all`, + **Then** all user-installed packages are updated. + +4. **When** I run `./fs5 app permissions `, + **Then** the granted and denied permissions are listed as JSON. + +5. **When** I run `./fs5 app grant `, + **Then** the specified runtime permission is granted to the package. + +6. **When** I run `./fs5 app info `, + **Then** version, install date, size, and permissions are returned as JSON. + +--- + +### User Story 3 — Device Backup & Restore (Priority: P1) + +**Why**: Autonomous operation requires resilience. The agent must recover from +factory resets, app crashes, or data loss without human intervention. + +**Independent Test**: Requires local storage path for backup destination. + +**Acceptance Scenarios**: + +1. **When** I run `./fs5 backup create --dest `, + **Then** a timestamped backup archive is created containing app data, SMS, + contacts, and call log. + +2. **When** I run `./fs5 backup list`, + **Then** available backups are listed with timestamp, size, and contents summary. + +3. **When** I run `./fs5 backup restore --from `, + **Then** the selected backup is restored to the device. + +4. **When** I run `./fs5 backup app --dest `, + **Then** a single app's data is backed up (APK + data). + +5. **When** I run `./fs5 backup verify `, + **Then** the backup integrity is checked and a pass/fail result returned. + +--- + +### User Story 4 — SMS Send & Management (Priority: P2) + +**Why**: SMS is required for OTP authentication and communicating with services +that don't support voice. Extends existing `sms list`/`sms otp` commands. + +**Acceptance Scenarios**: + +1. **When** I run `./fs5 sms send --to --message `, + **Then** an SMS is sent to the specified number via KDE Connect. + +2. **When** I run `./fs5 sms send --to --message --wait-reply`, + **Then** the SMS is sent and the command blocks until a reply arrives (timeout: 60s). + +3. **When** I run `./fs5 sms delete --id `, + **Then** the specified SMS is deleted from the device. + +--- + +### User Story 5 — Audio Control (Priority: P2) + +**Why**: Voice calls require microphone/speaker control. The agent must manage +audio routing for call quality and recording. + +**Acceptance Scenarios**: + +1. **When** I run `./fs5 audio volume set --stream call --level 80`, + **Then** the call volume is set to 80%. + +2. **When** I run `./fs5 audio mute --stream mic`, + **Then** the microphone is muted. + +3. **When** I run `./fs5 audio route --output speaker`, + **Then** audio is routed to the speakerphone. + +--- + +### User Story 6 — Contacts Management (Priority: P3) + +**Why**: The agent needs to maintain a contact list for reliable communication +with known parties. + +**Acceptance Scenarios**: + +1. **When** I run `./fs5 contacts list`, + **Then** all contacts are returned as JSON with name, numbers, and email. + +2. **When** I run `./fs5 contacts add --name --phone `, + **Then** a new contact is created on the device. + +3. **When** I run `./fs5 contacts export --dest `, + **Then** all contacts are exported as a VCF file. + +--- + +## Requirements + +### Functional Requirements + +- **FR-001**: Voice call control MUST work via ADB shell commands (telephony manager) + or UIAutomator automation of the dialer app. +- **FR-002**: App installation MUST support both local APK files and HTTP(S) URLs. +- **FR-003**: Backup MUST include: installed APKs, app data (via `adb backup`), + SMS database, contacts (VCF), and call log. +- **FR-004**: All commands MUST support `--json` output flag. +- **FR-005**: All commands MUST return exit code 2 when device is not connected. +- **FR-006**: SMS send MUST use KDE Connect CLI (already paired). +- **FR-007**: Backup/restore MUST be idempotent — running twice produces same result. + +### Non-Functional Requirements + +- **NFR-001**: `call dial` MUST initiate call within 3 seconds. +- **NFR-002**: `backup create` for a typical device MUST complete within 5 minutes. +- **NFR-003**: All commands MUST be scriptable (no interactive prompts without `--interactive` flag). +- **NFR-004**: Backup archives MUST be AES-256 encrypted when `--encrypt` flag is used. + +### Key Entities + +- **Call**: state (idle/ringing/active/held), number, direction (in/out), duration +- **Backup**: timestamp, path, size, contents (apps/sms/contacts/calllog), checksum +- **AppInfo**: package, version, installDate, size, permissions[], isSystem +- **Contact**: id, name, phones[], emails[], groups[] + +--- + +## Technical Context + +- **ADB**: Primary control channel (already established in lib/adb.sh) +- **KDE Connect**: SMS send/receive (already paired, used in sms commands) +- **UIAutomator**: Dialer automation fallback for call control +- **`adb backup`**: Android backup API (deprecated in Android 12+ but available on Android 9) +- **scrcpy**: Screen mirroring for visual verification during calls +- **tesseract**: OCR for reading call state from screen + +--- + +## Success Criteria + +- **SC-001**: Agent can complete a full authentication flow: dial number → receive OTP via SMS → enter OTP — without human intervention. +- **SC-002**: Device can be fully restored from backup to operational state in under 10 minutes. +- **SC-003**: All P1 user stories covered by BATS tests (≥90% pass rate on connected device). +- **SC-004**: `./fs5 --help` lists all new subcommands with descriptions. + +--- + +## Out of Scope + +- Video calls (WhatsApp/FaceTime) — requires app-specific automation +- Mobile data management (APN configuration) +- SIM card management (PIN, PUK) +- Root-required operations (device is non-rooted) diff --git a/specs/003-autonomous-agent-control/tasks.md b/specs/003-autonomous-agent-control/tasks.md new file mode 100644 index 0000000..8fa5ec8 --- /dev/null +++ b/specs/003-autonomous-agent-control/tasks.md @@ -0,0 +1,110 @@ +# Task Breakdown: Autonomous AI Agent Mobile Control + +**Feature**: 003-autonomous-agent-control +**Created**: 2026-03-02 +**Status**: Pending + +--- + +## Phase 1: Setup & Fixtures + +- [ ] T001 Create `tests/fixtures/dumpsys_telephony.txt` — captured `adb shell dumpsys telephony.registry` output (idle + active states) +- [ ] T002 Create `tests/fixtures/dumpsys_package.txt` — captured `adb shell dumpsys package ` output +- [ ] T003 Create `tests/fixtures/content_sms.txt` — captured `adb shell content query --uri content://sms` output +- [ ] T004 Create `tests/fixtures/content_contacts.txt` — captured `adb shell content query --uri content://contacts/phones` output + +--- + +## Phase 2: Foundational Libraries + +- [ ] T005 Create `lib/call.sh` — `call_dial`, `call_answer`, `call_hangup`, `call_status`, `call_list` functions +- [ ] T006 Create `lib/backup.sh` — `backup_create`, `backup_list`, `backup_restore`, `backup_app`, `backup_verify` functions +- [ ] T007 Create `lib/contacts.sh` — `contacts_list`, `contacts_add`, `contacts_export` functions +- [ ] T008 Extend `lib/adb.sh` — add `adb_install_url`, `adb_app_update`, `adb_app_permissions`, `adb_app_grant`, `adb_app_info` + +--- + +## Phase 3: Tests — RED Phase (write before implementation) + +- [ ] T009 [US1] Write `tests/test_call.bats` — call dial/answer/hangup/status/list (offline fixtures + online markers) +- [ ] T010 [US2] Write `tests/test_app_extended.bats` — app install --url, update, permissions, grant, info +- [ ] T011 [US3] Write `tests/test_backup.bats` — backup create/list/restore/app/verify +- [ ] T012 [US4] Write `tests/test_sms_extended.bats` — sms send, sms send --wait-reply, sms delete +- [ ] T013 [US5] Write `tests/test_audio.bats` — audio volume/mute/route +- [ ] T014 [US6] Write `tests/test_contacts.bats` — contacts list/add/export +- [ ] T015 Verify ALL new tests FAIL (RED phase confirmed) before implementing libraries + +--- + +## Phase 4: Implementation — P1 (Call + App + Backup) + +- [ ] T016 [US1] Implement `lib/call.sh` — `call_dial` via `adb shell am start -a android.intent.action.CALL` +- [ ] T017 [US1] Implement `lib/call.sh` — `call_answer`/`call_hangup` via UIAutomator button tap +- [ ] T018 [US1] Implement `lib/call.sh` — `call_status` via `dumpsys telephony.registry` parsing +- [ ] T019 [US1] Implement `lib/call.sh` — `call_list` via `adb shell content query --uri content://call_log/calls` +- [ ] T020 [US1] Add `call` subcommand to `fs5` entrypoint with `--help` +- [ ] T021 Verify T009 tests PASS (GREEN for US1) + +- [ ] T022 [US2] Implement `adb_install_url` — `curl` download to `/tmp` + `adb install` +- [ ] T023 [US2] Implement `adb_app_update` — `adb install -r` for single package +- [ ] T024 [US2] Implement `adb_app_update --all` — iterate user packages, `adb install -r` each +- [ ] T025 [US2] Implement `adb_app_permissions` — parse `dumpsys package` granted/denied sections +- [ ] T026 [US2] Implement `adb_app_grant` — `adb shell pm grant ` +- [ ] T027 [US2] Implement `adb_app_info` — parse `dumpsys package` for version/date/size + JSON output +- [ ] T028 [US2] Extend `app` subcommand in `fs5` with `install --url`, `update`, `permissions`, `grant`, `info` +- [ ] T029 Verify T010 tests PASS (GREEN for US2) + +- [ ] T030 [US3] Implement `backup_create` — `adb backup` + content provider pulls + tar.gz + manifest JSON +- [ ] T031 [US3] Implement `backup_list` — scan backup directory, parse manifests, output JSON array +- [ ] T032 [US3] Implement `backup_restore` — extract archive, `adb restore`, push SMS/contacts +- [ ] T033 [US3] Implement `backup_app` — single package APK pull + `adb backup -package ` +- [ ] T034 [US3] Implement `backup_verify` — sha256sum check against manifest checksum +- [ ] T035 [US3] Add `backup` subcommand to `fs5` entrypoint with `--help` +- [ ] T036 Verify T011 tests PASS (GREEN for US3) + +--- + +## Phase 5: Implementation — P2 (SMS Send + Audio) + +- [ ] T037 [US4] Implement `sms send` — `kdeconnect-cli --send-sms` with pairing check +- [ ] T038 [US4] Implement `sms send --wait-reply` — poll KDE Connect notifications with 60s timeout +- [ ] T039 [US4] Implement `sms delete` — `adb shell content delete --uri content://sms/` +- [ ] T040 [US4] Extend `sms` subcommand in `fs5` with `send`, `delete` +- [ ] T041 Verify T012 tests PASS (GREEN for US4) + +- [ ] T042 [US5] Implement `audio volume set` — `adb shell media volume --stream --set ` +- [ ] T043 [US5] Implement `audio mute` — `adb shell media volume --stream --set 0` +- [ ] T044 [US5] Implement `audio route` — `adb shell service call audio` for speaker routing +- [ ] T045 [US5] Add `audio` subcommand to `fs5` entrypoint with `--help` +- [ ] T046 Verify T013 tests PASS (GREEN for US5) + +--- + +## Phase 6: Implementation — P3 (Contacts) + +- [ ] T047 [US6] Implement `contacts_list` — `adb shell content query --uri content://contacts/phones` + JSON +- [ ] T048 [US6] Implement `contacts_add` — `adb shell content insert --uri content://contacts/raw_contacts` +- [ ] T049 [US6] Implement `contacts_export` — query all contacts, format as VCF, write to file +- [ ] T050 [US6] Add `contacts` subcommand to `fs5` entrypoint with `--help` +- [ ] T051 Verify T014 tests PASS (GREEN for US6) + +--- + +## Phase 7: Polish & Integration + +- [ ] T052 [P] Run `shellcheck fs5 lib/*.sh` — fix all warnings +- [ ] T053 [P] Update `README.md` — document `call`, `backup`, `audio`, `contacts` subcommands +- [ ] T054 [P] Update `NEXT_STEPS.md` — mark 003 tasks in progress +- [ ] T055 Add integration test: full auth flow (dial → wait OTP → enter OTP) as `tests/test_integration_auth.bats` +- [ ] T056 Verify all tests pass: `bats tests/` (target: ≥130 tests) + +--- + +## Dependency Graph + +```text +T001-T004 (fixtures) → T009-T014 (tests RED) +T005-T008 (libraries) → T016-T051 (implementation) +T009-T014 (tests RED) → T015 (verify RED) → T016+ (implement GREEN) +T016-T051 (all GREEN) → T052-T056 (polish) +```