feat(spec): add 003-autonomous-agent-control specification

Spec for full AI agent mobile control capability:
- Voice call control (dial/answer/hangup/status/list)
- App lifecycle management (install --url, update, permissions, grant, info)
- Device backup & restore (create/list/restore/app/verify, AES-256 encrypt)
- SMS send/delete + wait-reply via KDE Connect
- Audio control (volume/mute/route)
- Contacts management (list/add/export VCF)

56 tasks across 7 phases. TDD: fixtures → RED tests → GREEN impl.
Target: ≥130 BATS tests on completion.

Closes: specs/003-autonomous-agent-control/{spec,plan,tasks}.md
This commit is contained in:
Jane Alesi
2026-03-02 18:38:35 +01:00
parent ad6cbf5fa4
commit ad8c738160
4 changed files with 539 additions and 0 deletions
+202
View File
@@ -0,0 +1,202 @@
# Implementation Plan: Autonomous AI Agent Mobile Control
**Feature**: 003-autonomous-agent-control
**Created**: 2026-03-02
**Status**: Draft
**Spec**: `specs/003-autonomous-agent-control/spec.md`
---
## Technical Context
| Component | Technology | Rationale |
|-----------|-----------|-----------|
| Call control | `adb shell am start -a android.intent.action.CALL` + UIAutomator | ADB intent for dial; UIAutomator for answer/hangup (no root needed) |
| Call state | `adb shell dumpsys telephony.registry` | Exposes call state without root |
| App install (URL) | `curl` download → `adb install` | Reuses existing `adb install` path |
| App update | `adb install -r` (replace) | Standard ADB replace-install |
| App permissions | `adb shell dumpsys package <pkg>` | Full permission dump |
| App grant | `adb shell pm grant <pkg> <perm>` | Runtime permission grant |
| Backup | `adb backup -apk -shared -all` + `adb pull` for SMS/contacts | Android 9 supports `adb backup` |
| SMS send | `kdeconnect-cli --send-sms` | Already paired, used in existing sms commands |
| SMS wait-reply | Poll `kdeconnect-cli --list-notifications` with timeout | KDE Connect exposes incoming SMS as notifications |
| Audio control | `adb shell media volume` + `adb shell service call audio` | Media volume API |
| Contacts | `adb shell content query --uri content://contacts/phones` | Content provider query |
---
## Constitution Check
- [x] **Library-First**: New commands added to `lib/call.sh`, `lib/backup.sh`, `lib/contacts.sh`
- [x] **CLI Interface**: All new commands exposed via `fs5` entrypoint with `--help`
- [x] **Test-First**: BATS tests written before implementation (RED → GREEN)
- [x] **Simplicity**: Extends existing `fs5` CLI — no new entrypoints
- [x] **Anti-Abstraction**: Direct ADB/KDE Connect calls, no wrapper frameworks
- [x] **Integration-First**: Tests run against real device where possible; fixtures for offline
---
## Architecture
### New Library Modules
```text
lib/
├── adb.sh (existing — adb_check_device, adb_cmd, etc.)
├── output.sh (existing — out_human, out_json, out_error)
├── input.sh (existing — tap, swipe, key, type)
├── call.sh (NEW — call_dial, call_answer, call_hangup, call_status, call_list)
├── backup.sh (NEW — backup_create, backup_list, backup_restore, backup_app, backup_verify)
└── contacts.sh (NEW — contacts_list, contacts_add, contacts_export)
```
### fs5 Entrypoint Extensions
New subcommands added to `fs5`:
| Subcommand | Library | P1/P2/P3 |
|------------|---------|----------|
| `call dial/answer/hangup/status/list` | `lib/call.sh` | P1 |
| `app install --url` | `lib/adb.sh` (extend) | P1 |
| `app update [--all]` | `lib/adb.sh` (extend) | P1 |
| `app permissions/grant/info` | `lib/adb.sh` (extend) | P1 |
| `backup create/list/restore/app/verify` | `lib/backup.sh` | P1 |
| `sms send [--wait-reply]` | `lib/adb.sh` (extend) | P2 |
| `sms delete` | `lib/adb.sh` (extend) | P2 |
| `audio volume/mute/route` | `lib/call.sh` (extend) | P2 |
| `contacts list/add/export` | `lib/contacts.sh` | P3 |
---
## Data Model
### Call State JSON
```json
{
"state": "active",
"number": "+49123456789",
"direction": "outgoing",
"duration_seconds": 42,
"timestamp": "2026-03-02T18:30:00Z"
}
```
### Backup Manifest JSON
```json
{
"id": "backup-20260302-183000",
"timestamp": "2026-03-02T18:30:00Z",
"path": "/backups/backup-20260302-183000.tar.gz",
"size_bytes": 524288000,
"checksum": "sha256:abc123...",
"contents": ["apks", "appdata", "sms", "contacts", "calllog"],
"device_serial": "RD51QE202392",
"android_version": "9"
}
```
### AppInfo JSON
```json
{
"package": "com.example.app",
"version_name": "1.2.3",
"version_code": 123,
"install_date": "2026-01-15T10:00:00Z",
"size_bytes": 10485760,
"is_system": false,
"permissions": {
"granted": ["android.permission.CAMERA"],
"denied": ["android.permission.READ_CONTACTS"]
}
}
```
---
## Key Technical Decisions
### ADR-001: Call Control via ADB Intent + UIAutomator
**Decision**: Use `adb shell am start -a android.intent.action.CALL tel:<number>` for
dialing. Use UIAutomator to tap answer/hangup buttons (no root required on Android 9).
**Alternative rejected**: `adb shell service call phone` — requires root on Android 9.
**Consequence**: `call answer` requires UIAutomator XML dump to find button coordinates.
Add fixture for offline testing.
### ADR-002: Backup via `adb backup` + Content Providers
**Decision**: Use `adb backup -apk -shared -all -f backup.ab` for app data.
Use content provider queries for SMS (`content://sms`) and contacts (`content://contacts`).
**Alternative rejected**: Third-party backup apps — requires installation and UI automation.
**Consequence**: `adb backup` is deprecated in Android 12+ but fully functional on
Android 9 (FS5 target). Document this limitation.
### ADR-003: SMS Send via KDE Connect
**Decision**: Use `kdeconnect-cli --send-sms "<msg>" --destination <number> -d <device-id>`
**Alternative rejected**: `adb shell service call isms` — requires root.
**Consequence**: KDE Connect must remain paired. Add pairing check to `sms send`.
---
## Test Strategy
### Offline Tests (no device required)
- `backup verify` — checksum validation against fixture
- `backup list` — parse fixture backup directory
- `app info` — parse fixture `dumpsys package` output
- `call status` — parse fixture `dumpsys telephony.registry` output
- `contacts list` — parse fixture content provider output
### Online Tests (device required, marked `@requires_device`)
- `call dial` — initiates call (verify via `call status`)
- `backup create` — creates archive (verify size > 0)
- `app install --url` — downloads and installs test APK
- `sms send` — sends SMS (verify via `sms list`)
### Fixture Files
```text
tests/fixtures/
├── ui_dump.xml (existing)
├── dumpsys_telephony.txt (NEW — call state output)
├── dumpsys_package.txt (NEW — app info output)
├── content_sms.txt (NEW — SMS content provider output)
└── content_contacts.txt (NEW — contacts content provider output)
```
---
## Dependencies
| Dependency | Status | Notes |
|------------|--------|-------|
| `adb` | ✅ Available | Used throughout existing commands |
| `kdeconnect-cli` | ✅ Paired | Used in existing `sms` commands |
| `curl` | ✅ Available | For APK URL download |
| `sha256sum` | ✅ Available | For backup verification |
| `openssl` | ✅ Available | For AES-256 backup encryption |
| `bats` 1.13.0 | ✅ Available | Test runner |
| UIAutomator | ✅ Available | Used in existing `ui` commands |
---
## Risks
| Risk | Likelihood | Mitigation |
|------|-----------|------------|
| `adb backup` blocked by device policy | Low | Android 9 allows it; document if blocked |
| Call answer via UIAutomator fails (button layout changes) | Medium | Use multiple selector strategies (text + content-desc) |
| KDE Connect pairing lost | Low | Add `sms send` pairing check with re-pair instructions |
| APK URL download fails (network) | Medium | Retry logic + clear error message |
+216
View File
@@ -0,0 +1,216 @@
# Feature Specification: Autonomous AI Agent Mobile Control
**Feature Branch**: `003-autonomous-agent-control`
**Created**: 2026-03-02
**Status**: Draft
**Depends on**: 001-android-fs5-management, 002-remote-control
---
## Overview
Enable a fully autonomous AI agent to operate a physical Android device as a
**communication endpoint and tool platform**: make/receive calls, send/receive SMS,
manage applications, and maintain device state through backup/restore — without
human intervention.
**Key insight**: The FS5 device with its mobile number is the agent's identity in
the physical world. Full control means the agent can authenticate via SMS/voice OTP,
install tools it needs, and recover from failures autonomously.
---
## User Scenarios & Testing
### User Story 1 — Voice Call Control (Priority: P1)
**Why**: Voice is the primary human communication channel. An AI agent must be able
to initiate and receive calls to interact with services and humans.
**Independent Test**: Can be tested with a second phone or SIP endpoint.
**Acceptance Scenarios**:
1. **Given** the device is connected, **When** I run `./fs5 call dial <number>`,
**Then** the device initiates a phone call to that number.
2. **Given** an incoming call is ringing, **When** I run `./fs5 call answer`,
**Then** the call is answered.
3. **Given** a call is active, **When** I run `./fs5 call hangup`,
**Then** the call is terminated.
4. **Given** the device is connected, **When** I run `./fs5 call status`,
**Then** the current call state (idle/ringing/active) is returned as JSON.
5. **When** I run `./fs5 call list`,
**Then** recent call log entries are returned with number, direction, duration.
---
### User Story 2 — Application Lifecycle Management (Priority: P1)
**Why**: The agent must install tools it needs (authenticators, communication apps)
and remove them when done. Update management prevents security vulnerabilities.
**Independent Test**: Requires an APK source (local file or URL).
**Acceptance Scenarios**:
1. **When** I run `./fs5 app install --url <apk-url>`,
**Then** the APK is downloaded, verified, and installed on the device.
2. **When** I run `./fs5 app update <package>`,
**Then** the installed package is updated to the latest available version.
3. **When** I run `./fs5 app update --all`,
**Then** all user-installed packages are updated.
4. **When** I run `./fs5 app permissions <package>`,
**Then** the granted and denied permissions are listed as JSON.
5. **When** I run `./fs5 app grant <package> <permission>`,
**Then** the specified runtime permission is granted to the package.
6. **When** I run `./fs5 app info <package>`,
**Then** version, install date, size, and permissions are returned as JSON.
---
### User Story 3 — Device Backup & Restore (Priority: P1)
**Why**: Autonomous operation requires resilience. The agent must recover from
factory resets, app crashes, or data loss without human intervention.
**Independent Test**: Requires local storage path for backup destination.
**Acceptance Scenarios**:
1. **When** I run `./fs5 backup create --dest <path>`,
**Then** a timestamped backup archive is created containing app data, SMS,
contacts, and call log.
2. **When** I run `./fs5 backup list`,
**Then** available backups are listed with timestamp, size, and contents summary.
3. **When** I run `./fs5 backup restore --from <backup-path>`,
**Then** the selected backup is restored to the device.
4. **When** I run `./fs5 backup app <package> --dest <path>`,
**Then** a single app's data is backed up (APK + data).
5. **When** I run `./fs5 backup verify <backup-path>`,
**Then** the backup integrity is checked and a pass/fail result returned.
---
### User Story 4 — SMS Send & Management (Priority: P2)
**Why**: SMS is required for OTP authentication and communicating with services
that don't support voice. Extends existing `sms list`/`sms otp` commands.
**Acceptance Scenarios**:
1. **When** I run `./fs5 sms send --to <number> --message <text>`,
**Then** an SMS is sent to the specified number via KDE Connect.
2. **When** I run `./fs5 sms send --to <number> --message <text> --wait-reply`,
**Then** the SMS is sent and the command blocks until a reply arrives (timeout: 60s).
3. **When** I run `./fs5 sms delete --id <message-id>`,
**Then** the specified SMS is deleted from the device.
---
### User Story 5 — Audio Control (Priority: P2)
**Why**: Voice calls require microphone/speaker control. The agent must manage
audio routing for call quality and recording.
**Acceptance Scenarios**:
1. **When** I run `./fs5 audio volume set --stream call --level 80`,
**Then** the call volume is set to 80%.
2. **When** I run `./fs5 audio mute --stream mic`,
**Then** the microphone is muted.
3. **When** I run `./fs5 audio route --output speaker`,
**Then** audio is routed to the speakerphone.
---
### User Story 6 — Contacts Management (Priority: P3)
**Why**: The agent needs to maintain a contact list for reliable communication
with known parties.
**Acceptance Scenarios**:
1. **When** I run `./fs5 contacts list`,
**Then** all contacts are returned as JSON with name, numbers, and email.
2. **When** I run `./fs5 contacts add --name <name> --phone <number>`,
**Then** a new contact is created on the device.
3. **When** I run `./fs5 contacts export --dest <path>`,
**Then** all contacts are exported as a VCF file.
---
## Requirements
### Functional Requirements
- **FR-001**: Voice call control MUST work via ADB shell commands (telephony manager)
or UIAutomator automation of the dialer app.
- **FR-002**: App installation MUST support both local APK files and HTTP(S) URLs.
- **FR-003**: Backup MUST include: installed APKs, app data (via `adb backup`),
SMS database, contacts (VCF), and call log.
- **FR-004**: All commands MUST support `--json` output flag.
- **FR-005**: All commands MUST return exit code 2 when device is not connected.
- **FR-006**: SMS send MUST use KDE Connect CLI (already paired).
- **FR-007**: Backup/restore MUST be idempotent — running twice produces same result.
### Non-Functional Requirements
- **NFR-001**: `call dial` MUST initiate call within 3 seconds.
- **NFR-002**: `backup create` for a typical device MUST complete within 5 minutes.
- **NFR-003**: All commands MUST be scriptable (no interactive prompts without `--interactive` flag).
- **NFR-004**: Backup archives MUST be AES-256 encrypted when `--encrypt` flag is used.
### Key Entities
- **Call**: state (idle/ringing/active/held), number, direction (in/out), duration
- **Backup**: timestamp, path, size, contents (apps/sms/contacts/calllog), checksum
- **AppInfo**: package, version, installDate, size, permissions[], isSystem
- **Contact**: id, name, phones[], emails[], groups[]
---
## Technical Context
- **ADB**: Primary control channel (already established in lib/adb.sh)
- **KDE Connect**: SMS send/receive (already paired, used in sms commands)
- **UIAutomator**: Dialer automation fallback for call control
- **`adb backup`**: Android backup API (deprecated in Android 12+ but available on Android 9)
- **scrcpy**: Screen mirroring for visual verification during calls
- **tesseract**: OCR for reading call state from screen
---
## Success Criteria
- **SC-001**: Agent can complete a full authentication flow: dial number → receive OTP via SMS → enter OTP — without human intervention.
- **SC-002**: Device can be fully restored from backup to operational state in under 10 minutes.
- **SC-003**: All P1 user stories covered by BATS tests (≥90% pass rate on connected device).
- **SC-004**: `./fs5 --help` lists all new subcommands with descriptions.
---
## Out of Scope
- Video calls (WhatsApp/FaceTime) — requires app-specific automation
- Mobile data management (APN configuration)
- SIM card management (PIN, PUK)
- Root-required operations (device is non-rooted)
+110
View File
@@ -0,0 +1,110 @@
# Task Breakdown: Autonomous AI Agent Mobile Control
**Feature**: 003-autonomous-agent-control
**Created**: 2026-03-02
**Status**: Pending
---
## Phase 1: Setup & Fixtures
- [ ] T001 Create `tests/fixtures/dumpsys_telephony.txt` — captured `adb shell dumpsys telephony.registry` output (idle + active states)
- [ ] T002 Create `tests/fixtures/dumpsys_package.txt` — captured `adb shell dumpsys package <pkg>` output
- [ ] T003 Create `tests/fixtures/content_sms.txt` — captured `adb shell content query --uri content://sms` output
- [ ] T004 Create `tests/fixtures/content_contacts.txt` — captured `adb shell content query --uri content://contacts/phones` output
---
## Phase 2: Foundational Libraries
- [ ] T005 Create `lib/call.sh``call_dial`, `call_answer`, `call_hangup`, `call_status`, `call_list` functions
- [ ] T006 Create `lib/backup.sh``backup_create`, `backup_list`, `backup_restore`, `backup_app`, `backup_verify` functions
- [ ] T007 Create `lib/contacts.sh``contacts_list`, `contacts_add`, `contacts_export` functions
- [ ] T008 Extend `lib/adb.sh` — add `adb_install_url`, `adb_app_update`, `adb_app_permissions`, `adb_app_grant`, `adb_app_info`
---
## Phase 3: Tests — RED Phase (write before implementation)
- [ ] T009 [US1] Write `tests/test_call.bats` — call dial/answer/hangup/status/list (offline fixtures + online markers)
- [ ] T010 [US2] Write `tests/test_app_extended.bats` — app install --url, update, permissions, grant, info
- [ ] T011 [US3] Write `tests/test_backup.bats` — backup create/list/restore/app/verify
- [ ] T012 [US4] Write `tests/test_sms_extended.bats` — sms send, sms send --wait-reply, sms delete
- [ ] T013 [US5] Write `tests/test_audio.bats` — audio volume/mute/route
- [ ] T014 [US6] Write `tests/test_contacts.bats` — contacts list/add/export
- [ ] T015 Verify ALL new tests FAIL (RED phase confirmed) before implementing libraries
---
## Phase 4: Implementation — P1 (Call + App + Backup)
- [ ] T016 [US1] Implement `lib/call.sh``call_dial` via `adb shell am start -a android.intent.action.CALL`
- [ ] T017 [US1] Implement `lib/call.sh``call_answer`/`call_hangup` via UIAutomator button tap
- [ ] T018 [US1] Implement `lib/call.sh``call_status` via `dumpsys telephony.registry` parsing
- [ ] T019 [US1] Implement `lib/call.sh``call_list` via `adb shell content query --uri content://call_log/calls`
- [ ] T020 [US1] Add `call` subcommand to `fs5` entrypoint with `--help`
- [ ] T021 Verify T009 tests PASS (GREEN for US1)
- [ ] T022 [US2] Implement `adb_install_url``curl` download to `/tmp` + `adb install`
- [ ] T023 [US2] Implement `adb_app_update``adb install -r` for single package
- [ ] T024 [US2] Implement `adb_app_update --all` — iterate user packages, `adb install -r` each
- [ ] T025 [US2] Implement `adb_app_permissions` — parse `dumpsys package` granted/denied sections
- [ ] T026 [US2] Implement `adb_app_grant``adb shell pm grant <pkg> <perm>`
- [ ] T027 [US2] Implement `adb_app_info` — parse `dumpsys package` for version/date/size + JSON output
- [ ] T028 [US2] Extend `app` subcommand in `fs5` with `install --url`, `update`, `permissions`, `grant`, `info`
- [ ] T029 Verify T010 tests PASS (GREEN for US2)
- [ ] T030 [US3] Implement `backup_create``adb backup` + content provider pulls + tar.gz + manifest JSON
- [ ] T031 [US3] Implement `backup_list` — scan backup directory, parse manifests, output JSON array
- [ ] T032 [US3] Implement `backup_restore` — extract archive, `adb restore`, push SMS/contacts
- [ ] T033 [US3] Implement `backup_app` — single package APK pull + `adb backup -package <pkg>`
- [ ] T034 [US3] Implement `backup_verify` — sha256sum check against manifest checksum
- [ ] T035 [US3] Add `backup` subcommand to `fs5` entrypoint with `--help`
- [ ] T036 Verify T011 tests PASS (GREEN for US3)
---
## Phase 5: Implementation — P2 (SMS Send + Audio)
- [ ] T037 [US4] Implement `sms send``kdeconnect-cli --send-sms` with pairing check
- [ ] T038 [US4] Implement `sms send --wait-reply` — poll KDE Connect notifications with 60s timeout
- [ ] T039 [US4] Implement `sms delete``adb shell content delete --uri content://sms/<id>`
- [ ] T040 [US4] Extend `sms` subcommand in `fs5` with `send`, `delete`
- [ ] T041 Verify T012 tests PASS (GREEN for US4)
- [ ] T042 [US5] Implement `audio volume set``adb shell media volume --stream <n> --set <level>`
- [ ] T043 [US5] Implement `audio mute``adb shell media volume --stream <n> --set 0`
- [ ] T044 [US5] Implement `audio route``adb shell service call audio` for speaker routing
- [ ] T045 [US5] Add `audio` subcommand to `fs5` entrypoint with `--help`
- [ ] T046 Verify T013 tests PASS (GREEN for US5)
---
## Phase 6: Implementation — P3 (Contacts)
- [ ] T047 [US6] Implement `contacts_list``adb shell content query --uri content://contacts/phones` + JSON
- [ ] T048 [US6] Implement `contacts_add``adb shell content insert --uri content://contacts/raw_contacts`
- [ ] T049 [US6] Implement `contacts_export` — query all contacts, format as VCF, write to file
- [ ] T050 [US6] Add `contacts` subcommand to `fs5` entrypoint with `--help`
- [ ] T051 Verify T014 tests PASS (GREEN for US6)
---
## Phase 7: Polish & Integration
- [ ] T052 [P] Run `shellcheck fs5 lib/*.sh` — fix all warnings
- [ ] T053 [P] Update `README.md` — document `call`, `backup`, `audio`, `contacts` subcommands
- [ ] T054 [P] Update `NEXT_STEPS.md` — mark 003 tasks in progress
- [ ] T055 Add integration test: full auth flow (dial → wait OTP → enter OTP) as `tests/test_integration_auth.bats`
- [ ] T056 Verify all tests pass: `bats tests/` (target: ≥130 tests)
---
## Dependency Graph
```text
T001-T004 (fixtures) → T009-T014 (tests RED)
T005-T008 (libraries) → T016-T051 (implementation)
T009-T014 (tests RED) → T015 (verify RED) → T016+ (implement GREEN)
T016-T051 (all GREEN) → T052-T056 (polish)
```