infrastructure/android-fs5
2
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
android-fs5/specs/003-autonomous-agent-control/spec.md
T
Jane Alesi ad8c738160 feat(spec): add 003-autonomous-agent-control specification 
Spec for full AI agent mobile control capability:
- Voice call control (dial/answer/hangup/status/list)
- App lifecycle management (install --url, update, permissions, grant, info)
- Device backup & restore (create/list/restore/app/verify, AES-256 encrypt)
- SMS send/delete + wait-reply via KDE Connect
- Audio control (volume/mute/route)
- Contacts management (list/add/export VCF)

56 tasks across 7 phases. TDD: fixtures → RED tests → GREEN impl.
Target: ≥130 BATS tests on completion.

Closes: specs/003-autonomous-agent-control/{spec,plan,tasks}.md
2026-03-02 18:38:35 +01:00

7.8 KiB
Raw Permalink Blame History

Feature Specification: Autonomous AI Agent Mobile Control

Feature Branch: 003-autonomous-agent-control Created: 2026-03-02 Status: Draft Depends on: 001-android-fs5-management, 002-remote-control

Overview

Enable a fully autonomous AI agent to operate a physical Android device as a communication endpoint and tool platform: make/receive calls, send/receive SMS, manage applications, and maintain device state through backup/restore — without human intervention.

Key insight: The FS5 device with its mobile number is the agent's identity in the physical world. Full control means the agent can authenticate via SMS/voice OTP, install tools it needs, and recover from failures autonomously.

User Scenarios & Testing

User Story 1 — Voice Call Control (Priority: P1)

Why: Voice is the primary human communication channel. An AI agent must be able to initiate and receive calls to interact with services and humans.

Independent Test: Can be tested with a second phone or SIP endpoint.

Acceptance Scenarios:

  1. Given the device is connected, When I run ./fs5 call dial <number>, Then the device initiates a phone call to that number.

  2. Given an incoming call is ringing, When I run ./fs5 call answer, Then the call is answered.

  3. Given a call is active, When I run ./fs5 call hangup, Then the call is terminated.

  4. Given the device is connected, When I run ./fs5 call status, Then the current call state (idle/ringing/active) is returned as JSON.

  5. When I run ./fs5 call list, Then recent call log entries are returned with number, direction, duration.

User Story 2 — Application Lifecycle Management (Priority: P1)

Why: The agent must install tools it needs (authenticators, communication apps) and remove them when done. Update management prevents security vulnerabilities.

Independent Test: Requires an APK source (local file or URL).

Acceptance Scenarios:

  1. When I run ./fs5 app install --url <apk-url>, Then the APK is downloaded, verified, and installed on the device.

  2. When I run ./fs5 app update <package>, Then the installed package is updated to the latest available version.

  3. When I run ./fs5 app update --all, Then all user-installed packages are updated.

  4. When I run ./fs5 app permissions <package>, Then the granted and denied permissions are listed as JSON.

  5. When I run ./fs5 app grant <package> <permission>, Then the specified runtime permission is granted to the package.

  6. When I run ./fs5 app info <package>, Then version, install date, size, and permissions are returned as JSON.

User Story 3 — Device Backup & Restore (Priority: P1)

Why: Autonomous operation requires resilience. The agent must recover from factory resets, app crashes, or data loss without human intervention.

Independent Test: Requires local storage path for backup destination.

Acceptance Scenarios:

  1. When I run ./fs5 backup create --dest <path>, Then a timestamped backup archive is created containing app data, SMS, contacts, and call log.

  2. When I run ./fs5 backup list, Then available backups are listed with timestamp, size, and contents summary.

  3. When I run ./fs5 backup restore --from <backup-path>, Then the selected backup is restored to the device.

  4. When I run ./fs5 backup app <package> --dest <path>, Then a single app's data is backed up (APK + data).

  5. When I run ./fs5 backup verify <backup-path>, Then the backup integrity is checked and a pass/fail result returned.

User Story 4 — SMS Send & Management (Priority: P2)

Why: SMS is required for OTP authentication and communicating with services that don't support voice. Extends existing sms list/sms otp commands.

Acceptance Scenarios:

  1. When I run ./fs5 sms send --to <number> --message <text>, Then an SMS is sent to the specified number via KDE Connect.

  2. When I run ./fs5 sms send --to <number> --message <text> --wait-reply, Then the SMS is sent and the command blocks until a reply arrives (timeout: 60s).

  3. When I run ./fs5 sms delete --id <message-id>, Then the specified SMS is deleted from the device.

User Story 5 — Audio Control (Priority: P2)

Why: Voice calls require microphone/speaker control. The agent must manage audio routing for call quality and recording.

Acceptance Scenarios:

  1. When I run ./fs5 audio volume set --stream call --level 80, Then the call volume is set to 80%.

  2. When I run ./fs5 audio mute --stream mic, Then the microphone is muted.

  3. When I run ./fs5 audio route --output speaker, Then audio is routed to the speakerphone.

User Story 6 — Contacts Management (Priority: P3)

Why: The agent needs to maintain a contact list for reliable communication with known parties.

Acceptance Scenarios:

  1. When I run ./fs5 contacts list, Then all contacts are returned as JSON with name, numbers, and email.

  2. When I run ./fs5 contacts add --name <name> --phone <number>, Then a new contact is created on the device.

  3. When I run ./fs5 contacts export --dest <path>, Then all contacts are exported as a VCF file.

Requirements

Functional Requirements

  • FR-001: Voice call control MUST work via ADB shell commands (telephony manager) or UIAutomator automation of the dialer app.
  • FR-002: App installation MUST support both local APK files and HTTP(S) URLs.
  • FR-003: Backup MUST include: installed APKs, app data (via adb backup), SMS database, contacts (VCF), and call log.
  • FR-004: All commands MUST support --json output flag.
  • FR-005: All commands MUST return exit code 2 when device is not connected.
  • FR-006: SMS send MUST use KDE Connect CLI (already paired).
  • FR-007: Backup/restore MUST be idempotent — running twice produces same result.

Non-Functional Requirements

  • NFR-001: call dial MUST initiate call within 3 seconds.
  • NFR-002: backup create for a typical device MUST complete within 5 minutes.
  • NFR-003: All commands MUST be scriptable (no interactive prompts without --interactive flag).
  • NFR-004: Backup archives MUST be AES-256 encrypted when --encrypt flag is used.

Key Entities

  • Call: state (idle/ringing/active/held), number, direction (in/out), duration
  • Backup: timestamp, path, size, contents (apps/sms/contacts/calllog), checksum
  • AppInfo: package, version, installDate, size, permissions[], isSystem
  • Contact: id, name, phones[], emails[], groups[]

Technical Context

  • ADB: Primary control channel (already established in lib/adb.sh)
  • KDE Connect: SMS send/receive (already paired, used in sms commands)
  • UIAutomator: Dialer automation fallback for call control
  • adb backup: Android backup API (deprecated in Android 12+ but available on Android 9)
  • scrcpy: Screen mirroring for visual verification during calls
  • tesseract: OCR for reading call state from screen

Success Criteria

  • SC-001: Agent can complete a full authentication flow: dial number → receive OTP via SMS → enter OTP — without human intervention.
  • SC-002: Device can be fully restored from backup to operational state in under 10 minutes.
  • SC-003: All P1 user stories covered by BATS tests (≥90% pass rate on connected device).
  • SC-004: ./fs5 --help lists all new subcommands with descriptions.

Out of Scope

  • Video calls (WhatsApp/FaceTime) — requires app-specific automation
  • Mobile data management (APN configuration)
  • SIM card management (PIN, PUK)
  • Root-required operations (device is non-rooted)
Reference in New Issue View Git Blame Copy Permalink