Phone Harness Lets AI Agents Control Real iPhones Without Jailbreaking

The open-source phone-harness project has passed 3,600 GitHub stars by giving coding agents control of real iPhones and Android devices without a jailbreak, Xcode, or a phone-side app. On iOS, it captures the iPhone Mirroring window, reads it with optical character recognition, and sends synthetic mouse and keyboard input. On Android, it uses ADB. Developers can automate mobile workflows without instrumenting the target app or maintaining a conventional mobile testing stack.

Apple supplied the transport

macOS Sequoia’s iPhone Mirroring feature displays an iPhone as an ordinary Mac window and translates mouse and keyboard input into phone interactions. Phone Harness treats that window as an automation surface, leaving Apple to manage the device connection, display stream, and input forwarding.

Each iOS action follows a capture, interpret, act, and verify loop. The harness takes a screenshot, uses Apple’s Vision framework to extract visible text and its coordinates, and sends HID-level events to the corresponding location. Because the screen pixels provide the state, the system requires no DOM, semantic UI tree, WebDriverAgent session, or app installed on the iPhone.

The control loop in code

The CLI reads Python from standard input and pre-imports helpers such as open_app, tap_text, ocr, and type_text. A basic Notes workflow looks like this:

phone-harness <<'PY'
open_app("Notes")
tap_text("New Note")
type_text("hello from the harness")
print([o["text"] for o in ocr()][:10])
PY

Reliable agents should capture the screen again after every action, confirm that the expected text or state appeared, and recover when a tap misses. The same helper calls work across both platforms, while phone-harness config set platform ios|android selects the default transport.

One API, two back ends