# v0.8.0 Mobile WebView CAPTCHA — Postmortem

**Status:** Parked
**Date filed:** 2026-05-26
**Authors:** mk-qa-master maintainers

## TL;DR

v0.8.0 set out to extend the v0.7 AI Visual Challenge Solver (reCAPTCHA /
hCaptcha) from desktop browsers (Playwright) to **iOS / Android WebViews
via Maestro CLI**. Five PRs landed on this assumption (#51–#56). During
the first end-to-end run against a real iOS Simulator, the architecture
failed in a load-bearing way: **Maestro 2.6 cannot execute JavaScript
inside a WKWebView's DOM**. The mobile dogfood pipeline was abandoned
before shipping.

This document records what went wrong, what was salvaged, and what we
learned so this kind of miss is harder to repeat.

## What we shipped (and what we removed)

| PR | Status | Notes |
|---|---|---|
| #51 — PRD + Maestro perf spike | Kept | PRD remains as the historical record; this doc supersedes its conclusions. The spike script (`scripts/spike-maestro-perf.py`) is kept with a warning banner. |
| #52 — `VisualChallengeDriver` Protocol + `PlaywrightDriver` | **Kept** | Tool-agnostic abstraction; preserved verbatim for a future Appium pivot or any other driver. |
| #53 — `(page, …) → (driver, …)` signature pivot | **Kept** | Same reason as #52. |
| #54 — `MaestroDriver.detect_challenge` | **Reverted** | Built on the false assumption (see below). |
| #55 — `MaestroDriver.execute_solve` | **Reverted** | Same. |
| #56 — `examples/sample_captcha_mobile_app/` | Kept | The SwiftUI WKWebView reference app still works for any future bridge / Appium experiment; its README links here. |

After the reverts, `_driver="maestro"` once again returns
`driver_not_implemented`, exactly as it did before v0.8.0 work began.

## The actual blocker

Maestro's `runScript:` directive runs JavaScript in a **GraalJS sandbox
inside the Maestro CLI process (Kotlin / JVM)**, *not* inside the device's
WebView. That sandbox has no `window`, no `document`, no DOM at all.

Concrete failure mode observed against the SampleCaptchaApp on iOS 26.4:

```text
maestro.orchestra.Orchestra.executeCommands: [Command execution] CommandFailed:
  TypeError: Cannot read property 'location' of undefined
```

…raised from inside `maestro.js.GraalJsEngine.evalWithIIFE`. `window` was
literally undefined — not "WebView not ready," not "wrong frame." The
JS never had a browser context to begin with.

The v0.8 design's entire detection / token-read pipeline depended on
probing the DOM from this directive. With that off the table, both
`MaestroDriver.detect_challenge` and `.execute_solve` had no way to
extract the iframe shape or read `g-recaptcha-response`.

Survey of Maestro 2.6's command surface confirmed no `evalScript:` /
WebView-JS primitive exists. The available primitives are
coordinate-based (`tapOn: point: "x, y"`), accessibility-based
(`tapOn: text:` / `id:`), screenshot (`takeScreenshot:`), and
hierarchy (`maestro hierarchy`). None of those can read the contents
of a hidden `<textarea>`.

## Why the spike didn't catch this

`scripts/spike-maestro-perf.py` measured wall-clock latency for several
Maestro flows. Its `runScript` flow looked like:

```python
- runScript: |
    output.dpr = window.devicePixelRatio;
    output.viewport = window.innerWidth + 'x' + window.innerHeight;
```

The spike only asserted `proc.returncode == 0`. **It never read
`output.dpr` back and checked it was a number.** GraalJS threw on
`window` being undefined; Maestro logged the error but still exited 0;
the spike reported success.

We then ratified the spike's findings — including the (untested)
inference that `runScript` reads WebView DOM — into the PRD's §11
decision table, and built PRs #54 and #55 on top.

The latency numbers from the spike (~30 s fixed overhead per
`maestro test` invocation; 5+ taps trigger throttling) are real and
remain useful. The capability inference was not.

## What we learned

1. **A spike that validates capability must assert the produced value,
   not just `rc == 0`.** Latency spikes can be rc-only; capability
   spikes cannot. Before merging a spike-driven PRD, re-read the spike
   asking: *"if this capability silently failed, would this script
   still report success?"*

2. **Mock-based tests can validate the orchestrator without validating
   the world.** The 47 tests that backed PRs #54 + #55 passed because
   `subprocess.run` was mocked to return synthesized `output.token=…`
   stdout. Excellent for catching regression in the orchestrator;
   useless for catching that the underlying tool can't actually
   produce that stdout. Tier 1 dogfood — an actual device run — was
   the first thing to surface the issue, exactly as designed.

3. **Driver Protocol abstractions are cheap insurance.** Even though
   `MaestroDriver` had to be removed, the Protocol from #52 and the
   `(driver, …)` signature pivot from #53 are kept verbatim. If we
   ever pick up an Appium / browser-extension / bridge-based driver,
   the orchestration code in `visual_challenge.py` does not need to
   change.

## Options we considered (and rejected, for now)

- **A. Native HTTP bridge inside our sample app.** Add a WKWebView
  message handler + tiny HTTP server inside `SampleCaptchaApp`;
  `mk-qa-master` queries the bridge for DOM state. Works for apps we
  own. Does *not* work for the PRD's Tier 3 goal — driving the real
  Google reCAPTCHA demo page from a mobile WebView — because we don't
  control Google's iframe.

- **B. Switch driver from Maestro to Appium.** Appium speaks the
  WebDriver protocol and has full WebView JS execution. Solves the
  blocker outright. Cost: another driver implementation,
  appium-server install, and platform-specific drivers (XCUITest /
  UiAutomator2). Deferred — picking it up means committing the
  4–6 day rewrite the original v0.8.0 plan budgeted, but on a
  fresh foundation.

- **C. Hybrid Maestro-for-taps + Appium-for-JS.** Two CLIs to
  install, twice the failure surface, no real upside over plain B.

- **D. Park v0.8 mobile work.** Selected. The Driver Protocol stays;
  the rest is reverted; we honestly say "mobile WebView is not in
  v0.8."

## Forward-looking notes

- If a future contributor revives mobile, **start by re-running the
  Tier 1 dogfood against a real device** before any orchestration
  changes. Without that gate, the next iteration risks the same kind
  of false-positive cascade.
- The `examples/sample_captcha_mobile_app/` reference is left in
  place. It's a working WKWebView host for the v0.7 fixture and would
  be the natural starting point for either an Appium pivot or a
  bridge experiment.
- The PRD (`docs/prd-v0.8-mobile-webview-captcha.md`) is preserved
  unchanged as a historical artifact, with a banner at its top
  pointing to this postmortem.
