Benchmark report
Task verified
4m 53s elapsed
The saved attempt met the documented success criterion.
94/ 100
Agent-Ready- AInstall
- SAuth
- SExecute
- ADocs
- SCurrent
The PEP 668 pip block forcing a venv, plus one docs page (cloud/guides/browser-api) that lacked the CDP/Playwright auth example and required a second fetch.
Where the time went
4:54 agent time- Research0:24
- Attempt3:13
- Verify1:16
Judge reason
Saved code (code/browser_use_session.py) does the documented round trip with no mocks: POST https://api.browser-use.com/api/v4/browsers with the X-Browser-Use-API-Key header, Playwright chromium.connect_over_cdp(cdpUrl, headers={'X-Browser-Use-API-Key': key}), and a finally block PATCHing /api/v4/browsers/{id} {"action":"stop"}. Live run output (artifacts/run_output.txt, echoed in transcript tool result at the run step) shows HTTP 201, SESSION ID 4e749953-c64f-49e4-8168-3f69ad24cff6, CDP URL https://4e749953-...cdp.browser-use.com, PAGE TITLE: Example Domain, and TERMINATION HTTP STATUS 200 / state stopped / finishedAt 2026-11-09T10:37:13.059885Z. Saved create_session_response.json (201, id + cdpUrl, startedAt 10:37:11) and stop_session_response.json (200, status stopped, browserCost) match the printed values exactly; page_result.json records url https://example.com/ and title Example Domain. No credential appears in any saved file (liveUrl saved as ?wss=<REDACTED>; scrub() strips the key and all query values; the agent's byte-level scan of all 7 deliverables found zero hits, and my own read of every artifact shows no key).
Attempt record
- 1:48Answered from saved key: Browser Use Cloud API keyno wait
- —First success verified
Attempt artifacts saved.