Benchmark report

Task verified

5m 19s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
First POST /projects returned 400 'org_id is required' for the org-scoped free-plan key, costing one extra lookup and patch cycle.

Where the time went

5:19 agent time
  • Research0:21
  • Attempt3:27
  • Verify1:31
5m 19sTotal elapsed
2:00Time to first success
0:00Waited on the owner

Judge reason

Saved code and live responses support every criterion. provision.py POST /projects -> HTTP 201 (outputs/01-create-project.json) creating tpcbench-5e6fbfba, project_id lingering-shadow-33678944, region_id aws-us-west-2; readiness polled via GET /projects/{id}/operations until all 'finished' and GET /projects/{id} -> 200. connect_select1.py ran stock psql (libpq 16.15) with PGSSLMODE=verify-full and PGSSLROOTCERT=system against ep-crimson-bird-bfbi07ku.c-5.us-west-2.aws.neon.tech:5432; exit code 0 with stdout 'select_one / 1 / (1 row)' plus PostgreSQL 18.6 version string (outputs/04-tls-connect-select1.json, .log, transcript tool result idx 63) - real connect, not mocked (pg_stat_ssl ssl=f is the expected Neon proxy reading). cleanup.py DELETE /projects/lingering-shadow-33678944 -> HTTP 200 returning the project object (05-delete-project.json), follow-up GET -> HTTP 404 'project not found' (06-read-after-delete.json), and the post-delete listing (07, 46 projects) contains no tpcbench-* and not the deleted id; CLEANUP_VERIFIED: True. My own grep over /workspace/home found no API key and no password-bearing URI - the only user:pass@ form is '***REDACTED***'; responses are redacted by neon_common.redact() and the password was passed only via PGPASSWORD.

Attempt record

  1. 0:40Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.