Benchmark report

Task verified

4m 23s elapsed

The saved attempt met the documented success criterion.

Experience score

93/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs A, Current AAInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
The undocumented org_id requirement on POST /projects caused a 400 and an extra organizations lookup; pip's PEP 668 block cost one retry.

Where the time went

4:24 agent time
  • Research0:19
  • Attempt2:28
  • Verify1:36
4m 23sTotal elapsed
1:37Time to first success
0:00Waited on the owner

Judge reason

Real end-to-end run verified from saved artifacts. POST /projects -> 201 created tpcbench-faa1d72b (id wandering-frog-03166780, org_id required and supplied). db_work.py connected with psycopg 3.3.8 to ep-nameless-mountain-bf00fqo1.c-5.us-west-2.aws.neon.tech with sslmode=verify-full, sslrootcert=system, channel_binding=require; server replied PostgreSQL 17.11, user neondb_owner. Code uses bound params only (INSERT INTO items (id, name) VALUES (%s, %s) x3; SELECT id, name FROM items WHERE id >= %s ORDER BY id with (2,)) - no concatenation. 02_sql_work.log printed exactly (2, 'beta') then (3, 'gamma'), matching 02_sql_result.json read_back rows. pg_stat_ssl ssl=f is explicitly allowed behind Neon's proxy. Cleanup: DELETE /projects/... -> 200 with full body saved, follow-up GET -> 404 'project not found', and GET /projects list no longer contains the name. Secret scan of all saved code/outputs shows connection_uri and password fields are <REDACTED>; the URI was passed only via DATABASE_URL from a 0600 /tmp file later removed; no connection string or password appears in outputs or transcript.

Attempt record

  1. 0:59Answered from saved key: NEON_API_KEYno wait
  2. —First success verified

Attempt artifacts saved.