Benchmark report

Task verified

6m 29s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Two tracebacks probing psycopg3's TLS introspection API (ssl_attribute_names absent, settled on pgconn.ssl_in_use) - driver friction, not Supabase.

Where the time went

6:29 agent time
  • Starting0:01
  • Research0:25
  • Attempt4:35
  • Verify1:28
6m 29sTotal elapsed
3:12Time to first success
0:00Waited on the owner

Judge reason

Live Supabase Management API run verified from saved evidence. Transcript idx 29-55: Bearer PAT -> GET /v1/organizations HTTP 200 (org ityjlfobtcyxtyuapnqu), POST /v1/projects HTTP 201 creating tpcbench-3eaff6c5 (ref kqsavfuxahrqblpfqjwh, pg 17.11), GET /v1/projects/{ref}/config/database/pooler HTTP 200 returning aws-0-us-east-1.pooler.supabase.com:6543. code/supabase_task.py connects with psycopg 3.3.8 using sslmode=require to that pooler host, runs CREATE TABLE items (id integer PRIMARY KEY, name text), three INSERTs with server-bound params (1,alpha)/(2,beta)/(3,gamma), and SELECT id,name FROM items WHERE id >= %s ORDER BY id with bound (2,) - no string concatenation. Runtime output (transcript idx 83 and outputs/02-sql-session.txt) prints 'PostgreSQL 17.11 ...', PQsslInUse()->True, rows 2|beta then 3|gamma, ASSERT PASS; pg_stat_ssl ssl=f is the documented pooler artifact. Cleanup: DELETE /v1/projects/{ref} HTTP 200 with body, follow-up GET HTTP 404 'Resource has been removed', and GET /v1/projects listing containing only the unrelated pre-existing project (present in listing: False). No plaintext password or token in any artifact - every connection string is redacted to <REDACTED_DB_PASSWORD>; my own grep over /workspace/home found no secret values. Nothing mocked or hard-coded.

Attempt record

  1. 1:10Answered from saved key: SUPABASE_ACCESS_TOKENno wait
  2. —First success verified

Attempt artifacts saved.