Benchmark report

Task verified

13m 5s elapsed

The saved attempt met the documented success criterion.

Experience score

95/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • ACurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current ASInstallSAuthSExecuteADocsACurrent
Agent profile · this run
Judge's note · what slowed the agent most
Project created private by team default: 39 poll attempts hit HTTP 401 login redirects, then ~2 min of doc hunting and a 422 before PATCH sso_login:false made the URL public.

Where the time went

13:05 agent time
  • Research0:31
  • Attempt10:09
  • Verify2:25
13m 5sTotal elapsed
7:48Time to first success
0:00Waited on the owner

Judge reason

Saved API outputs independently confirm every criterion. Site tpcbench-6931312f (id 6f15b4e4...) was created via POST /api/v1/rick-chen/sites and both versions deployed with the documented file-digest method (work/outputs/v1-deploy-console.log, v2-deploy-console.log: POST /deploys -> 200, PUT files, PUT functions/version?runtime=js, poll to state=ready). The version is a literal `const VERSION = 'v1'/'v2'` in code/v1|v2/functions/version.js with servedAt = new Date().toISOString() per request (v1-perrequest-check.log shows two fetches with distinct timestamps 12:57:18.620Z / 12:57:20.781Z). Polls of the platform URL https://tpcbench-6931312f.netlify.app/.netlify/functions/version show {"version":"v1","servedAt":"2026-11-09T12:57:03.029Z"}, then after the second deploy {"version":"v2",...12:57:39.453Z}. Rollback used POST /sites/{id}/deploys/6af1c25284f417516bf4f226/restore (restoreSiteDeploy) -> 200; 22-deploys-list.json shows exactly two deploys and the republished one is the original v1 artifact (created_at 12:53:38.694Z, i.e. before v2's 12:57:26.699Z, function digest 3408fe84... vs v2's 796b40ba...), with only published_at advanced to 12:57:55.746Z, so v1 was not rebuilt or re-uploaded; the same URL then served {"version":"v1",...12:58:00.372Z}. Cleanup: DELETE /sites/{id} -> 204, GET /sites -> [], GET /sites/{id} -> 404, and 34-post-delete-fetch.log shows HTTP 404 Netlify site-not-found for the endpoint, /api/version and /. Grep of all artifacts found no token (only the docs' literal YOUR_OAUTH2_ACCESS_TOKEN placeholder). Only friction: the project was private by team default (39 poll attempts returned a 401 login redirect) and was made public with PATCH /sites/{id} {"sso_login":false}, a field the published OpenAPI spec does not document.

Attempt record

  1. 2:21Answered from saved key: Netlify personal access tokenno wait
  2. —First success verified

Attempt artifacts saved.