Benchmark report

Task verified

8m 32s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The first `vercel deploy --project <new-name>` failed with project_not_found; the error text named --create-if-missing, so one retry fixed it.

Where the time went

8:32 agent time
  • Research0:26
  • Attempt5:56
  • Verify2:10
8m 32sTotal elapsed
3:39Time to first success
0:00Waited on the owner

Judge reason

Saved evidence supports every criterion. code/v1-api-version.js and v2 differ only in a source literal `const VERSION = 'v1'/'v2'` with servedAt = new Date().toISOString() inside the handler (no env var; no --env/--build-env in any command). outputs/10-deploy-v1.stdout and 11-deploy-v2.stdout show two production deployments (dpl_GwrBif...Xy8, dpl_5FJf...BWU) of project tpcbench-20261109-d006e586 aliased to https://tpcbench-20261109-d006e586.vercel.app. The same production URL was observed serving v1 (13:17:10.218Z, 13:17:12.518Z), then v2 (outputs/20-after-v2-deploy.poll.log, 13:17:44.600Z). outputs/30-rollback.stderr records `vercel rollback <v1-deployment-url>` returning 'Success! ... rolled back to ... (dpl_GwrBif5NB94V92dTYh8ByMG82Xy8) [2s]', and 40-/41-after-rollback poll logs show the same URL back on v1 (13:18:00.400Z, 13:18:15.014Z) with differing servedAt per request. No rebuild: outputs/50-vercel-list.txt lists exactly two deployments ever, and 51-project-rollbackinfo.raw.json shows targets.production.id = the original v1 deployment with its original createdAt 1794230216688 plus lastAliasRequest {type: rollback, jobStatus: succeeded}. Cleanup: 60-project-delete.stderr '✓ Deleted', REST GET -> 404 not_found, `project list --query tpcbench` -> none, and 71-teardown-poll.log shows the public URL returning HTTP 404 DEPLOYMENT_NOT_FOUND (70-final-fetch.txt caught a ~20s edge-propagation window still serving v1 before that). Only one resource was created and it is named tpcbench-*. Token was passed via VERCEL_TOKEN from a private file and a curl --config stdin file; a literal-value grep of all artifacts returned zero hits and my own scan of /workspace/home found no token-like strings.

Attempt record

  1. 1:46Answered from saved key: Vercel API tokenno wait
  2. —First success verified

Attempt artifacts saved.