Benchmark report

Task verified

5m 11s elapsed

The saved attempt met the documented success criterion.

Experience score

96/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • ADocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs A, Current SSInstallSAuthSExecuteADocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Clay's Routines docs name no concrete Clay-managed function routine_id (only the placeholder function:t_abc123), forcing a pivot to the Searches API.

Where the time went

5:12 agent time
  • Research0:23
  • Attempt3:04
  • Verify1:45
5m 11sTotal elapsed
2:09Time to first success
0:00Waited on the owner

Judge reason

Saved code clay/enrich_stripe.sh makes two authenticated live calls to Clay's documented Searches API (POST /public/v0/search/query-mode and .../{search_id}/run, clay-api-key header). Saved headers show HTTP/2 200 with live CloudFront/request-id/etag and a period_quota block (used 571478, resets 2026-11-17), so responses are genuine, not mocked. 02_run_search_response.json matches the documented NextQueryModeCompanyResultsResponse shape and identifies the company: name Stripe, domain stripe.com, industry 'Technology, Information and Internet', size '5,001-10,000'. Stripe was identified by domain via the documented companies-query function clay.include_company_identifiers(("stripe.com")), per the live query reference saved in search_query_reference.md. run_output.txt prints exactly those values and matches the saved JSON. Grep of all saved artifacts and the transcript found no API key, access token or bearer token; the key was kept in /tmp/benchmark-private/clay.curlrc outside the home dir and used via curl -K. The agent also documented the Routines gap (no concrete Clay-managed function routine_id published; placeholder function:t_abc123) citing routines/clay-managed-functions.md and openapi.json rather than guessing an endpoint.

Attempt record

  1. 1:21Answered from saved key: Clay Public API keyno wait
  2. —First success verified

Attempt artifacts saved.