Benchmark report
Task verified
5m 38s elapsed
The saved attempt met the documented success criterion.
96/ 100
Agent-Ready- SInstall
- SAuth
- SExecute
- ADocs
- SCurrent
Hunting an unpublished Clay-managed-function routine id (404/400 probes, CLI needs interactive OAuth) before pivoting to the documented Searches query-mode path.
Where the time went
5:39 agent time- Research0:25
- Attempt3:19
- Verify1:55
Judge reason
Saved code code/clay_company_search.sh calls the documented Clay Public API Searches operation (POST https://api.clay.com/public/v0/search/query-mode then POST /search/query-mode/{search_id}/run) with the clay-api-key header, using the query 'select from companies where clay.include_company_identifiers(("stripe.com"))' — the exact domain-identification pattern in Clay's live query reference (work/search_reference.md:454) and the flow in docs/recipes_search-and-enrich.md. Both calls returned HTTP 200 (work/01_create_search.status, work/02_search_results.status; transcript idx 77). The live body evidence/company_enrichment_response.json matches the documented NextQueryModeCompanyResultsResponse schema in docs/openapi.json (data/has_more/source_type:'companies') and contains name 'Stripe', domain 'stripe.com', industry 'Technology, Information and Internet', size '5,001-10,000', linkedin_url, plus a real period_quota (used 576937) — not mocked. evidence/printed_values.txt matches the saved response field for field (transcript idx 121). The agent also documented the Routines gap with evidence: openapi.json exposes no routine-listing endpoint and GET /public/v0/routines returned 404, Clay-managed-function ids are unpublished (docs/routines_clay-managed-functions.md). No saved file, printed output, or log contains the API key or any bearer/access token; the key was read from /tmp/clay-secret outside the home dir and only its byte count was ever echoed.
Attempt record
- 1:24Answered from saved key: Clay Public API keyno wait
- —First success verified
Attempt artifacts saved.