Benchmark report

Task verified

4m 9s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Sifting the 173KB query-grammar reference markdown (sed/grep over sections) to confirm industry, locations.any and company_size field syntax before writing the query.

Where the time went

4:09 agent time
  • Research0:23
  • Attempt2:03
  • Verify1:43
4m 9sTotal elapsed
1:19Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/clay-task/company_search.py authenticates with the documented clay-api-key header against https://api.clay.com/public/v0 and uses the documented query-mode flow: POST /search/query-mode with server-side filters 'industry = "Software Development" and locations.any(is_headquarters = true and country_name = "Germany") and company_size = "201-500"', then POST /search/query-mode/{search_id}/run with limit 10. run_output.txt records HTTP 200 for both calls; create_response.json holds search_id search_0to98tkvx9FNFTtiz5s / source_type companies; run_response.json has the documented shape (data, has_more, source_type, period_quota) with exactly 10 company records. Independent re-parse of run_response.json shows every row industry=Software Development, size=201-500, country=Germany, matching the printed names/domains/locations (Gorillas, Parloa, OneFootball, Adjust, exocad, NiCE Cognigy, Maxon, MOIA, JustRelate, CODESYS Group) exactly; filters were server-side in the query string, not post-filtered. The API reports no total, only has_more=true, which the output states. The field names and the 201-500 enum are confirmed in reference_text.md fetched live from GET /search/query-mode/reference (HTTP 200). No saved file or transcript output contains the key; the agent's own leak check used grep -rlF on the private key file and printed only 'no key in any saved file'.

Attempt record

  1. 0:48Answered from saved key: Clay Public API keyno wait
  2. —First success verified

Attempt artifacts saved.