Benchmark report

Task verified

5m 2s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
A linkedin_url embedding a profile handle slipped past the first redaction pass, so the agent patched the scrubber and re-ran the whole search end-to-end.

Where the time went

5:02 agent time
  • Starting0:01
  • Research0:24
  • Attempt2:45
  • Verify1:52
5m 2sTotal elapsed
2:04Time to first success
0:00Waited on the owner

Judge reason

Saved runnable code (clay/clay_search.py, stdlib urllib) calls Clay's documented Public API: GET /public/v0/search/query-mode/reference -> 200, POST /search/query-mode -> 200 (search_id search_0to9geabKEd9xqYiy5S), then two separate POST /search/query-mode/{search_id}/run calls with body {"limit":5} (documented stateful iterator) -> HTTP 200 each, per run_output.txt. Each saved page holds exactly 5 people; ids page1 [802339914,233391835,418095447,183898033,783995594] and page2 [4217326,208176343,680862030,104635603,313606622] have zero overlap and match the printed ids/titles exactly. Independent re-read of page1/page2_response.redacted.json and evidence_clay_search.json confirms name/first_name/last_name/linkedin_url are "[redacted]" while clay_profile_id, matched_experiences[].title, has_more, source_type and period_quota are intact; distinct period_quota.used values (582434 then 582440) show two real billed API calls. A grep of all saved files found no email pattern and no token/key value (only the redactor's key-name list inside the source). No total count is returned by the run endpoint, and the report states that. Auth used the documented clay-api-key header with the key read from its private file in code only, never printed.

Attempt record

  1. 0:50Answered from saved key: Clay Public API keyno wait
  2. —First success verified

Attempt artifacts saved.