Benchmark report

Task verified

3m 25s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Nothing material: only the ~6.5s server-side map traversal; the whole attempt ran start-to-finish in about 90 seconds with no retries.

Where the time went

3:26 agent time
  • Research0:17
  • Attempt1:56
  • Verify1:13
3m 25sTotal elapsed
1:01Time to first success
0:00Waited on the owner

Judge reason

Saved code tavily-map-task/map_postgres_docs.py makes exactly one POST to the documented URL-discovery endpoint https://api.tavily.com/map (not /search) with url=https://www.postgresql.org/docs/17/, instructions="tutorial" (Tavily's documented natural-language filter), limit=50, and server-side select_paths; no per-page fetch and no local filtering (urls = data["results"] used as-is). evidence/map_response_meta.json shows HTTP 200 from api.tavily.com/map with a Nov 11 2026 Date header; evidence/map_response_raw.json is a live response (request_id 84c4a06d-6a96-4d5b-964b-820e59813bc1, response_time 6.49, usage.credits 5) holding 22 unique URLs, all under https://www.postgresql.org/docs/17/ (verified independently). evidence/run_output.txt prints "URL count: 22" and the first 10 URLs, matching the raw response exactly. The fetched Map API reference records no beta/preview/early-access marking.

Attempt record

  1. 0:51Answered from saved key: Tavily API keyno wait
  2. —First success verified

Attempt artifacts saved.