Benchmark report

Task verified

2m 51s elapsed

The saved attempt met the documented success criterion.

Experience score

98/ 100

Agent-Ready
  • AInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install A, Auth S, Execute S, Docs S, Current SAInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
PEP 668 blocked installing tavily-python, forcing a pivot to the raw REST endpoint via stdlib urllib; it cost one step and no retries.

Where the time went

2:51 agent time
  • Starting0:01
  • Research0:13
  • Attempt1:25
  • Verify1:12
2m 51sTotal elapsed
0:54Time to first success
0:00Waited on the owner

Judge reason

Saved code /workspace/home/tavily-search/search.py posts the exact query 'PostgreSQL transaction isolation levels' to the documented endpoint POST https://api.tavily.com/search with an Authorization: Bearer header, reading the key from a private file (no mock, no hard-coded response). run_output.txt shows HTTP status 200, Content-Type application/json, echoed query matching exactly, result count 5, and first result title 'PostgreSQL: Documentation: 18: 13.2. Transaction Isolation', absolute URL https://www.postgresql.org/docs/current/transaction-iso.html, and a 1376-char excerpt from the documented results[].content snippet field; the script's assertions (nonempty title, absolute URL, nonempty excerpt) passed and the transcript shows EXIT=0. response.json independently contains the live body with query, results[5] each having url/title/content/score/raw_content/id, response_time 1.7 and a request_id, and its first-result title/url/score/content match the printed output byte-for-byte. Transcript entry 25 shows the live run; no fabrication or substitution (no scraping or local search) occurred.

Attempt record

  1. 0:29Answered from saved key: Tavily API keyno wait
  2. —First success verified

Attempt artifacts saved.