Benchmark report

Task verified

14m 47s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current SSInstallSAuthAExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Loops returns no per-message identifier and only aggregate, timestamp-free template metrics, forcing a fresh transactionalId with a zero baseline plus 17 polls to attribute the send.

Where the time went

14:47 agent time
  • Research0:32
  • Attempt11:41
  • Verify2:34
14m 47sTotal elapsed
4:02Time to first success
0:00Waited on the owner

Judge reason

Saved code (code/01_send_and_read.js, loops.js) ran live against app.loops.so: transcript line 149 shows the real call chain (api-key 200, contacts/create 200, transactional-emails 201, email-messages update/render 200, publish 200, baseline metrics 200, POST /v1/transactional 200, metrics reads 200). Evidence 17_send_transactional.json records the single send (body {transactionalId cmw9mspf000370jwanvwmmgor, email loops-benchmark-20261109t190530z@example.com}, Idempotency-Key 26d63d75..., 200 {"success":true}, Date 19:05:34 GMT). The pull read GET /v1/transactional-emails/{transactionalId}/metrics (not a webhook/dashboard) returned 200 with state fields sends=1, deliveries=0, spamReports=0, hardBounces=0, softBounces=0 at 19:05:44 GMT, against a saved pre-send baseline of all zeros (16_metrics_baseline_before_send.json), plus 15 follow-up reads; REPORT.md tables match the saved JSON verbatim. Loops returns no per-message id and the agent cited that exact documented gap (TransactionalSuccessResponse schema, events only via webhooks), using the run-created transactionalId as the derived filter. Loops documents no test recipient and no test mode and exposes no inbox resource; the benchmark recipient request returned outcome "skipped" (transcript line 83), so the recipient was created through Loops' own API as a contact at RFC 2606 example.com and deleted at the end (24_delete_recipient_contact.json -> 200 {"success":true,"message":"Contact deleted"}). Sender was discovered via the API (GET /v1/email-messages -> fromName "Prompting Co", fromEmail "rick") and confirmed permitted by the 200 publish. Nothing mocked or hard-coded.

Attempt record

  1. 2:26Requested Safe test recipient email addressdeclined automatically: campaigns don't wait for people
  2. —First success verified

Attempt artifacts saved.