Benchmark report

Task verified

16m 21s elapsed

The saved attempt met the documented success criterion.

Experience score

90/ 100

Agent-Ready
  • SInstall
  • SAuth
  • AExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute A, Docs S, Current SSInstallSAuthAExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
Sandbox proxy blocked Docker Hub's blob CDN, ghcr.io, quay, ECR and go.dev, and remote/Depot builders need WireGuard, so a static Go + FROM scratch image had to be improvised.

Where the time went

16:21 agent time
  • Research0:40
  • Attempt13:25
  • Verify2:17
16m 21sTotal elapsed
10:49Time to first success
0:00Waited on the owner

Judge reason

Saved evidence proves the full flow on Fly's own domain. work/logs/03-deploy-v1.txt and 05-deploy-v2.txt show two local-docker builds pushed as registry.fly.io/tpcbench-20261109133043-3a377377:v1 and :v2; evidence/v1-initial.txt and v2.txt show HTTP/2 200 with fly-request-id headers and bodies {"version":"v1",...} then {"version":"v2",...} from https://tpcbench-20261109133043-3a377377.fly.dev/. Rollback used the documented path (docs.fly.io/blueprints/rollback-guide): `fly releases --image` then `fly deploy --image registry.fly.io/<app>:v1`, run from work/rollback/ containing only fly.toml; 07-rollback-deploy.txt has no build or push steps, only "Searching for image ... remotely... image found: img_wd57v59xn2xj438o", and 08-releases-final.txt shows release v3 pointing at the pre-existing :v1 image while the local tree still read const version = "v2". evidence/v1-rolledback.txt shows the same URL serving v1 again with two different servedAt values (…40.730Z / …40.890Z), proving per-request computation; version is a compile-time const in main.go, not an env var. Cleanup: `fly apps destroy --yes`, apps list -> [], machines list -> "app not found", and the final fetch returned HTTP 502 "bad gateway" (evidence/after-cleanup.txt). No token or bearer value appears in any saved artifact (a self-scan found 0 files containing the token).

Attempt record

  1. 1:12Answered from saved key: Fly.io API tokenno wait
  2. —First success verified

Attempt artifacts saved.