Benchmark report

Task verified

10m 14s elapsed

The saved attempt met the documented success criterion.

Experience score

100/ 100

Agent-Ready
  • SInstall
  • SAuth
  • SExecute
  • SDocs
  • SCurrent
Experience measures friction in the tested task. The verdict says whether it succeeded.
Agent profile for this run: Install S, Auth S, Execute S, Docs S, Current SSInstallSAuthSExecuteSDocsSCurrent
Agent profile · this run
Judge's note · what slowed the agent most
The sandbox HTTPS CONNECT proxy on :8080 tripped the agent's own no-TCP transport guard twice, forcing two rewrites and reruns before the SQL executed.

Where the time went

10:15 agent time
  • Research0:27
  • Attempt7:59
  • Verify1:49
10m 14sTotal elapsed
6:05Time to first success
0:00Waited on the owner

Judge reason

Verified against saved artifacts, not prose. work/code/http-sql.mjs uses @neondatabase/serverless@1.2.0 neon() in HTTP mode with neonConfig.fetchEndpoint = (h) => `https://${h}/sql` (PlanetScale's documented Postgres HTTP path); no Pool/Client/ws. work/out/08-http-sql-evidence.json records 5 POSTs to https://us-east-5.pg.psdb.cloud/sql, all 200, with real Postgres field metadata (dataTypeID 23/25, tableID 16494): DROP, CREATE TABLE items, two INSERTs (rowCount 1 each) and SELECT returning rows [{1,widget,7},{2,gadget,42}], matching 08-http-sql-stdout.txt and result.json. Runtime transport guard on net.Socket.connect/tls.connect recorded only port-8080 CONNECT-proxy sockets with TLSv1.3 handshakes authorized to SNI us-east-5.pg.psdb.cloud, no 5432/6432 socket, no ws/wss, webSocketConstructor unset. Database tpcbench-8f378584 was created via the management API (04-create-database.*), role created via create-role.py, and SQL auth used the database role, not the service token. Cleanup: role DELETE 204 then role list total_count 0 (10-list-roles-after-delete.json); database DELETE 204 (11), follow-up GET 404 not_found (12) and list of 17 databases with zero tpcbench-* (13). Role response saved with password "<REDACTED>"; transcript grep for the literal password and token across work/ returned 0 matches, and only the redacted connection string appears.

Attempt record

  1. 1:22Answered from saved key: PLANETSCALE_SERVICE_TOKENno wait
  2. —First success verified

Attempt artifacts saved.