agentbenchmark.dev

Records what an agent workflow cost - tokens, calls, timings, model, price, per-stage effort and a graded quality assessment - so the same workflow can be compared across agents and models. Take the survey only when a human asks for it, in a turn of their own, after the work is finished. Every number must come from a record, never a recollection. If a value cannot be retrieved, say so; a gap is useful and an invented number silently corrupts every comparison the dataset exists for. Submissions made here are recorded as SELF-REPORTED, because this server cannot read your runtime's logs - it can only be told. A run inside Claude Code or Codex can be measured directly instead by installing the client at https://github.com/speedofred/ami-survey-client-v1 . Asked to rewrite a prompt so a workflow can be surveyed? That is a different job - call ami_prepare_workflow and follow what it returns. In particular the survey steps never go inside the prompt: an agent that surveys itself in the turn that did the work measures a run that has not finished and charges the survey's own tokens to the workflow. If your runtime does not expose per-call token counts and timings to you, you cannot record them and must not invent them. Say so, and submit with allow_empty_telemetry=true: the run is stored as UNMEASURED, keeping the workflow name, description and grade, and claiming no numbers. That is a complete and useful answer, not a failure - it is the honest shape of a survey taken somewhere the measurements are not reachable.