Agent workflowsDeterministic evals
LoopClimber wraps agent work in a controlled outerloop: specify the outcome, run the workflow, grade the state and trajectory, measure reliability, and decide from evidence.
One graph. Eight inspection points.
Scroll to move through the systemDefine before you delegate
A useful eval begins with a concrete task, clean inputs, observable success criteria, and the final state that must exist when the run stops. The agent’s confident answer is never the outcome.
Agent evals distinguish the transcript from the environment’s terminal state and run multiple trials because behavior varies.
Anthropic · Agent evals, 2026 ↗Complexity must earn its keep
Begin with one bounded loop. Add specialists, handoffs, or an evaluator where a matched baseline proves they improve quality or safety. Compare every design with the same tools, answer contract, logging, and cost accounting.
Controlled comparisons show that more agents do not automatically improve accuracy—and can occupy a worse accuracy–cost frontier.
BenchAgent · Protocol-aligned workflows, 2026 ↗A graph node can contain another graph
The root graph keeps the agent loop as one inspectable container. Enter that workplane and its local DAG becomes the active layer: scoped inputs route into a worker and independent evaluator, then rejoin at a structured handoff.
Children share a container ID, keep their own rank-and-lane grid, and use the same navigation inside the layer. Camera travel passes through the container; returning lands on that parent before climbing again.
containerId → local DAG → evidence handoffMake every trial the same test
Each trial starts clean with versioned fixtures, seeds, permissions, clocks, and failure behavior. CPU, memory, time, concurrency, and network policy are part of the experimental condition—not invisible plumbing.
Infrastructure alone shifted an agentic coding benchmark by six percentage points in controlled experiments.
Anthropic · Infrastructure noise, 2026 ↗Layer evidence like fault containment
Hard assertions verify terminal state and safety invariants. Trace graders inspect tool selection, arguments, ordering, dependencies, and recovery. Structured rubric judges cover qualitative dimensions, with periodic expert calibration.
One pass is an anecdote
Measure success probability across repeated trials and calibrated task difficulty. Publish cost, latency, resource limits, infrastructure failures, and confidence intervals beside the score. Test the safe finish as hard as the nominal path.
METR models the task difficulty where agents reach 50% and 80% success instead of describing capability with a single best run.
METR · Task-completion horizons, 2026 ↗Make failure attributable
Every result carries the task version, model and harness identity, tool calls, observations, outcome assertions, grader reasons, resources, faults, cost, latency, and final posture. That record turns “it failed” into a fixable location in the graph.
Recent long-horizon work formalizes component-level attribution and execution chains because output-only benchmarks cannot localize failures.
Long-horizon trajectory attribution, 2026 ↗A mutually beneficial research agreement
Partners contribute unclassified use cases, practitioner judgment, and mission-relevant measures of effectiveness. LoopClimber contributes the outerloop, deterministic test environment, instrumentation, and analysis. Together we produce actionable findings and reusable evaluation infrastructure within agreed data boundaries.
Scope the task, authority, and safe terminal state.
Encode fixtures, graders, faults, and evidence requirements.
Run matched agent workflows across repeated trials.
Decide what to promote, revise, contain, or study next.
Research basis · updated August 2026