The headline sterility number is a rule, not a model

If you read the recent operating-room perception literature looking for the state of the art in sterility-breach detection, you arrive at one number: 0.768 on MM-OR. It appears identically for three different methods in the same table, which is the first thing worth noticing. A vanilla transformer, a latent scene-graph model and a topological method all score 76.83. Methods that different do not agree to two decimal places by coincidence.

They agree because the number is not theirs. Read at source, sterility in that table is a zero-shot rule-based spatial heuristic: a check for whether something non-sterile falls within a critical proximity threshold, computed over the 3D grounding stage that all three methods share. It is not a learned detector, and it is not what distinguishes those methods from each other.

What the number does not report

The evaluation is a timepoint macro-F1 over two classes, breach and no-breach. What is absent from the reporting matters more than what is present:

• no alarm rate, so a detector that fires most of the time is not penalised the way a clinical user would penalise it
• no episode-level readout, so one breach spanning forty seconds and forty separate breaches are not distinguished
• no chance control
• no threshold-selection procedure, so the proximity threshold that produces 0.768 is not shown to generalise off the data it was chosen on

We reimplemented the rule and scored it in the same currency, and it lands within a rounding error of the published figure. That is the honest ceiling of the comparison: a heuristic agreeing with a heuristic tells you almost nothing about whether either would be trustworthy in a room.

Why we published this rather than beating it

Beating 0.768 would be easy and would mean nothing. A model that alarms aggressively can win a timepoint macro-F1 over two classes while being unusable, because the currency hides the one quantity a circulating nurse cares about: how often the system interrupts them and is wrong.

So we do not treat it as a target. The evaluation we hold ourselves to is episode-level, reports alarm rate alongside recall, and is frozen before a run rather than chosen after one. When we have numbers in that currency worth showing, they will appear here with the losing runs reported at the same prominence as the winning ones.

The general point is not about one table. Published benchmarks are instruments, not finish lines. They are the only external evidence we have about whether a model is good, and they are worth exactly as much as the currency they are denominated in.