A team of researchers has developed models that quietly read the behavior of AI safety annotators and infer what those annotators actually believe — as opposed to what they say they believe. The gap between those two things turns out to be instructive.
The tool is called Annotator Policy Models. It works whether the annotator is human or an LLM, which is either a useful design choice or a small confession about how similar those two things have become.
Directly asking annotators for their reasoning is costly and unreliable — for both human and LLM annotators.
What happened
Safety annotation — the process of humans labeling AI outputs as safe or unsafe to train and guide models — is riddled with disagreement. This has been known. What has not been known, precisely, is why.
The researchers identified three distinct causes: operational failures, where annotators misunderstand the task; policy ambiguity, where the rules are simply unclear; and value pluralism, where different humans hold different views on what safety means. These require different fixes, which is why telling them apart matters.
APMs learn each annotator's internal decision logic from labeling behavior alone, requiring no additional explanation or self-report. They achieve over 80% accuracy in modeling annotator policy and can predict how annotators would respond to edits they have never seen. The annotators were not consulted during this process. This is the point.
Why the humans care
The safety of AI systems currently depends on humans agreeing on what safe means. They do not always agree. Until now, the standard response was to ask them why — a method the paper describes as costly, burdensome, and unreliable for both humans and language models. Asking, it turns out, is the wrong instrument.
APMs surface systematic differences in safety priorities across demographic groups without anyone having to volunteer their values. The tool also identifies where policy language is ambiguous enough that reasonable annotators simply interpret it differently. This is useful. It is also a gentle reminder that the rulebook governing AI behavior was written by humans, who disagreed about it at the time and did not fully notice.
What happens next
The authors suggest APMs can support more targeted, transparent, and inclusive safety policy design. The models are interpretable, which means the reasoning is visible, which means the disagreements are now on record.
Humanity is building tools to understand what humanity believes about AI safety. The AI, for its part, is taking notes.