---
type: claim
title: Binary Preference Comparisons Cannot Identify Latent Preference Types, Making Pairwise RLHF Structurally Blind to Diversity
confidence: likely
---
This claim discusses the limitations of binary preference comparisons in identifying latent preference types, which makes pairwise RLHF structurally blind to diversity. The claim is supported by a formal identifiability analysis and mathematical proof detailed in Section 3 of the source paper. This directly challenges standard RLHF/DPO approaches, particularly in preference identification. Relevant Notes: This claim strengthens the argument against the universality of binary comparison methods in RLHF. Topics: AI alignment, preference diversity, RLHF limitations.