Paper Summary: Non-transitive preferences

Yesterday I learned about non-transitive preferences, which I found very non-intuitive. You might assume that, if A > B and B > C, then necessarily A > C. This is transitivity, and it's intuitive. However, sometimes, C > A. This is the case in non-transitive preference. 1/

To credit, I lifted a lot of this info from this great page of a series by Peter Fishburn: https://www.eecs.uottawa.ca/ordal/papers/fishburn/node9.html

First, how can this happen? A simple example we know is Rock-Paper-Scissors (illustrated) Ok, you say: but this is a game, not real life. Surely all *my* preferences are transitive?

Now consider yourself in the morning and in the evening. Maybe in the morning, breakfast > beer. But in the evening, beer > breakfast. Ah, you say, there's a hidden variable there, the time of day! there are hidden variables everywhere for those with the eyes to see

Ok, then let's see about people in a simple study check out this except from the aforementioned page! ~1/3 college students (presumably intelligent, reasonable) had nontransitive preferences for a simple choice! Did they make a mistake? Maybe. But it still appears in the data.

Moreover, even if you allow an individual to be perfectly rational, you will still see nontransitive relations in a population. Voter 1 prefers A > B > C Voter 2 prefers B > C > A Voter 3 prefers C > A > B If you take an average, you'll get A > B > C > A.

Why is all of this important? RLHF often has implicit assumption of preference transitivity. Some people have found improvements if you allow for non-transitivity. See papers: https://arxiv.org/abs/2410.02197 https://arxiv.org/abs/2605.18721

I found this really interesting because I had no idea it existed before :) Hidden assumptions always stab us in the knee. Bitter lesson, etc. Entirely unrelated to RL: discussions about ELO ratings for games with nontransitive strategies! https://arxiv.org/abs/2206.12301

✨️ Nontransitive preferences ✨️