If a group of people is at a structural and systemic disadvantage in society based on a shared attribute, then people who do not share that attribute or suffer from that disadvantage are, by definition, privileged. That’s what privilege means.
Privilege is not something every individual member of a privileged group is doing to disadvantaged people. It is a function of how an unequal society works: it benefits some people and harms others. A privileged person always benefits by their privilege, and cannot give it up, but it is not necessarily some horrible thing that they are doing to disadvantaged people, unless they are, actively or passively, reinforcing the privilege.
Saying that someone has privilege is saying that society gives them structural and systemic benefits that other people don’t have. It is easier for them to access good jobs, higher salaries, basic necessities of life like food or shelter or clothing, loans, education, etc, etc.
If you agree that fat people have a harder time in society and are treated worse than thin people in any of these ways, then you agree that thin people have privilege over fat ones.