Growing up I was ashamed. I never made any comments about who I found attractive. Friends would make comments about some girl being 'fit' but it never ever registered for me. The only women I ever found attractive were big mature women. At some point in my teens my Dad found my porn stash on his PC and we had a talk about it saying "It's ok, your uncle Frank was 'into fatties' as well".
After that I became more comfortable with my sexuality and what I'm attracted to. A friend asked me if I was gay in my late teens because I never made comments on women and would go quiet whenever sex talk started. I confided in him that I just preferred fat girls and he laughed. He told everyone.
But by that point, I'd grown out of trying to appease everyone and I've got thick skin. I can take teases and jabs about it, it's not going to change my interests and if someone was trying to actually insult me about it its not like that person was ever going to be a friend anyway so why let it bother me?
It worked out though, because in uni I met my ex (5' 11" 300lbs) and we ended up dating for 12 years. All my mates knew what to expect so it never became an issue, neither did it matter when I introduced her to colleagues. Nobody ever said anything to my face.
(The relationship ended because I wanted kids and she didn't, but we've still met up for the odd shag - its complicated)