Whatever, fuck 'em. People say shitty things all the time, so don't let them bother you. Go to any video of anybody famous on youtube and there'll be dozens of comments about how so and so sucks. It could be somebody awesome and world famous, talented...doesn't matter.
As for fitness, you're not obligated to look any way or be in any kind of shape that you, yourself, do not want to be whether for health reasons, personal reasons, etc. If you like going to the gym, by all means, do so! It's fun! I'd be in the gym all day if I didn't have to work. But don't think you have to do it to please anybody but yourself.
current rotation: Sweden, England, Guts, Tony Stark, Netherlands, Blackwall, Nephrite, Aragorn, McCree
future stuff: Genjyo Sanzo, Wolverine, Saionji Kyoichi, Tatewaki Kuno, Nakago, Auron, Dorian, Hawke (rogue)
past stuff: a lot