Do what makes you comfortable. You plan on toning up a bit more but in the meantime if you can wear it slightly modified and want to, go ahead you aren't doing this for the naysayers. As far as recognizability, I think so. I mean how many characters have that outfit/look like that?
They probably said that so it just looks like skin instead of a white "shirt". I could be wrong though.
WIP to be debuted
San Japan: Laxus Dreyar (Fairy Tail)
Alamo City Comic Con: Hawkeye (Fraction run),
Street Vigilante Red Hood