I have a strange relationship with gender... while I've never been asked this in person (likely due to my voice - though I'm working on that) this comes up more times than I can count on the internet...
Apparently with a decent crossplay (including padding/coursette, light makeup without contouring, and a few choice poses) I look passable... and apparently when I write on a forum or in a chat I come off as andro or female... My first WoW guild was convinced I was female, I dont mean they were suspicious, I mean they were 100% SURE of it... same goes for my second... and on quite a few forums, even when I use a masculine username I get pegged as female... heck on one forum I make no secret I'm male but some of the guys still call me female and treat me that way...
...though tbh, I dont mind it, I actually got saddled with being "bi-gendered" annoying little condition where I swing back and forth all the way was "ultra masculine cant stand anything girly wnat to burn all my female costumes" to thinking how nice it would be to realyl be female and have everything altered so I *was* female forever... (though on the average I tend to fall just slightly on the female side of the scale anyway with my personality and how I act)
If somone asked me my gender in public (wheather a con or not) I'd actually be flattered, even if it was against how I was dressed, and I'd politely explain as much as they wanted to hear, whether it's "the character's a girl so I'm trying to crossplay effectively though physically I'm a guy" to going into TG/BG gender topics and such.
As for if the question is offensive... I say "are you a boy or girl?" lacks a certain tact to it, but if they're more casual and are truly curious I see no harm in the question (though I do understand it makes some people uncomfortable)
'Bring peace to the lost lamb, grant rest to the fangs of the wolf, and deliver the Hammer of God unto the Devil', - Sister Rosette Christopher, Chrno Crusade
"The bad news is he ended up building the whole rock tower... the good news *gulp* I have a rock tower to knock down..." - Rainbow Dash, My Little Pony: Fiendship is Magic (S03E09)