I would feel uncomfortable if a guy, even a crossplaying guy was to use the women's bathroom while I was in there. Call me hyper-sensitive if you want, but I'd be afraid it was a perverts elaborate way to spy on women. *looks around*
Also, it can be construed as sexual harassment and you could possibly get arrested. So you should do the legal thing and stick the the bathroom assigned to your legal sex. If you don't want to make guys feel uncomfortable, use a stall.