I agree; I once did a rather similar uniform, and I use a medium-heavy weight cotton twill. It's thick enough to provide good coverage (as in, not see through, as cheap cotton plainweaves or broadcloths often can be) and drape, but not too heavy.
Another option would be a white wool suiting, but that is likely to be somewhat more expensive.
Pictures and tutorials from my costuming portfolio at Syagria's Cosplay Site!
Star Trek Original Series Uniform!
Original Creation for AB13
Most Recent Con:
PAX East, Anime Boston 2013