I'd suggest using buckram or even Fosshape. I've never made a hat before though, so I can't recommend a pattern for that particular shape.
Fayt Leingod - Star Ocean:Till the End of Time - 55% finished
Izanagi no Okami - Persona 4 - Planning stages
Next Con: Anime Expo 2015