I'd suggest using buckram or even Fosshape. I've never made a hat before though, so I can't recommend a pattern for that particular shape.
Fayt Leingod (Ethereal Blast version) - Star Ocean:Till the End of Time - 85% finished
Fidel Camus - Star Ocean: Integrity and Faithlessness - planning stages
Yoshino - Date A Live - planning stages