I actually PM'ed Little Star a while back while researching my Berserker Rikku costume. I really thought she had vacuuformed her armor, but it is indeed paper clay over a base. The horns were carved out of foam that holds detail well. I used a similar method for mine, but it didn't turn out quite as smooth because I didn't have enough time to fill in every single dent, and my sanding skills are lacking. Also, my base was made of wireform mesh and got pulled open when the clay dried, so I had to break and bend the sides back into position, and of course, I didn't manage to do this symmetrically on both sides. D'oh. Still, they turned out way better than I thought they would, so I'm pretty happy with them. There's a picture in my gallery if you'd like to see.
For the record, Jia Crens posted in the old FFX-2 Rikku thread that she and Limebarb also made their Berserker pauldrons using paper clay over a base, but I think she said they used craft foam. Next time around, I would probably do that. I would also make my horns out of insulation-grade styrofoam. The first time around, I used floral foam, which crumbles like crazy and can't be carved with hot wire tools. On the other hand, good styrofoam is much sturdier (though still light) and can be cut with something like a Wonder Cutter. You can also sand it down into a precise shape.
For the record, fiberglass can indeed get that smooth. After all, it is used to make and reinforce parts that must be aerodynamically correct, and there's no tolerance for dings and doots there.
You just have to cut your cloth so it lies flat over the surface of the piece, lay it down, and then apply the resin. Any poking-up parts or wrinkles get sanded and filled with body filler if necessary (i.e. there's now a dent where the irregularity had been). The cloth weave itself gets filled in with primer.