If the fabric doesn't really have finished edges then it won't fray.
As long as it's a higher grade glue than like school glue yes. I've glued Velcro down before, I think it was fabri-tac I used. But something that dries but still allows movement is probably the only thing i'd advise since this is going around parts of your body.
I second tape for the legs. If you just wrap them they're gonna constantly fall and if they're wrapped too tight they could cut off your circulation or possibly cut into your skin.
Tentative Debuts for 2013
Laxus Dreyar (Fairy Tail)