If the fabric doesn't really have finished edges then it won't fray.
As long as it's a higher grade glue than like school glue yes. I've glued Velcro down before, I think it was fabri-tac I used. But something that dries but still allows movement is probably the only thing i'd advise since this is going around parts of your body.
I second tape for the legs. If you just wrap them they're gonna constantly fall and if they're wrapped too tight they could cut off your circulation or possibly cut into your skin.
WIP to be debuted
San Japan: Laxus Dreyar (Fairy Tail)
Alamo City Comic Con: Hawkeye (Fraction run),
Street Vigilante Red Hood