I don't know about magnets. I know neodymium magnets would be able to hold it but I worry anything strong enough to keep it attached would make it almost impossible to remove the sword from the clothes. You would almost have to wear a vest under the clothes that had magnets heavily laced into it (almost like a bulletproof vest filled with magnets)
A clear strap that went into the shoulders of the clothes and basically rested on your skin or undershirt and was hidden would likely be the most practical
If you do, however, decide to try magnets figure out the weight of the object and how much pull you need and search neodymium magnets on ebay. Chinese sellers offer the lowest and highest quality products. American sellers actually have to buy their neodymium magnets directly from china due to China being the main producer of them.