For an application like this, it probably won't see all that much wear, so yeah, rubber cement should work just fine. However, many of the rubber cement brands (e.g. Elmer's) have reformulated in the past couple decaes, I believe largely to reduce instances of huffing, and to a lesser extent, to improve the ability to make temporary bonds. Industrial-type "contact cements" tend to work a bit better for permanent holds. It's sold in the adhesive section of the hardware store.
In any case, the process is important. Brush the stuff on both surfaces, let the solvents flash off (dry) and them press the surfaces together firmly. The pressure ("contact") is what activates the chemical bond of the adhesive. When working with heavier leathers, it is common to hammer the surfaces together with a wooden or rawhide mallet.