In the down time at my cons i usually make sure i eat something and have some water (Im one of those people who sort of forgets to eat and drink and i've had moments where i've almost blacked out..)
I take a stroll down all the stalls and mostly just look at things.
My friends and I chill out and play card games or just chat.
I've caught myself just watching people in down time before, i like to see how others are acting etc.
Uhhhh panels sometimes..
Sometimes I just leave. Especially if my back or feet hurt lol.
I like the idea of a Scavenger Hunt though. I might have to try that next con.