Yep, thrift stores can be your best friend. Most of my steampunk ensembles have at least an item or two scrounged up from the local Goodwill or DAV.
I'm not as familiar with men's clothing, but I looked at the basic shapes and designs of Victorian women's clothing and I just keep an eye out for items that work. For women's clothing, I've discovered that high-collared, loose-sleeved blouses and granny boots from the 1980's tend to work very well into a lot of steampunk costumes, and long, loose skirts can be worn under a bustle or pulled up into a bustle shape.
The same theory should be the same for men's clothing... get familiar with the style of shirts, vests, jackets, or pants you'd like to incorporate and keep an eye out for them. What style you're looking for will vary based on your personal aesthetic and the type of costume you're creating. Is it a down-and-dirty airship mechanic, or a wealthy aristocrat?
Like Ororo mentioned, our local university's theater department has an annual surplus costume sale that can be an awesome place to get discount pieces. You might give any schools or theatre companies in your area a call to see if they do something similar.
My Cosplay Adventures on Tumblr
Joyleaf - Elfquest || Belial/Mad Hatter - Angel Sanctuary
Surprise Hulk Variant - Marvel Comics || Black Widow - Hawkeye #9 || Mary Jane Watson - J. Scott Campbell version
Maevaris Tilani - Dragon Age Comics || Sugar Rush Racer Sticky Wipplesnit - Wreck-It Ralph