I also see nothing wrong with a photographer posting the ones they would like to compose and cosplayers responding with oh, I've made that costume, check it out and see if it is what you are looking for. The photographer doesn't have to photograph every cosplayer that responds, they can choose the ones that they think would suit. I wouldn't feel offended if a photographer said "I don't think that your costume is quite good enough to match the vision I have for this photo." I would get up and make it better. I'd also probably also ask what they thought was off about it, and ask for constructive criticism. They might be seeing something that I didn't consider about the photography of a costume.
If someone gets upset that you don't want to take pictures of their cosplay, then they have that right, but they are posting on your page in response to an open call so it is a bit silly. As long as you aren't a complete jerk about it, i.e. telling people that their costume sucks, and you are able to tell people no, but thank you for your offer, it shouldn't be a problem.
On the other hand, it might alert you to cosplayers that you haven't seen yet, or interacted with that are accessible to you faster than word of mouth can travel, and allow you to take the pictures that you have been dreaming of without lucking upon someone at a huge convention that happens to be wearing that outfit.