Hello JWR,
I sell foreclosures much more frequently than I see short sales but will see if I can offer you some direction. I think you've answered your own question.
I believe that fallout is a relevant factor and agent rankings should probably be based upon closings, not listings taken. It would still be fair because they are competing against agents with the same problem- short sale fallout issues.
Why are you ranking agents based upon listings taken, and not closed? Do you do this only for short sales?
Perhaps there is a reason you do it this way that you could share and maybe someone on here could help.
I can see how just getting the agents to take listings is something you'd want to encourage, but perhaps you can rank them for that separately.
_________________________
"It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change"-Charles Darwin