I'd say you risk losing business if you DON'T bring them in. Most of your clients who use you, must use you for a reason. Same goes for him. Too many agents have this mentality of "there's not enough". I bet if you bring him on board, he can help bring business your way. And a good reason for him to come over, may be that you can bring some his way. Or the two of you can just compete it out at different companies and one of you may lose business to the other??
I work with a guy who lives in another county, about 40 minutes away. One day I went to do a BPO and the agent mentioned that he'd done one a few weeks earlier (which means he drove an hour to get to a spot that was 20 minutes into my territory).
Turns out he was doing a showing, not a BPO, but my original thought was, "Why didn't he tell me he was doing these?" I figured he was afraid that I'd not want him doing them. I would have called him up and given him my top 20 clients. I wouldnt' do this for everyone, but this guy's helped me and I would help him. It would also make him less likely to work my territory if he could find a ton of business closer to his house. Maybe he would have given me one or two names that I don't have and I'd have more near my house too? Who knows.
Try to offer to help him, make him part of a team, and maybe you can both benefit in these difficult times.