This may be a little off-topic but I'm going to mention it anyways - for whatever it's worth....
If you want to survive in this business, people looking at you funny is a small matter. I'm a non-confrontational, compliant, never-in-your-face personality, and I've been jumped and trempled on so badly in the past (in 1 transaction: gosh, the salesperson was telling lies about me to his corporate and I could never get him to return my phone calls until I get the Corporate Operations Manager on the phone). Looking funny at me at that point would be nothing!!!
I say Go for it. Explain your reason (if you feel like it). If he thinks that's weird, and you cant build a relationship with your broker, go to the next one!