I would say that a brick and mortar office can work for you. But it needs to be smartly done. Someone previously mentioned being near an anchor store like a home depot.
While I agree with the anchor store part, the example of a home depot is not so good. What you need is what some folks call an overflow anchor. When people come to a place like home depot, they are usually coming for something specific. They walk in, get what they want, walk out and leave. This is NOT a business or traffic builder for stores around it.
Now some of this also depends on the kind of town you are in. You want to position yourself in a way that is likely to be seen by the largest segment of the buying population. Why buying and not selling? The stats show that office location for business generation is far more influential relating to buyers than sellers. Sellers tend to go with agents that have signs in their neighborhood, or bigger advertisements.
I live in a bedroom community. Folks who come here to buy are normally not from the immediate area. The average has been folks who live anywhere from 30 minutes to an hour away. My town also tends to host lots of sporting events for kids - soccer tournaments, etc. This is another way folks discover my town.
So with all that in mind, you have to think like those folks. One of the first things many people do when coming to an event or maybe driving down because they are already interested in possibly buying - is to grab a cup of coffee, maybe look for tourist info from the chamber of commerce,etc. In my case, I would look for something close to a starbucks or a bookstore. Closer to the freeway (if there is one) is likely to be better. With that in mind, maybe you find a coffee shop near the freeway. Maybe you even find something close to the hotels for the town. In any event, that is a strategy that would work for MY town. What would work for your town might be very different.
My point is that while there might be some general ideas that tend to work in more areas (I think the coffee shop proximity idea is pretty universal), you need to cater what you do to your market.
Can a virtual office work? Not always, I have lost a couple of clients who when they found out I did not have a brick and mortar office to meet at , were put off and decided to work with someone else.
I think the right approach is a balance. I don't think you need a 20,000 square foot office space with cubicles for all your agents. You can rent a suite with a meeting room and a supply room and have that work just fine for you. You are then able to satisfy all the possible needs of a client or an agent that works for you without as much overhead.
When you are looking for a place, make sure and take a look at craigs list. I was amazed at how much was available on there in my little town. In fact, I found one location where you rented an office and shared a receptionist, several meeting rooms and a break room with other office space renters all for about $300 a month. That included the above and full cleaning, all utilities and an internet connection.
Good Luck,
R