I would have a qualified home inspection do a pre-listing inspection and see what turns up. Then based on the report decide what to do. It may not be as bad as it looks then again it could be worse. That is the why I would get an inspection.
I think there is a diplomatic way to tell the builder that in your experience the average buyer will not accept certain things and ask if there is any way for him to deal with his contractors to get these things fixed early.
I wouldn't be too quick to dump him, if you can educate him and convince him to inprove his product you might have years of sales with him because of that!
I agree with Secret Agent and Deep Sea. I would definitely try to educate the builder. It is like having a listing where the seller refuses to do the dishes, make the bed, mow the grass, etc. Maybe the builder can do some cosmetic fixes and get better over time.
However, if it is krap and the seller won't fix the problem, then the Principal Broker can terminate the listing. You play good cop bad cop.