This may be true in fields you have worked in, but I have personally seen them enforced (or attempted enforcement, which caused significant burden to the person attempting to avoid their non-compete) several times .... and that's in my very short time in this field.
So at least in my limited experience, I would view it as more significant than I think you are viewing it. It would be financially devastating if you invested a lot of money opening up a clinic only to find yourself defending a non-compete clause enforcement.
Whether it is technically "ethical" or "unethical" might not be all that significant in the end. It's a small field. If you want a rep in your local community as that doctor who 'stole' a bunch of clients when you were an associate, you might be doing yourself more harm than good. Regardless of the actual truth behind the perception.
I think a better approach is to plan ahead: If you are taking a job as an associate and know you want to leave at some point down the road to open a practice, or buy into another practice, then make sure you negotiate in a way that gives you that opportunity. Nothing wrong with saying "I won't accept your non-compete clause as written; but I will accept a non-compete clause with these changes......"
My non-compete only says I can't go open an ER hospital down the road, which I'm ok with. I'm perfectly free to go work next door in GP, open a GP practice, or whatever. And I'm pretty ok with that agreement. Had it been more restrictive, I would have negotiated a change or not taken the job. I think that's a better way to handle it.