One word.
KARMA
But seriously if you say it is serious with life long consequences then it sounds like you need to see it through. As others have said he may be on the boards or connected with those on the boards or residencies.
So the risk of him retaliating is real.
This is complete and utter nonsense. And ignorant to the reality not only of medical practice but any sort of bureaucracy. Conflicts of interest are typically taken very seriously in formal settings that are intended to be impartial.
1) If the physician being sued by the OP is a member of the state medical board and the OP came in front of the board for disciplinary action, the physician in question would have to recuse himself. If he didn't, that would be grounds for a successful appeal.
2) This is absolutely not the way the specialty boards work. Worst case, the OP ends up in a field that requires oral boards - and the examiner is the physician in question. And the same thing happens, the OP asks for him to recuse himself. If the physician says he can be impartial but still fails the OP, I guarantee the OP would get a rescore or second attempt in some form. Pain in the ass to get an appeal through? Absolutely, but the OP would have a significant case.
Now, the importance of having a broad network can be very important for residency and fellowship, but again, there are lots of programs out there and unless the connections are apparent, it's not like people are checking with everyone they've ever met in their career, particularly if the people aren't academic.
For a job, short of joining a practice in the same city, same field, with some people who are VERY close to the physician getting sued, totally unlikely to make a difference. Academic positions may matter a little more, but for private practice? Not likely. When I discuss my group's recruitment efforts, it's all very general and I only ask people their opinion if I know they likely overlapped directly. But it's not like I expect my peds pulmonologist who trained at Denver 10+ years ago to have connections that they can ask about a PICU fellow we're interviewing. It's just not a reliable piece of information.