1) if your referral sources are PCP's, and then you encroach on their turf, how likely do you think they will be to refer patients to you in future?
2) if something goes wrong, and you hold yourself out to be any manner of FP/GP/PCP, IM doc, you will be held to the standard of an ordinary and reasonable FP/IM doc in court, not that of a phyiatrist practicing a little internal medicine on the side.
We are specialists. Would you go to your local orthopod to address your primary care needs? There is enough confusion about what we do without muddying the waters further, IMHO.
Unless you are double-boarded, I think it is a bad idea (but then again, I also think it is a bad idea to do fluoroscopically-guided ESIs and other spinal interventions without being fellowship-trained, so I clearly have a particular viewpoint on the broader subject of practicing without what I consider to be adequate training)