Living in Australia or Switzerland would be an upside, at least for me.
The downside is that you are reading studies from dozends of hospitals on whose QA you have no influence. Your clinical information is even sketchier (is that a word) than in an on-site practice. As you don't know most of the referrers (except for maybe some of the high-volume ED docs), it is hard to gauge how much value their clinical information has. You are a robot that receives a study and spits out a fax, not very personable, some people say that this will increase the likelihood of people trying to shift blame on you if things go south.