since there is an element of surgery to RE, i would doubt that an IM endocrinologist could do it... just a thought...
and regardless, you can't be board certified in RE unless you've gone through OB?Gyn and then an RE fellowship. (you're not eligible to sit for the exam) without board certification, you'll have a hard time getting hospital privileges to see RE patients.
i suppose you could just setup your practice around fertility and skip the procedure-type interventions. (in vitro, surgeries, etc)