What do you want to do with your life?
FM will keep open several doors. Primary care, Urgent Care, ER in smaller towns, Hospitalist. You can do fellowships in Sports Med, Geriatrics.
IM will leave open Primary Care, I would imagine you can find Urgent Cares/ER's that will take you, Certainly Hospitalist. Countless specialty fellowships.
If your concern is your low stats--I think you'll be able to match IM.
If you think you actually want to do Family Medicine--I did it and I don't love OB/GYN or Surgery either. I wouldn't bring this up in interviews. But I don't recall being asked about those things a ton. Most people don't anticipate delivering babies after residency in FM. GYN: If you do primary care--people will probably want the occasional pap. But, you can probably develop your practice such that all that gets sent to your local GYN doc.
I did FM and am now months into my first real job as a hospitalist. Last week, I did a GYN exam on a 20 year old with PID. I've done them on women at Urgent Care shifts when I moon-lighted. Do I enjoy it? Nah. But, you'll want the skills. And, I'm pretty sure you'll do your share of paps/pelvics in GYN.
Surgery wise? I cut off a few skin lesions. A lot of stitching. If you do ER or Urgent care, you'll do some stitches. Same hospitalist gig, I've done 1 punch Biopsy. Again, all important good skills to have.
But you won't go into Family Medicine and learn to take out Gallbladders or Appendices.