I'm 100% sure. My issue with Cornell is that I got a call offering a prelim which basically puts my out of the categorical spot. A prelim is a gamble.
Tufts have faculty who work at brown infertility and dartmouth head of program knows everyone. Just last year UVA matched a girl to REI at UVA. So it seems possible...
You would be correct in realizing that many random factors can come into play.
My story is an example. My mentor started helping me during med school at UC-Irvine. He is world-renown in Gyn-Onc (author of classic major Onc textbook) and helped me get into OB residency, also at UCI. During 3rd year, I had a drastic change of heart and wanted to do RE over Onc. Approached him with my situation. He gave me his full support, told me he would recommend me generically no matter where I wanted to go, but had especially good connections at two local programs. I chose the one I preferred (UCLA) and the rest is history.
How much did his help have to do with my fellowship? Nobody will ever know for sure, but while I had a good reputation and good CREOG scores, I had no research in RE at all. Also, it became evident during Fellowship that the program director highly respected my Gyn-Onc mentor professionally and praised him often. Of note, 3 of the residents in my class back then matched into REI, 1 into Onc. The others chose private practice.
Also, you might be interested to know, as an attending currently at UCI, (and we do not have a Fellowship program at present), I know that at least 3 of our residents have matched into REI (Stanford, USC, UCLA) in the past 5 years or so.
Bottom line: What program you go to and with which mentors you bond likely makes a difference. However, it is not that black-and-white predictable. It is one thing to take into account in residency selection but not the only thing. I realize this is nothing new for you to hear. Good luck!