i agree, no right answer. there are advantages and disadvantages to every choice.
early:
+get it out of the way, see if you really like IM or not, if not then you can plan your electives and fourth year rotations better to search for the specialty you want, less tired from core rotations, fewer expectations from people writing your evals, IM skills and knowledge can be applicable to most of other rotations
-less clinical experience so may not be doing your best, less experience with shelf exams, interns are just starting and interns just got promoted to seniors so teams can be a little disorganized and may not be teaching you as well as possible
late:
+more clinical experience built up, more experience on what to expect on shelf exam, interns/residents may be a little better at teaching
-more expections of you, maybe you are grumpy at the end of the year, less time to plan ahead if you decide "no" to IM
middle: kind of in between, and i, too, have also heard the impression that many people who are competing for IM choose the middle and try to appear the best student on the team as possible
whatever you choose, if you want to be competitive for IM, getting a good eval, good shelf score, and finding an attending to write a LOR should be your goals with regards to your application during the rotation, well, the goals for whatever specialty you're looking into.