On my interview day (and I might be misremembering this, so I apologize if that's the case), everyone had morning interviews, and we all got out fairly early in the afternoon. They probably just have "morning and afternoon" interviews listed because whoever the interviewers happen to be that day might be busy during one or the other. It allows busy, practicing physicians some flexibility.
Maybe you'd just sit around in the morning if you had an afternoon one. We all got to go to a lecture, maybe you'd go to a couple haha.