I think it's just a lot harder to make a lasting impression over 2 weeks than if you were a regular student there and maybe already knew some people. It's certainly not impossible, but I think you'd be more "extra" to be memorable because you're only there for 2 weeks and then disappear again.
If you're interested in academic internships, the best way to learn about the culture there is to talk to current and recent interns about their experience. You can get an idea of it from watching as a student to see how interns are treated and the kinds of responsibilities they're given, but sometimes it might be challenging to balance that and mentally noting things while you're also juggling all of the aspects of being a 4th year student - especially in a hospital that you're not as familiar with, so you're trying to remember their flow (and possibly their floor plan 😛 ).
On the other hand, for other specialties, it may be quite valuable, like Jayna mentioned. I've never seen a visiting student at my institution for surgery, but I have seen them come here before for things like radiology and ophthalmology because the departments here are quite large compared to some other programs, so the amount of cases you get exposed to on those services is a lot higher. I went to vet school at an institution that didn't have an ophthalmologist at all when I was a 4th year, and only had 1 radiologist, so in those situations, there's probably a pretty good learning benefit to going! Just not sure about the benefit for surgery.