Specifically for Emory last year I wore a skirt or a dress with comfortable shoes both days, they say casual is fine, which is true, but you will have the opportunity to mingle with professors and alumni who work at CARE, Carter Center, CDC, etc who will have student work opportunities in the fall so you want to make a good impression. However, there is the CDC museum tour which you walk to (about 1/4 mile), a housing tour, and all-campus tours, all of these are optional, but if you are attending them you want to make sure that you can walk comfortably. There's also a brown bag lunch on the lawn with your department (unless they changed it - or weather permitting) which you will be sitting on the ground for. The weather is fickle right now in the south, we have some cool days, but yesterday it was probably over 80 and sunny, you don't want to be a nasty sweaty mess because you thought you'd one-up everyone and wear a suit 🙂 (joke only).
I told a student the other day, you will see the whole gambit of dress at visit days, from people in hole-y jeans to a few people in suits. I think (at least at Emory) it's good to be somewhere in-between. Emory has a lot of great extra-curricular activities, quality and involved student organizations, and really cool hands-on learning opportunities, I think professors want to know that you are taking it seriously, but also want to see that you have personality and are ready to get your "hands dirty". FOR ME PERSONALLY (i.e. my opinion only) I thought a suit would make me seem too uptight, especially since the invitation said casual. I'm not saying you can't wear one, but that was just my opinion. Also - one of the big professor mingling times is the brown bag lunch where everyone sits out on the lawn and there is keg beer, haha, it's hard to do that if you are over-dressed. Of course every department is different and I can see HPM dressing up more than say GH, but that's obviously something you can decide for yourself.
Again, that is just my opinion from my experience last year...I made some great connections and ended up with some unique opportunities that stemmed out of relationships I started at Visit Emory...if you can't make it, it won't put you behind, but if you can make sure you take advantage of the opportunities, do your research before and talk to people in your field of interest, start laying the groundwork for those connections now, it will pay off later!
Good luck at all of your visit days guys!!