Oh, I see you.
1/3 of the undergrads are in greek life. 2/3 aren't. UVA is a "Southern" school which basically means there is a minor but significant portion of faculty & students that dress like they're headed to the yacht club.
But since
med students don't have frats to begin with (and are, I believe, noticeably more diverse than the undergraduate body),
I do not think you will not fit in if you don't dress exclusively in pastels. The only time when everyone, med students included, gets fratty is at foxfields, but you'll be too inebriated to care.
I'm headed there in the fall, and I'm one of the least preppy people you'll ever meet. There's just as big of a hipster scene in Cville as there is a frat scene.
I'd suggest you do you, and people will like you for who you are
.