I just checked my university's site(Univ. of Hawaii, Manoa) and they offer a B.A. in geography. On a side note I entered a Geography Bee in 8th grade and did horrible. I only got one question right. That was because the guy phrased the question so that my kid sister could have answered it.
I think geography kicks ass. I haven't taken any in college, but it is still one of things that permeates everypart of life. Just say you want to do international medicine or something if your worried, I was at UW (seattle) at one of the two days for non-resident students and everbody there was either rural or had some experience in international stuff. Most people experience was not even medical tourism BS, just experience overseas. The admission people love diversity, I'd say don't worry. This degree will certainly not keep you out of med school. I think most med schools love to see pasionate people, no matter what they are passionate about. Just have a good story line as to why you have been led to medical school.
I majored in Urban Planning, which came from Geography and Public Health. I enjoy having a place-based perspective. I felt like when I found the major, the world suddenly made a lot more sense to me than it had before!
Geography such a broad major, there is no such thing as a "Geographer" , most Geo majors go on to grad school in Urban PLanning, some go to law School, and I'm going to med school. The geo majors from your school are probably lawyers.