Okay.... I actually think this could be a big advantage, just not the way you framed it. I'm a nontraditional student who had very little vet med experience, especially given my age (and it was almost exclusively equine shadowing). I was very concerned that I would have a hard time getting in to vet school. I was advised to play up some of the things I had that were maybe a little more uncommon - an extensive employment and business owning experience that required me to work very closely with a broad range of people. Since I had an extraordinary amount of animal (non vet) experience and had demonstrated a strong interest in medicine in other ways, my advisor thought I could get by with low vet experience and that playing up "people skills" would help me stand out. This is a much more people-oriented field than people tend to think, and it's often where pre-vet and vet students are weak. It apparently worked for me; it might work for you, too.
So.... How about looking at it a little differently? Your undergrad experiences taught you that you're really passionate about medicine, perhaps, and gave you a lot of opportunity to build skills for working with people in difficult situations. I wouldn't go in with the attitude that you don't like human medicine so vet med looks better (for all the reasons the previous responses have provided). I would focus on the love of medicine (because really, if you don't love medicine itself, you aren't going to like vet school anyway) and what opportunities you've discovered on the veterinary side that you're even more passionate about than what you could do in human medicine. Do some soul-searching on what you love about vet med and what your goals are, and I bet you can find a lot of ways that human medical experience has helped you towards that goal. You just need to have some very positive, clear ideas so that your application doesn't look like vet school is plan B, but rather like a natural progression of your life experiences. I'm living proof that you don't have to be born wanting to be a vet in order to go to vet school - my life took a lot of twists and turns to get here. But figure out why it's right for you and sell that, rather than acting as though it's a default because something else was wrong.