welllll....that's hard to say. i mean, that's like saying you hate drawing but want to become an architect. modern medicine (Western med, that is) has its foundations in science. you can't very well treat patients with nice thoughts and good intentions. you'll have to understand scientific concepts, chemical interactions, saturation points, etc. enough to apply them and make judgments for each patient based on that knowledge and understanding - especially in this pharmaceutical-based world.
though i understand where you're coming from, b/c sci/math is harder for me than anything else - or at least, i don't have to work at all to do well at humanities/social sci, but i do have to put in an effort to do science. i have a real hard time with chemistry and calculus, but biology and physics I ace. i also did poorly enough in my chem and calculus classes during my college years that i didn't go into medicine (i'm in law right now). but now i'm determined to take another stab at it, and it's easier for me to grasp chemistry b/c i am thinking of how i'll need to apply it later on.
your post begs the questions: why do you want to be a doctor? to help people? there are a million and one ways to help people w/o the science track. think you'll be comfortable basing your career on your mastery of a subject you find difficult?
also, what level are you at? high school? college? post-college? i ask because, like i said above, general chemistry sucked royally, but as i got into more advanced classes (like biochem) it got more interesting, and i was able to understand the principles of chemistry better from that perspective. my general chem class was too scattered, the basic principles were nothing more than route memorization, just figures and numbers that had no meaning. so if you're just beginning your premed reqs, and are basing your science aptitude on that, i'll give you hope and say it gets better.
but then again, i'm not in med school yet. i'm not a doctor yet. so maybe i don't know what i'm talking about....
i actually posed the same question to one of my professors in my last semester in college - telling him that i really like it, but i'm better at other things. he didn't give me much of an answer (went on some diatribe about doing what you love vs. making money), and i went into law instead. but now here i am, going into medicine again.
good luck!