I'm not really a math weakling (I get Calc), but I DO suck at Physics. It's just so hard for me to understand and ... stupid! Yeah, I know I'm being pigheaded, but this is the most widely shared pigheaded sentiment I know. It's not really math-based, but more conceptual like the previous poster said.
However, I did make A's in Physics I and II with horrible teachers both times. The first semester I dragged my own ass along pouring over my textbook and problems to teach myself. It was fairly easy, albeit time-consuming, because the material (gravitation and what not) was pretty easy to understand. The second semester I was taking the Princeton Review classes for the MCAT and their Physics lessons totally saved my butt for Physics II- that stuff (electrostatics and other physics words I don't even remember now) is super hard for non-physics-inclined people like me. My TPR teacher rocked and I had lightbulbs popping up left and right! I got a great grasp of the basics from TPR, so it wasn't hard for me to go back and teach myself the little caveats and subtleties I needed to know for my college Physics class.
BTW, I highly recommend taking TPR concurrently with undergrad Physics because you really get a bidirectional benefit from taking two classes that can fill in each other's holes (ew that sounded gross), I mean make up for each other's inadequacies. Beware of taking too much TPR into your undergrad classroom though. g=9.81 m/s^2 there, not 10. Don't get points off for doing that.
In summary, go find a good teacher. One that can explain things to you in terms you understand, i.e. a hip, young type. And do as many problems as you can. I found the problems in our assigned text to be too complicated when trying to learn the concepts, so finding a "Physics for dummies" or other simple-type book can help you cement those ideas initially. Since I was studying for the MCAT, I used Examcracker's 1001 Physics problems because they drill the stuff into you with each mini-section dedicated to a single topic, and nothing they would put on the MCAT is that difficult to understand. So start there, then tackle the textbook problems. Good luck!
P.S. My Physics lectures weren't that bad when compared to the lab

. All's I can say there is sit next to a smart person.
😉