1. I don't know of any schools that do not require math and science classes. I don't know where you went to school, but every place I've heard of requires at least basic math and science classes to graduate. Three years of math to catch up? Exactly what kind of physics did you take? The physics I took was trig based. Correct me if I'm wrong, but I believe you take trig after you take algebra. So even if you didn't take pre-algebra in high school, you can take pre-algebra your first semester, college algebra your second semester, and then trig the first semester of sophomore year. Traditional students take physics their junior year, so even if you were behind in math, you wouldn't be behind in terms of pre-requisite timing. Furthermore, I took math through Calc 1 in high school. My first semester of college I took Calculus 1 again. Between then and Physics 1 (First semester of Junior year) I didn't take any math classes and it wasn't a problem.
2. The reason you didn't realize you needed physics is because you didn't do the research and look at the pre-reqs. That's no one's fault but your own. It's beyond annoying to constantly read about people making excuses and blaming other people for their shortcomings. We have all messed up at some point. Take responsibility for it.