I don't think either one of you is necessarily wrong.
Anecdote time: In high school I took calculus-based AP Physics and, like Lawper, I felt like I had a deep understanding of how physics worked even though I wasn't the best student in the class. I haven't taken a real math class since high school, since I didn't need to in college.
Now that I've been out of college for a couple of years, I decided to take algebra-based physics for my medical school requirements. I did this because I hadn't had math in so long and there's not much that I remember from high school. I found that I felt I had a good conceptual understanding of the material beyond the plugging and chugging, but that was due to my own effort. The algebra-based physics classes came easily for me, and I'm sure some can get good grades just by plugging and chugging, but that doesn't make for a good understanding of concepts.
I guess what I'm trying to say is I can agree with the penguin that if a student puts the effort into understanding physics concepts, that can be done with simple algebra-based physics. But I also agree with lawper that it's sort of a "you don't know how much you didn't know until you try it" with regards to calculus-based physics. Even though I don't remember any of the material from my high school days, I do remember feeling astonished at how all the concepts melded together in my high school physics class. I had to work harder at that to get a conceptual understanding in my post-bac.