I took physics 1 and 2 without taking either class- however, my physics classes were algebra based rather than calculus based so I wouldn't have needed calculus anyways
Trig was a prerequisite for physics and I should have taken it before - but after my sophomore year I changed my option in my major (same major, different direction basically) and my advisor accidentally told me I could waive trig (it was also required for my major) - when it came time for me to take physics, she was like "I can't believe I told you that was OK" but said I could go ahead and take it if I felt comfortable.
I basically just did a quick trig refresher with one of my older engineering friends and I was ok (you basically just have to know sin/cosine/tangent) - but a good handful of grad schools do require trig so you might want to take it later anyways, if you can
I can't speak for calc though - if your physics is calc based I'd be concerned about that