It doesn't kill your app but I've always found it to be a bit of a black eye on your application. If I'm sitting on a residency admissions committee, I would wonder why your home department wouldn't even give your their seal of approval by giving what is typically considered to be a perfunctory honors grade for the rotation.
I do agree that derm rotations can be frustrating because most are just a shadowing experience.
Is there anyone that you were close to on the rotation that you could ask for insight? (A resident? A faculty member?) Is this typical of your home program to NOT honor rotators?
My former training program typically honors everyone (especially if we know they want to do derm). It would take a considerable amount of work to NOT honor the rotation but I do realize this may differ from program to program