Do well means, yes, honor your clerkships-- ideally. So if you get high pass in all your rotations, that is just average-- the saving grace there is if your letters of recommendations from working with specific faculty are fantastic. So if you high pass peds, you must have a very strong letter from a peds person. I had the same issue. I actually high passed pediatrics, because I sucked it up on the shelf exam. But I had three incredible letters from my med school's programs peds residency director and a PICU attending. Then I did an away rotation at my dream institution. I'm pretty sure I wouldn't have necessarily matched at my dream program with the high pass, even with the great letters, without that "extra" connection.
Now, in terms of matching at "average" stat programs, high pass is fine-- please don't take from this that a high pass or even a pass means that you won't match in peds. But for the most competitive programs, these clerkship scores do make a difference, unless you have something extra to bring to the table.