Depends. As others have noted, this issue is of more importance with competitive residencies and academic medicine. Then again, I, an IMG, got into an academic general surgery program so I guess anything is possible!
The major factors determining residency placement are your USMLE scores (some more competitive places and residencies will also request MCAT scores), your grades, letter of recommendation (what they say and who they are from), clerkship evals and any away audition rotations. Going to an Ivy might bump ya up to a more prestigious residency but it won't do a thing without the above factors also being in line with their current residents.
And I wholeheartedly agree with the idea about "clumps". Frankly once you get out of say the Top 5 or so, the difference between a school ranked by US News and World Report (which, BTW, has highly skewed rankings. All the PDs know that and look at those rankings somewhat suspiciously.) at number 15 and one ranked number 35 is probably not that big of a deal. They're both still good schools. Numbher 15 and number 150 - well there's a difference.
If you are planning on returning to California you might check at some programs you are interested in and see whether they prefer California medical school grads as well. Some programs are highly inbred and prefer their own graduates to "outsiders".
Again, the difference between Yale and a numer 30 school is only likely to make a difference in really competitive programs and specialties.
Hope this helps.