I'd just want to point out again that this is largely historical recognition building. Back when most presidents and supreme court justices were looking at colleges, HYP was much easier to get into and much less rivaled by other universities.
I'd also point out that UC Berkeley is probably more famous than many schools that are tougher to get admitted to, especially globally. Layperson rep isn't a great yardstick here. How many elite universities in China can you name? Does that make it safe to think they're easier to get into than University of Michigan? Etc.
Peking, Fudan, Nanjing, and Tsinghua off the top of my head. But you're not comparing apples to apples here. Since we're talking about the U.S. applicant pool, let's keep schools inside the U.S. - otherwise you'd have to take into account the hypothetical opening of universities to the entire Chinese pool of students and I don't think you want to do that. Yes, Berkeley is more famous than many schools that are tougher to get into but Berkeley is pretty tough to get in if you don't live in CA. I don't think how tough it is to get into a school should be a measure of that school's prestige.