World-class educators don't work for the same salaries as those at public state schools, plus class sizes are smaller, buildings are of higher quality with higher quality equipment that is divided amongst fewer students, etc. I'd be surprised if the cost was only 80k, given the facilities and professors at Princeton, coupled with their small number of students relative to larger universities.
And, while many could afford to have their families pay, ideally, no one would have to pay as a matter of principle if the endowment were to grow large enough. Getting to that point will take another few decades (and possibly closer to a century) of endowment growth, which would provide enough leftover funding from interest to pay for students' educations, while also being able to grow the university's facilities and fund its research without having a chance of depleting the endowment, even in a crisis.
The purpose of an endowment isn't to take care of now, it's to take care of an institution forever. Through thick and thin, crises and wars and changes of government and every possible eventuality. That's why you don't spend all the interest, and allow the thing to grow, even if it comes at the expense of those people currently in need.