Cancel as soon as you know! Perhaps programs in larger cities don't have problems arranging last-minute fill ins from their waiting lists, but programs in areas without direct major airport access have a much more difficult time. Aside from that issue, great applicants who are wait listed for a specific date often make other arrangements as time passes and they receive additional interview offers. I've had to go pretty far down my wait list to find someone still available to fill a cancellation, and it's time consuming at a time when I just don't have a lot of time. Interview days involve a lot of effort and planning so that a cancellation even 2-3 days before an interview has the same effect as not showing up: The chance of filling the spot is vanishingly small, which requires re-doing EVERYONE'S itinerary and interview packet, and it's too late to adjust the catering order for either the night-before dinner or the day-of lunch. (I know, I hear the world's tiniest violin playing too.)
Cancellations happen and are part of the process and programs expect them. It's a mark of professionalism that applicants can make the tough choices of where to interview (and where not to interview) and give programs notice that their plans have changed. Anything to help mitigate the number of spinning plates that have to fall at one time is truly appreciated.