This is just me, but I always leave "prestige" where it belongs - in the trash. To me, completing medical school/residency requirements are of the highest priority than the illustrious power that the name of a program gives off. This is analogous to clothes, for instance: a $5 shirt covers your body the same way a $5,000,000 shirt does. The only difference is in designer name/brand, which is what you're paying for. In the end, you're still wearing a shirt.
Of course if you happen to match into a prestigious program through your own merits, then go for it if you want. Just don't devalue yourself over it if it doesn't happen. Many "non-prestigious" residencies are just as great as ones that are. After all, they wouldn't be an option for soon to be docs if they weren't. 😉