Please don't go crazy!!! I'm sure your Ivy League school feeds many attractive candidates to med schools every year, and the schools are very well aware of your school's letter process. No, it's not great, but med schools will not spite themselves and miss out on all of the talent from your school due to this.
It's very easy to freak out based on the II reports on SDN, and then allowing FOMO to make you nuts. All you can do is chill.
You are being delayed, and you won't be reviewed without the letter, but, rest assured, the letter will come, you will be reviewed, and plenty of people from your Ivy League school will have very successful cycles. You will put yourself at a much bigger disadvantage by allowing your impatience to deny the schools the committee letter they covet if you choose to circumvent the committee and just send your strongest letters to the schools.