I can empathize with what you have described, as I attended a couple of the top party schools in the country. To be frank, at many large undergraduate universities, there is going to be a large proportion of people who drink heavily. I was also in Greek life, so I can empathize with the party culture there as well. The 'party scene' definitely can suck you in and hold you there like quicksand, but you can also manage if you cultivate proper priorities and exercise your ability to say no. You can go out; you cannot go out 5 nights a week. These things in and of themselves will not drag you down unless you let them. Additionally, many pre-meds switch to business/other majors at every university. The longer you spend in these forums, the more threads you will see popping up along the lines of 'what proportion of freshman premeds actually get into medical school?!?!' and then everyone basically saying there is no way to measure that. One of my professors in one of the prereqs had everyone stand up in class that was premed (the majority of the class), and then said that 1 out of every 10 will actually submit an application, then one out of every two of those will get in. Essentially the moral here is that attrition of premeds will happen wherever you go during your undergrad years.
That being said, removing yourself from the temptation is definitely a solution. Just make sure you aren't being a martyr for a goal that requires lots of further sacrifice. I am a large advocate of enjoying your undergraduate years. Don't inhibit your ability to do that with this move; make sure it is something that you believe will land you in a place you will be able to thrive academically, socially, and otherwise.