Here's my recommendation. Go shadow people who have careers you are interested. Split your time between vets and non vets. There are a lot of career paths in things like epidemiology, wildlife management and conservation, biochemical research, lab animal technicians, biology anything especially field work, etc. Chances are your undergraduate work has given you a good base and you won't have to start over.
Then, find vets who have found unorthodox ways of paying for vet school, like scholarships or loan repayment programs, or who have found higher paying jobs. It's possible. Also, You don't have to be a clinician, and you can also be a clinician and then move to a different job. USDA, NIH, big companies, they all pay very well and you can do a stint there. Learn how to manage money: really take the time to cut living costs and maximize your dollars.
Don't quit because it's hard, quit because you are making an informed decision and have found an alternative pathway with a career you will enjoy, so you're never really anxiously wondering what if.
Or, bust your butt and reach those goals! Vet med can be a great profession and there are so many directions you can take with it.