I say yes. I was able to do it successfully. As long as you have a strong application across the board (not just 4.0, but high DAT, good extracurriculars/shadowing/volunteering,etc), then it should be fine. Make sure to apply to schools that don't require a bachelors so you don't waste your money.
Others have different opinions as to whether or not you should (some have cited residency programs requiring bachelors degrees so consider that if you want to specialize), but I have yet to regret my decision. I saved 1.5 years of undergrad and was able to get out without being forced into a pointless (in my opinion) degree.