I'll tell you, first off, that if you like the teaching aspect, you have other options than outright academia. I'm in private practice, but I have an affiliation with the local university and, as such, lecture to the residents and fellows. I also assist in training of the retina fellows. They spend a half day once per week in my clinic. I'll soon be staffing the VA resident retina clinic a couple times per month. I've been involved in some clinically-based research. I feel that gives me the best of both worlds.
For a long time, I was headed toward academia. If you do decide to pursue it, there are typically different tracks available. They are different blends of clinical, teaching, and research. Depends on the department. I'll say that most faculty I know do at least clinically-based research.
There are a number of downsides, but some upsides. You have to answer to the Chairman, who then answers to the Dean. You'll never be your own boss, and depending on the personalities involved, it can be okay or terrible. You're right about the pay difference (mainly due to the Dean's tax), but there are other perks. You have residents (and sometimes fellows) to cover call. You can also attend a lot of meetings on the university dime. There's prestige, if you care about that sort of stuff, in being an academic.
I don't think finding an academic job is as hard as you are perceiving. May not be Wilmer, but there are plenty of opportunities.