There is a reason that neuro is one of the least competitive specialties. This thread sums it up nicely.
Not all that glitters is gold.
Clinical work might be more complicated than you surmise. For instance, I do full time adult sleep medicine. I do strictly outpatient work (in the clinic seeing patients) for *only* 2.5 to 3.5 days a week. The balance of my time is reading portable and in-lab sleep studies (rather like a radiologist). I don't take call in the traditional sense (I am available each and every night for my labs, though), I don't work holidays, and I don't have to go in on weekends if I don't desire. Sounds cushy, right? I should easily finish a work week in under 40 hours, right?
Guess what? A week that is 40 hours or less for me is an extreme rarity. Maybe in two years, it's happened once or twice when the workload was light.
When I first started and things were new, I would clock in 80-90 hours in a week. I always clock in at least 50-60 hours, even now when I'm much faster than I used to be. My clinic days are quite long, really. And beyond that, there is just too much else to do. Dictations, editing my electronic notes, calling patients with problems, calling doctors, clearing up orders with DME companies, calling pharmacies to straighten out medicines I've ordered, paperwork related to insurance requests/denials for testing on my patients, arguing with CMS or insurance companies over drugs I can get for a narcoleptic patient, VA paperwork (for disability requests), questions my techs have about scoring or medical knowledge, meetings with other doctors in the practice, meetings with techs about their performance, and researching patients' notes prior to seeing them in clinic to help the work flow more smoothly. And this is *not* including vacation or fun-type responsibilities like continuing CME, or attending lectures/reading/studying to keep up on sleep medicine.
My point is that there is more to even the most seemingly simple, straightforward, outpatient job than may be readily apparent. And these traits may make a true 40 hour work week a bit unrealistic for most clinicians.