This is probably somewhat of a generic reply but ultimately you have to love what you do. Unfortunately, sometimes it takes some time to realize that medicine is not what you thought it was or what you wanted and unfortunately for some people they had to spend several years and taken up loans before that realization is made. Medical school is tough, it could be stressful, you will go nights without sleeping, you will wake up at 4 in the morning during some rotations and that is medical school. Residency is its own beast. However, you are also given a gift of experiencing some of the most intimate moments in patients' lives whether it is being able to diagnose, to treat, to deliver a baby; but also to share bad news or to be present with family when a family member passes. I enjoy both ends- the good and the bad- because it is humanity and one that very few experience. Also, as a physician, you can do much more in helping your community if that is what you seek. Finally, there is the medicine behind it all which is great.
That paragraph was to answer why some people are unhappy with their decision. As for neurology, it is probably no different than most specialties with the following exceptions. For the most part, as residents we are surrounded by laid back and helpful residents and faculty unlike the stereotypical type found in some surgery or ob/gyn programs. Some neurology programs have better hours than others (mine is quite busy) but overall the hours are better than your typical surgery program and probably medicine too. But again, being busy is not so bad when in all honesty you need to be busy in order to learn and also when we have residents that volunteer to see a patient for another when they know that person is absolutely swamped. Also, neurology has a lot of possibilities as for further specialization if you want. Finally, the whole science behind it is amazing (very biased last statement).