Although it is true that the medical field probably offers its practioners the lowest effort:financial reward ratio compared to other profession, it does have certain outright advantages including for the most part stability and a tremendous amount of social respect and prestige (something stock brokers or ~ahem~ laywers dont get).
As far as the advantage of specialization, I feel that there are significant advantages of doing so. I struggled with decision to do a fellowship vs primary care at the end of my intern year as well. It's about choosing what you are REALLY interested in doing and focusing your efforts on your foremost strength. It's the same argument as choosing a major in college: liberal arts/general studies vs. molecular biology/comparative lit/latin... If you truely feel excited about cardiology or oncology or ID, and not so much the other fields, you should specialize. Given how broad medicine is, there is no way in hell you are equally passionate about every single field. In my experience, a good measure of how much you like a field is to look at the bread and butter diseases in that field (ie. chest pain in cardiology, copd in pulm, esrd on dialysis in nephro) and ask yourself if you mind seeing patients with those problems 7 days a week. Being able to become a specialist in the area that I feel most passionate about is the biggest reason why I chose to specialize.
Other perks to note for specialization is your role in patient care and the degree of knowledge required in your field. In major cities, specialists serve generally as consultants in the hospital and sees referral patients in clinic with one or two focused problem, not the "doc, I have fibermyalgia and I hurt all over" or "i need my dilaudid refilled" patients. In the hospital, your opinion and recommendations serve as a guiding light for other specialties and generalists alike, and you get respect. Although a broad knowledge of medicine is generally necessary, you dont need to stay at the forefront of literature with any field but your own. IN small towns/rural areas in the US, as a specialist, you may be the only one in the particular town or region, and in addition to the above, you have the option to be really busy, and be REALLY RICH really soon.
These are the reason why I think why at least 1/2 of the graduating medicine residents decide to go into fellowships. Primary care is becoming less popular and the American public focuses on "sick care" rather than health care (ie.pouring money into having the state of the art cath lab for acute MI and not spending enough on primary prevention of obesity and hypertension). In addition, it's becoming increasingly encroached upon by NP's and PA's to a point where people spending 1/2 the time (and money) you spent in school do the same job you do. Guess what? NP's can't do what a specialist do.
At the end of the day, if you have done a fellowship and truly hated what you do, you can always go back to primary care. If you graduate from residency primary care all the way, and then 5 years into practice you decide you hate it, you are kinda screwed.