I was fortunate with my shadowing experiences as a premed in that everyone knew me at the hospital, so I got to DO stuff a lot and they let me wander around and follow patients from the ED to the OR to the ICU.
Nearly every specialty is boring to watch until you gain a base level of knowledge. One of the things that makes surgery so appealing early on is that it is easier to say, "That is really cool!" when you are watching them do something. Watching people think isn't nearly as exciting. As you progress, you also begin to notice the small details that separate good docs from bad docs.
One of my preceptors was in the same kind of deal where he spent a lot of time talking about random things with the patient and it seemed like he wasn't doing anything, except he was simultaneously doing most of the physical, checking meds, etc. We'd walk out and he'd ask me, "Did you see that dark spot on his arm? I'm a little worried about it. That wasn't there before." (When before was 5 months ago)
The way our system is built, many premeds and med students scoff at primary care, but a great PCP is worth a ton and is hard to come by. There is tremendous flexibility in their job with as much or as little as one wants to do. In one day with my preceptor I did compressions on two people in the ED, ran stress tests, discussed hospice with a patient, froze warts off, spoke German (in rural Florida that isn't too common), stapled laceration, debrided a little old lady's arm and met a dozen really nice people with a variety of bread and butter problems, as well as a couple of really difficult people.
Is it what I want to do? Not really, but if I wound up in it I also would be perfectly content with my life and able to get a variety of things to keep me interested. Yes, if you are stuck behind someone who strictly keeps it 8 to 5 with HTN, DM and the sniffles, then you're going to be bored. As one gets older, that predictability is not necessarily a bad thing though.