I don't think it's necessarily a normal experience to not enjoy inpatient rehab--though certainly the trend these days is for people to want to go into outpatient PM&R. However, there is a whole thread devoted to the "PGY2 blues," so it's clearly not an uncommon experience either.
Personally, I loved my inpatient months, and there's usually something you can do for those "rock patients." Talk with your therapists/nurses--there's almost always something you can do to help the patient do better in therapy--whether better pain management, a peripheral nerve or motor point block, etc.) Remember, your best treatment for your patients is therapy, and your goal is to do what you can to medically optimize their participation in therapies and minimize any medical complications/commodities.
There were definitely some times I felt I was babysitting my patients, but honestly that was pretty rare unless I was on SCI at the VA. Even then there's often something you can do--even if they're just stuck on rehab for dispo reasons, you might as well see what you can do for them while they're waiting placement. It's definitely easier to not change things and go with the status quo, but you can usually do something. Most stroke patients will have shoulder pain, and it's fairly common to see general MSK pain in any rehab patient--even if you really hate inpatient rehab, you can practice your "outpatient" physiatric/MSK exam and convince your attending to let you do some peripheral joint injections (if indicated), etc.
I really found inpatient to be more fun--you have the whole team coming together and the atmosphere is just a fun place to work with your nurses, therapists, social workers/case managers, etc. It's also generally a pretty optimistic place, and you have more control over your time (assuming your attending doesn't micromanage rounding, consults, etc.). I find clinic a little more frustrating due to lack of control, and there's not much face-to-face interaction with your therapists. Outpatient is still a lot of fun and quite interesting, but it's certainly very different.
Your main goal as a physiatrist (inpatient or not) is diagnose and/or treat disabling conditions and maximize function. If you have a good teaching attending, that makes a lot more sense as you learn what braces and orthotics to prescribe for your inpatients, what dermatomes to check in your post THA/TKA patients (more common at the VA where these patients actually come to rehab). The confusing part is when inpatient, the diagnosis is generally already handed to you by the acute service. Your job is to take that and try to restore as much normal function as possible, and when not possible, to teach/encourage compensatory mechanisms (it's the whole team's job obviously--not just yours, but I find the really good inpatient physiatrists take a much more active role in this rather than just letting the therapists figure it out). But you can still uncover a lot of MSK pathology that is limiting a patient's full potential in therapy, catch/prevent CRPS, etc. etc.
With all that said, plenty of people really hate inpatient rehab and only want to do outpatient rehab. So hating inpatient rehab doesn't necessarily mean PM&R isn't the specialty for you--there are plenty of outpatient jobs out there. If outpatient PM&R is where you really see yourself, try to do as much "outpatient" exams as you can on your patients--most have some pathology, so why not catch it, treat it, improve your patient's lives, improve your own in the process, and learn/hone those outpatient skills?