I am not in this position (I'm internal med -> adult HPM), However I would suppose that there would be some possibilities
For example my professor (attending adult HPM doc, originally PM&R trained) was offered a pediatric palliative job while he was finishing his HPM fellowship. So theoretically one could do the "reverse".
Since the majority of "adult HPM" fellowship is taking care of adults and elderly pts, you would theoretically be qualified. Maybe someone with first hand experience can provide input?
One important thing to know is that all "normal" (adult) HPM fellowships have at least 4 weeks of "token" pediatrics rotations per ACGME (or is it ABIM?) rules (and there are also a smaller number of pediatric HPM fellowships that focus specifically on peds).