A couple of things here:
First, pedi endocrine is not a competitive match, even for IMGs. As long as you are a reasonable applicant and can speak English understandably, there shouldn't be much of a problem in finding a spot.
Second, fellowship interviews are generally low key. They don't really care much about why you want to be an endocrinologist, as that is a moot point by then. Rather, they want to learn about you and about your plans for the future. so, if asked "Why do you want to be an endocrinologist?", the answer might be something like "I am fascinated by the complexities needed to provide comprehensive care to children with Type 1 diabetes. I am excited potentially to do research into new therapies (such as....) and look forward to doing clinical (or basic science) studies on .......... as a fellow."
In other words, they are asking this question to learn how you might best fit into their program, not to have you prove to them your love of pediatric endocrinology. They also want to see how you communicate and learn about your training and interests a bit.
Finally, your "feeling" about how the interview(s) went may not be accurate. Answer questions truthfully and express knowledge and interest in the program, and you'll be fine.