Well if it makes you feel any better MedPR, I understand what you're saying and I agree on the word usage you've suggested. Having children is most definitely challenging while in school (or any time of life really!) but I certainly would call it an adversity, which has a negative connotation in my mind.
Using the word "adversity" in a PS to describe the challenge of being in school and having kids probably isn't the best thing due to that negative connotation. So in the sense of being nit picky on the PS, don't describe it as an adversity unless you want to express that negative feeling towards the challenge of having kids.
In regular conversation or on a forum, it's probably not that big of a deal, and it would be silly to point it out.
Anyway, that's my $0.02 on the word adversity.
As for using your challenges in a PS, it's a no-brainer. Do it. As many have said, highlight the strength your family gives you or how having a family has shaped you into the person you are now. Having a family of my own, and having our daughter in the middle of the semester when we were both in school full-time and I was working 3 part-time jobs, I can say I understand how much of a challenge it can be, and it definitely is worth mentioning and will set you apart.
