So how the committee letter works is kind of what you said. With my school (idk if there is any other way in schools) it's a LOR written after they have an interview with you, much like med school interviews, and they write the letter with that in mind, with your other LORs in mind, with all your information, basically all the things you need for med school primary applications.
For me it was pretty good practice for actually doing my applications for AACOMAS, helped me with interviews and the primary application as a whole, but I recommend you still get that committee letter while you're in school. When I applied for the letter, I had already graduated from my school, so I had to go back and do the interview in person with them (I live 8 hours away from my school if that helps paint a picture for you guys). I got lucky that a friend of mine was happy to have me for the week that I was there. It was my best option choice and I wouldn't redo it, but keep it in mind if going back to your school isn't an option later on.
However, the reason people say it's bad if you don't have a committee letter (If I recall right from my pre-med advisors telling me) is because it kind of implies that you didn't try to apply for your committee's letter.....if your school has a committee in the first place. If your school doesn't have one, then y'know, you kinda can't get a committee letter, they can't penalize you for that because you don't have that resource. If that's the case, take note of schools you're interested in and keep track of what kind of letters they want, make sure you can gather the alternates to the committee letter.
I've heard (don't know personally, so if anyone can help me out on this it'd be great) that if you don't have a committee letter, and your school HAS a committee, then yeah it kinda looks bad. Again it kinda questions why you didn't apply for one (and you better have a response if they actually ask that, a GOOD one).
TL;DR, particularly for OP: If you don't have a committee letter because your school doesn't have a committee, then it's not the end of the world, just do your research and plan accordingly for your letters.