You can assign LoR who whichever specialty/program you like.
Word of advice, it's best to assign LoR's based on content. i.e. if a spefic LoR is written for a specific specialty - say peds then it probably wouldn't be a good idea to assign it to so IM. So, if your LoR's are broadly/generally written then assign it to a FP program. If they are specific pediatrics then you could still use it for FP and blow the LoR off as proof that your peds side of FP is good. Conversely, you could ask your LoR writers to write a more general LoR. But whatever you do, you should do it quickly. It's November, and with Thanks Giving and Xmas comming programs should start becomming content with their current applicant pool.
GL