Probably not as there are a lot of hemepath fellowship programs out there.
Advantages of doing a residency at a program with fellowships:
1) The fellowship director will likely get to know you, and potentially you have an advantage in application (which may not get you the spot, however)
2) You will work in residency with acknowledged experts in the field, who can write you letters and provide you contacts with outside programs
3) The program has sufficient volume to support a fellowship, which means your residency training is not going to lack in that area either for specimen volume or teaching expertise (i.e., a program without a dermpath fellowship, for example, may have less dermpath volume and not train you as well)
Those are the major reasons. But many people get fellowships from outside programs. If you really like a program you should probably go there. You may have to work a bit harder at expressing interest in the field, and perhaps doing an away rotation, but you might not have to go that far.