Pretty much. Most specialities expect letters from its own practitioners. Whether you are allowed to "share" a letter from another specialty depends on the field. For example, you could probably use an FM letter from IM (and vice versa), but not likely for Surgery.