Sorry to not comment on this earlier - been a bit busy.
I love letters from neurologists who have worked with the applicant. They will definitely be seen as stronger in support of your ability to be a neurologist.
If you have one from a non-neurologist, it is not terrible so long as it is a really strong, glowing endorsement - internal medicine ones are probably next best for this.
Three letters are fine - pick your best three and use those - lukewarm letters can hurt you.