Do what you are most interested in and what will keep your GPA up while still hitting the med school requirements. Usually they correlate pretty well.
GPA is more important than LOR (though obviously both are important). I see no reason why you wouldn't be able to also get great letters from physiology or public health if you think you can get them from neurobiology. Who writes them is not nearly as important as what they say (at the undergrad-med school transition phase).
Being prepared for med school will do you no good if you don't get in.