You want to submit letters from people who know you well and can speak to your abilities well.
A doctor who you watched do a couple of procedures and chatted with for 30 minutes? Absolutely not.
A doctor who you shadowed regularly for a year, worked on clinical trials with, thoroughly discussed his career and your intentions? Sounding better...
A doctor who was your direct supervisor in research or clinic for several years? Absolutely!