From some people involved in Admissions, although not to medschool, I've heard that brevity is actually desired.
Personally, I think it could go both ways. If it is so short that it gives the impression the writer doesn't know your very well (by say, lacking specifics) it could be bad. Even if the letter is favorable overall. Now if the letter is short but has lots of specifics, and is persuasive, it could be really good. Even better than a really long and drawn out letter, if you believe the people I've heard from (involved in residency admissions). The concept there is that most admissions people barely skim over the letters, since they have other things to do/so many LoRs to read. So, short but memorable beats long but good... I guess this varies a lot depending on who is reading the letter.
I'd say it really comes down to how persuasive your writer can be in a short letter. Can he convince the adcoms that you are going to make a good Dr. in 3 paragraphs? It's definitely possible!