Yes, there has to be...and there is. Your buddy doesn't sound very organized.
How do I get around it? I get my notes done while I'm seeing patients. I take care of phone calls, faxes, forms, and whatnot as they come in throughout the day. I don't let them pile up. I refill prescriptions during regularly-scheduled office visits so I don't have to deal with refill requests in between appointments. I prescribe generics as much as possible, and try to adhere to insurance formularies as best I can. That cuts down on prior authorizations, and my staff is trained to handle the majority of the ones I do get. Patients get their labs done prior to follow-up visits for chronic diseases like diabetes, hyperlipidemia, etc. so I can make all of my management decisions during the visit rather than after the fact. I charge a form fee, which cuts down on stupid/unnecessary form/letter requests. I use our electronic patient portal as much as possible. I don't waste time at the office. When I'm there, I'm there to work.
My desk/in-box is usually clean by the time I leave the office, typically by 5:30pm.
I start my day by logging into our EHR from home while having my first cup of coffee, and take care of anything that came in during the night or over the weekend. This takes anywhere from 5 minutes on a weekday to maybe 20-30 minutes on a Monday. Sometimes I'll go online Sunday night instead, just so I'm not pressed for time on Monday morning. By taking care of things before I get to the office, I start every day with an empty in-box.
"We are what we repeatedly do. Excellence, then, is not an act, but a habit." - Aristotle