There are always people that are going to be poor communicators in the field, but I feel that is true for every field.
My first two experiences working in clinics were not great ones. The first one, there was a big miscommunication about the hours I would be working (originally was told 7pm-3am when they actually wanted 7pm-7am). I got them to agree to let me leave at 5am because I had classes at 11, but I was sleep deprived and miserable and the job wasn't even fun. The second one, it was just like I didn't mesh that well into the culture. It was a small clinic (talking 3 other employees + the vet) and they were all quite a bit older than me. The next youngest employee was in his 30s and had been working at the clinic since he was 16. Wild. Anyways, twice my boss got mad at me for drinking my coffee up at the reception area and would make me write a list of things that had to be done (essentially, he saw me as being lazy at those moments). We never had any official "break periods" except the people working 8 hour days so honestly that was probably less of a problem with me because legally we are entitled to that, but nonetheless I will say that I have been molded to not be able to sit still at work. Always have to be doing something.
The 3rd clinic that I worked at, granted I am a seasonal worker, has been a great fit for me. But, not everyone there is happy.
Do you like the job? If not, maybe this isn't the right field. If you like the job, think about what you can do differently to fit in better. The answer won't be the clients, or the co-workers, or the bosses... though any of those can make a miserable experience. Those can't be changed, unless you go to another clinic. However, your own personal communication skills can change a miserable experience into an awesome one. I'm not attempting to suggest at all that it is your own fault that this happened. Rather, if you are working with someone that is a bad communicator, being an excellent communicator can almost make that problem vanish.
This is just an example: at the clinic where I am at seasonally, I approached one of the doctors with a question once from a client on the phone. The doctor snapped at me. But, she wasn't angry at me - she was frusterated with the server problems we were having that was making it difficult to do the SOAPs. So, I didn't take it personally; I told her that it sucked she was dealing with those problems and I was sorry that it was so frustrating. And, then she apologized to me and answered my question and everything was totally fine.
So just try to put yourself in your doctor's shoes. If people are complaining around him all of the time, his mood is probably not great. You're right that it probably IS coming right out of the blue, because it has nothing to do with the problems themselves but just the timing that they are presenting.
Despite that you don't even know what issues your co-workers have with you, does it hurt to apologize to your boss? Saying "I'm not sure what I'm doing wrong but I apologize for any trouble I'm causing you" <--- validating his feelings "I will talk to them about the problem so it can be resolved promptly" <---- showing that you are going to be pro-active and taking it off his plate <--- "I really enjoy working here and want to do a good job." <--- end on a positive note.