I'm probably a minority opinion on this but I think it's fine for the OP to get frustrated and show it.
We validate far to much absolutely horrible behavior from patients. Refusing to answer simple questions from a doctor who you came in to see is awful. If you are in too much pain to talk, I understand that, but most of these patients are not.
I have ejected multiple patients for failing to cooperate with the history and physical. Obviously you have to select your candidates pretty carefully here but recently I had a 21 f come in for "syncope" by EMS who wouldn't talk at all. Just sat on bed and stared at us. Asked her several times, verified she spoke english, etc. Then I said very clearly: "you can either answer my next question or I am going to removed by security and document that you are attempting to defraud the hospital and the EMS system so that your insurance will not pay for this visit. The other option is that I can have you committed to a psych facility. There is no 3rd option. Now why are you here?" All of a sudden her ability to talk returned.
If patients will not fully participate then I just leave the room and wait a minimum of an hour before going back it. When teh family comes out saying they need pain medicine/juice/sammich I say, "sorry, your mother hasn't allowed us to evaluate her so we can't treat her yet."
I don't think we have a duty to haul information out of people. We have a duty to provide care but if the patient isn't on board there isn't much we can do.
I worked with a guy at one of my ML gigs who takes it a step further. He tells a lot of patients, "you made a really bad decision today to come to the ER." He gets lots of complaints but he also gets tons of compliments and is probably the staff's #1 favorite doc.