I think what a lot of surgeons and other proceduralists don't really understand about consultant opinions and "clearances" is that they can be damning, but very rarely reassuring. That is, if they say a patient is poorly optimized, they're probably correct. However, when they give their stamp of approval, they are often not considering factors that we care about. In the end, anesthesiologists and nobody else can really clear patients for anesthesia.
I always hated the idea of a "periop anesthesia home" clinic where all patients are screened by us on a day prior to anesthesia. I hated that job so much that when I wrote our scheduling software, I had it hardcoded to refuse to allow anyone named pgg to be assigned to the preop clinic. But. I admit that it's useful in avoiding day-of-surgery cancellations. In the end though I don't believe the cost and hassle and pain of such a preop clinic run by anesthesiologists is really worth the rare case that gets flagged ahead of time rather than cancelled on the day of surgery.
A chest xray can be normal despite significant respiratory problems. Same for the CBC. Like the PCP's opinion, they have value when abnormal ... less so when they're normal.
SOB when walking is concerning.
I think you shouldn't have scheduled this case, or even referred the patient to the PCP for an opinion, because you should've recognized that SOB when walking, a few weeks after a pneumonia requiring antibiotic therapy, in a patient with asthma at baseline ... is not the picture of a patient who's optimized for elective surgery.
I also think I probably would have done the case, provided my in-person assessment of the patient was not alarming.