They do not
1. Describe precisely what their algorithm is, it's outputs, its architecture, or what populations / images the system was trained on. Their citations describing their system are for random papers dating back to 1998.
2. Demonstrate or cite the "validation"
3. They used a heterogeneous sampling of reads by cobbling together multiple datasets. They did not prospectively test this system (see #2).
4. This is the equivalent of an AI "meta-analysis". They uncritically accepted the outputs from the original trial studies, without using a cohort evaluation approach for each of the studies. The same 101 radiologists did not interpret all the exams; different numbers interpreted variable numbers of studies from each included trial.
5. I do not understand how you can pool 101 radiologists' AUC when every rad didn't read every study. You could compare the system to the individual results for each dataset, but they didn't do that nor present the performance data
It's too hard to evaluate the claims of this article as it is the Radiology/AI equivalent of saying "We gave a validated chemotherapy regimen. It was 'better' than most chemo regimens (trust us)". In short, there's a reason this was published in a non-imaging cancer journal, and NOT in a Radiology journal nor in an engineering/CS journal. I bet they shopped it around.
I'd like to look at the supplemental data, but I can't get access to it.
