1. SDN Mobile is now free on iTunes and on the Google Play Store. Enjoy!
    Dismiss Notice

Updated thoughts on AI/ML

Discussion in 'Radiology' started by Naijaba, Nov 6, 2018.

  1. Naijaba

    10+ Year Member

    Joined:
    Apr 2, 2007
    Messages:
    1,057
    Likes Received:
    105
    Status:
    Pre-Medical
    I figured I'd give a little insight into my experience with what's happening in AI/ML and radiology. There's a great opinion piece from the New York Times that came out today. It echoes my thoughts and prompted me to write this post.

    The gestalt of the article is that deep learning has enabled progress on several once-thought-to-be-impossible challenges, but with the same essential limitation as machine learning models of old.

    Let me give a real-world example from speech recognition.

    Speech recognition plateaued in performance between 1990 and 2010. Despite computer speeds doubling along Moore's Law, speech recognition models would still make the same, infuriating mistakes regardless of processing power. If you've ever used PowerScribe, you know what I'm talking about. On a daily basis, "Low lung volumes" is misinterpreted as "The lung volumes." PowerScribe's flagship product was/is the best in the medical dictation industry, but has long relied on older machine learning models.

    Nuance is, of course, updating their system to use deep learning. Deep learning models broke plateaus in nearly every field, including speech recognition. Try going to Google.com and clicking on the microphone. Google's voice recognition is significantly better that Nuance's, even for many complex medical terms!

    The problem is, both Nuance and Google's systems are still just for voice transcription. They don't make pizza, or fly helicopters, or distinguish images of cats from dogs. The models do one thing - voice to text, and that's it. Those are obviously contrived examples, but what if you wanted the model to do something tangentially related to transcription? Like, transcribing Spanish? Or, translating English to Spanish? Or, just recognizing a word like "ginormous" that it hasn't been trained on? Models can't make inferences outside their learning task.

    Herein lies the rub. Each deep learning model is tuned to a specific task. Many achieve super-human performance on that given task. This is not without utility or clinical significance; a pneumothorax detector that could reliably find basilar pneumos on semi-upright films would be very useful, and likely save a few chest tubes from being removed prematurely.

    I could give more examples, but I started writing this rather late at night. My thoughts on AI are more conservative than in the past. I think we will see some tremendous algorithms helping radiologists with fairly arduous or time-consuming tasks, but the problem of generalizability is daunting. Deep learning is rooted in calculus, linear algebra and probability theory, and these fields belie "intelligence." Neural networks trained using GPUs are simply more advanced model that are ultimately limited by the structure and parameters of their network and the generalizability of the training data.
     
    Gurby likes this.
  2. Mo991

    Joined:
    Aug 2, 2018
    Messages:
    9
    Likes Received:
    4
    So you think AI still have a long way to go in radiology?
     
  3. OP
    OP
    Naijaba

    10+ Year Member

    Joined:
    Apr 2, 2007
    Messages:
    1,057
    Likes Received:
    105
    Status:
    Pre-Medical
    Long is an understatement. There's little to no integration of AI at the radiologist's workstation, and most (all?) vendors are going the traditional PACS-integration route. When the dust settles, I believe AI will be a PACS feature, likely owned by the major vendors.

    Each AI model solves a specific problem: pneumothorax detection, rib fracture detection, bone age classification, pulmonary nodule detection, etc. How much money can an institution spend on a product that may help with just one of these tasks? There are some products (e.g. perfusion imaging or RAPID) that target high-reimbursement, high-volume imaging like CT heads, but these are the exceptions rather than the rule.
     
    Gurby likes this.
  4. 808s&heartbreak

    Joined:
    Mar 24, 2017
    Messages:
    25
    Likes Received:
    15
    Status:
    Resident [Any Field]
    Agreed. I think AI in radiology will have a minuscule impact in the grand scheme of things in the daily radiology work flow. I speak as someone who wants AI to succeed.

    The greatest enemy to radiology is declining reimbursement.
     
  5. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    I remember lurking in the forum last year and you were very optimistic in AI replacing radiologist in the near future. What’s the biggest driver leading to your change in opinion. Is it starting your first DR year?
     
    Gurby likes this.
  6. OP
    OP
    Naijaba

    10+ Year Member

    Joined:
    Apr 2, 2007
    Messages:
    1,057
    Likes Received:
    105
    Status:
    Pre-Medical
    Yes, I think my intern year and first few months of radiology opened my eyes quite a bit. My main reason (similar to the NYT article) is limited generalizability. I absolutely believe AI is better than radiologists for many isolated problems (pneumothorax detection, lesion tracking, pneumonia segmentation, bone age classification, etc.). There are certainly models that do multiple tasks (pneumothorax + pneumonia + central line verification + goiter detection + ... ) all in one package, but there's always something the model lacks that's obvious to the radiologist. Here are a few real examples I saw last week: "The patient's eyeglasses are seen in his front shirt pocket." "Right-sided prominent skin fold should not be confused for pneumothorax." "Radiograph mislabeled left and right." I don't think AI will ever replace radiologists without a very broad contextual awareness.

    Another problem from a business side is limited volumes. The value proposition of AI is increased accuracy and cost reduction. I believe AI can improve accuracy in certain contexts, but cost reduction is dependent on the number of studies performed. The millions of yearly radiology studies pales in comparison to the number of spoken words that could be translated or transcribed. That is, there's a much, much larger market for voice-to-text AI, foreign language translation AI, self-driving AI, advertising AI, etc. than there is for radiology. Moreover, companies are targeting one diagnosis in one imaging modality (e.g. lung cancer detection on low-dose screening CT, bone age in pediatric hand radiographs, pneumonia in chest radiographs, etc.). There simply isn't enough volume to make these companies viable with single-diagnosis products. The only exception I know of is acute stroke detection through perfusion imaging (iSchemaView RAPID). Maybe lung-cancer screening and nodule tracking will turn out to be a viable product, but the space is inundated with startups and old players alike. There isn't enough volume to go around.
     
  7. Mo991

    Joined:
    Aug 2, 2018
    Messages:
    9
    Likes Received:
    4
    In my opinion, AI will get better and better if the funding continue.
    Eventually, it will be able to interpret the normals very accurately...
    To me, the future is very blurry
     
  8. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    What make you think AI will continue to improve with just funding?

    I think what the OP has alluded to is that without a general artifical intelligence, it’s difficult to automate radiology completely.

    Throwing funding at something doesn’t solve problems completely. Nuclear fusion still isn’t solved.
     
  9. Mo991

    Joined:
    Aug 2, 2018
    Messages:
    9
    Likes Received:
    4
    Narrow AI improves with Data. The more data you give, the better AI will perform.
    For now, one of the problems is that we dont have enough labelled data for AI because data is expensive in radiology (to get/label)
    I'm not talking about AGI. once we develop AGI I think the world will be a much different place for us to be disscussing now.
     
  10. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    Did you read the OP? The whole OP is about how narrow AI can achieve good performence for specific things. Radiology is too general for a narrow AI or an array of it to solve. Have you started cross sectional imaging rotations yet?
     
    Gurby likes this.
  11. Radology

    Joined:
    Jun 3, 2017
    Messages:
    92
    Likes Received:
    111
    No offense, but this is what pisses me off about this whole AI thing. The people who speak the loudest about AI know absolutely nothing about radiology in practice.

    OP was loud as hell last two years about how quick, inevitable and seamless the AI transition would be, and IIRC, was telling us how many less radiologists would be needed to run a department...before dictating a freaking chest x Ray.

    Now he/she has some (still barely any) experience in practical radiology, and immediately start to walk it back. I appreciate the honesty, but the truth is: radiologists are the only people who realize they don’t know everything about radiology.
     
    BorntobeDO?, DeucesHigh3 and Gurby like this.
  12. OP
    OP
    Naijaba

    10+ Year Member

    Joined:
    Apr 2, 2007
    Messages:
    1,057
    Likes Received:
    105
    Status:
    Pre-Medical
    Yes, I admit I was overly zealous. Moreover, apart from the technical feasibility of AI, I question the profitability of AI-companies. In the current environment, accuracy isn't the bottleneck. Volumes are. Radiologists have to look at every image, even those flagged as "normal." IMHO, the best AI product will pre-populate the report, to save the radiologist dictation time. That's a feature I'm looking forward to in PACS.
     
    DeucesHigh3 likes this.
  13. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
  14. Mo991

    Joined:
    Aug 2, 2018
    Messages:
    9
    Likes Received:
    4
    The AI in its current form is more of a algorithmic statistic interpretation.
    We are no way near an AGI and some say that the current techniques are not getting us closer
    But what I am certain of, is that in 50 years, the world will be a lot diffrent then today
     
  15. radsisrad

    2+ Year Member

    Joined:
    Dec 7, 2016
    Messages:
    109
    Likes Received:
    104
    Status:
    Pre-Health (Field Undecided)
  16. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    It's simple. AI has never existed before this point, the same way the engine never existed before the industrial revolution. With that in mind, there is no reason to suspect that our progression from this point onward will be "traditional."

    I'm not sure I understand your claim regarding Moore's Law.
     
  17. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    Actually, AI is more than the industrial revolution.

    The definition of tool is that they are extensions of ourselves, to make us work better. The rational proponents of AI work toward that goal.

    The irrational proponents want to use AI to replace ourselves rather than being extensions of our ability.
     
  18. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    AI could very well serve as a tool to make us work better but a tool can also cause destruction even if that was not the intended goal. This is true for nuclear energy utilization. It can be beneficial for humanity by providing an alternative energy source but could very well destroy the planet if used incorrectly.

    Discounting the dangers would be a foolish oversight.
     
  19. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    Nope, nuclear energy is an extension of our ability to perform physical labor. The way some AI proponents are arguing for AI, they are arguing for replacement of the actual tool wielding human factor.

    AI need to remain a tool, welded by people.
     
  20. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    And my question is: Who's to say we're not opening Pandora's box in our quest to have AI as a "tool wielded by people?"
     
  21. cyberchicken001

    7+ Year Member

    Joined:
    Jun 2, 2011
    Messages:
    290
    Likes Received:
    137
    Status:
    Medical Student
    Are you a pre-med?
     
    LBJLA23 likes this.
  22. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    Why is that relevant
     
  23. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    Because most people who aren’t in medicine don’t understand the limitation of application of AI to medicine.
     
  24. Gurby

    2+ Year Member

    Joined:
    Jul 28, 2014
    Messages:
    1,660
    Likes Received:
    3,811
    Status:
    Medical Student
    This is actually a fascinating article that I would encourage anyone interested in AI to read. There is a Part 2 as well. It's nothing particularly Earth shattering but some interesting, well-written speculation about the future of humanity.

    It has literally nothing to do with radiology though.
     
    PickleRickelus likes this.
  25. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    Fair enough, but would you agree that a large part of it is just pattern recognition? Medical school and residency gets you really good at recognizing those patterns.

    If you're a radiologist, you only have so much time to look at films to become an expert at discerning x vs y on a CT scan whereas an AI could theoretically have every CT scan ever performed and even make predictions based on the accuracy of past findings. No slight against anyone but we're constrained by our biology.
     
  26. Kuratz

    7+ Year Member

    Joined:
    Oct 13, 2011
    Messages:
    44
    Likes Received:
    176
    Status:
    Medical Student
    Just today I was at a tumor board conference where highly-respected surgical oncologists, one with a focus on pancreatic cancer, had to be walked through a scan that showed pancreas tumor involvement of the SMA and another with very subtle metastatic findings. In both cases the surgeons would have operated without a radiologist consultation requiring some attending-level discussion. Being able to effectively communicate your expert opinion to the ordering physician is not an insignificant part of the radiologist's job and rads who can do this well are highly valued. Many diagnostic radiologists also spend quite a bit of time doing procedures and barium studies, much more than the lay person or even many within the medical field realize. Radiologists also absorb legal liability. And no radiology isn't just pattern recognition. My guess is that your perception of radiology is just looking at films in a dark room all day where every study is either normal or pathognomic. I don't doubt that AI will continue to play a larger role in radiology but AI making final reads is nowhere near close even if the technology was available today. Not to mention that pathology, dermatology, retina etc would also be significantly affected. Any field has risks. If tomorrow someone invented an eye drop that could dissolve and restore a cataract then we would need much fewer ophthalmologists. If CMS looked at spine operation outcomes and decided that they don't add value they could slash spine reimbursement. It's almost impossible to accurately predict the future on a long-term timeline. I'm picking radiology because I like it the most. If it goes away then so be it, I'll adapt.
     
    Gurby and Cognovi like this.
  27. doctorsdatdo

    doctorsdatdo Osteopathic Medical Student

    Joined:
    Sep 22, 2018
    Messages:
    134
    Likes Received:
    76
    Is a large part of radiology pattern recognition? Let’s see, QA for equipments, QA for techs, QA for other rads, developing and implementing new protocols, protocoling studies, doing procedures, tumor boards, talking to clinicians, and then there is reading the study and developing an appropriate differential.

    It’s pattern recognition like flying an airlane is just driving a plane from A to B.
     
  28. Mo991

    Joined:
    Aug 2, 2018
    Messages:
    9
    Likes Received:
    4
    I think AI will mostly have an impact on Xrays and films identifying the normal ones.
    It will have some troubles with scans/MRI.
    First of all because you will always need a radiologist to check for the stuff that the AI cant identify (like a breast cancer on an chest CT, or a thrombosis in the heart on a ct scan of the abdomen etc)
    Eventually, once the AI can identify ALL pathologies, anatomical variants.. it will have to know what is clinically important and what is not (a pancreas divisum is relevant if the patient is programed for surgery, not so much if the patient have renal stones)
    And if the AI is as impactfull as they say, it will add so much info that you have to decide what is clinicaly relevant or else you will end up with a 30 pages stupid report.
    Not to mention the work that needs to be done for comparaison, and the critical thinking you have to do in the cases of a complicated scan
    Its not like a cbc where you just put a number for wbc hb etc
    You have a looot of data to handle and you need a general human MD radiologist intelligence to decide what to tell other doctors/patient.
     
    jo_da, PickleRickelus and Gurby like this.
  29. PickleRickelus

    Joined:
    Jul 27, 2018
    Messages:
    319
    Likes Received:
    246
    Meant no disrespect to aspiring or practicing radiologists nor am I trying to downplay the role they play on the patient care team. My aim was to generate a discussion regarding the impact of AI on the field.
     
  30. RadX

    Joined:
    Jul 27, 2017
    Messages:
    46
    Likes Received:
    31
    I have some thoughts,, Why it is always AI vs. Radiologist,, May be it will be Internist + AI vs. Rad or Surgeon + AI vs. Rad? I mean many of the referring physicians are already very knowledgable with their scans and in many cases they know more than the average general radiologist. So with the help of AI, They will be at a very better position to replace radiologists.

    I definitely believe that a human factor is still a must but who said that it must be the radiologist? and even if, I think that will dramatically reduced the number of jobs for radiologists. May be we will have a radiology digital farm with many radiologists sitting on computers reporting studies from many hospitals far away with the help of AI.
     
    Gurby and Mo991 like this.
  31. Cognovi

    Cognovi Knowledge worker
    Physician 2+ Year Member

    Joined:
    Feb 10, 2016
    Messages:
    351
    Likes Received:
    259
    Status:
    Resident [Any Field]
    Turf is a matter of expertise, liability, and choice. A competitor must have the expertise to interpret the study to an acceptable standard, assume the liability if the study is misinterpreted, and choose to spend their time interpreting the study rather than other clinical tasks.
     
    Stop hovering to collapse... Click to collapse... Hover to expand... Click to expand...
    Gurby likes this.
  32. DoctwoB

    7+ Year Member

    Joined:
    Jan 10, 2010
    Messages:
    1,646
    Likes Received:
    677
    Status:
    Resident [Any Field]
    The key point that is often missed I think is that AI, whether better or worse then humans at a given task, is DIFFERENT and will screw up in different ways then a human radiologist would. Like the Tesla autopilot that has a lower accident rate then humans but drove into a lake based on faulty maps/cameras, AI will screw up (even if at a lower rate then a human) in ways that would have been obvious to a human operator.

    Because of that, the future (in all fields of medicine, not just rads) at least in our careers, will be AI assistance where both human and AI perform a task to a higher level of accuracy then either alone.
     
    LikeDaniel, Dro133 and Gurby like this.

Share This Page