AI

4006 readers

1 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 3 years ago

157

‘Embarrassingly simple’ probe finds AI in medical image diagnosis ‘worse than random’ (venturebeat.com)

submitted 2 months ago by [email protected] to c/[email protected]

23 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] can 2 points 2 months ago (1 children)

They used one to create the dataset for their experiments:

In their experiments, they introduced a new dataset, Probing Evaluation for Medical Diagnosis (ProbMed), for which they curated 6,303 images from two widely-used biomedical datasets. These featured X-ray, MRI and CT scans of multiple organs and areas including the abdomen, brain, chest and spine.

GPT-4 was then used to pull out metadata about existing abnormalities, the names of those conditions and their corresponding locations. This resulted in 57,132 question-answer pairs covering areas such as organ identification, abnormalities, clinical findings and reasoning around position.

[–] [email protected] 0 points 2 months ago (1 children)

The seven models tested included GPT-4V, Gemini Pro and the open-source, 7B parameter versions of LLaVAv1, LLaVA-v1.6, MiniGPT-v2, as well as specialized models LLaVA-Med and CheXagent. These were chosen because their computational costs, efficiencies and inference speeds make them practical in medical settings, researchers explain.

It seems like this is a case of "they just aren't using AI right, if they used it right it works" when it sure looks like they are using the models intended for these specific medical tasks.

[–] [email protected] 3 points 2 months ago* (last edited 2 months ago)

Those are not the sort of model anybody in the field would use (medical CV with deep learning based analysis is a vibrant field with many breakthroughs in recent years). These are the sort of models tech bros are trying to sell to the public as general AI. There is a world of difference.