Evaluating images collectively

Anything related to deep learning, machine learning, data science, and artificial intelligence. Course topic requests can go here.
Post Reply
AndyH
Posts: 1
Joined: Mon Dec 16, 2019 5:10 pm

Evaluating images collectively

Post by AndyH » Mon Dec 16, 2019 5:14 pm

I have a problem where I'm trying to use AI to evaluate a large number of images collectively and arrive at an answer for the collection.

So imagine attempting to diagnose cancer in a patient given 1000 images from the patient. The problem I see is that it doesn't matter how great your AI is, even if its correct 98% of the time, in that sample of 1000 it will get it wrong 20 times, now that may mean a false positives or even worse, false negatives.

How are these kind of situations presented to the end user, because from their point of view, they want a reliable AI over the set of 1000 images, and they want an answer that says 'Cancer' or 'No Cancer'. If we quote it as being 98% accurate that's what they expect it to be. But the reality is that the AI will get it wrong 20 times in a sample of 1000 so how do we present that in a useful way?

How do we deal with these situations where the expectation is a collective overall decision, but the AI is executed on an image by image basis?

lazyprogrammer
Site Admin
Posts: 14
Joined: Sat Jul 28, 2018 3:46 am

Re: Evaluating images collectively

Post by lazyprogrammer » Thu Dec 19, 2019 7:35 am

Thanks for your question.

You would be interested in the ROC and AUC. I believe you are taking the courses where this was discussed (Logistic Regression, Deep Learning part 1) so take a look there.

In practice, multiple tests are done.

E.g. an initial blood test, then follow up tests if the first test shows a positive, etc.

Post Reply

Return to “General Discussion”