I know it's a whole field in itself, but do you have any advice on quantifying, or putting numbers to, the accuracy of my classifier?
I'm currently looking at 5 phenotypes, and have built a few classifiers from ~200 images, using ~100 cells for each class. I have also kept images back to use as first seen data to test the ability of the classifier.
I usually plot my accuracy by scoring all, then looking at correct cells versus incorrect. I can also look at the enrichment scores for the image sets as well, which always looks better of course.
Now, I'm comparing the ability of a couple of classifiers against each other - i.e. one looks at all 5 phenotypes, another looks at a subset of 4, then I have two which look at 3 phenotypes.
I'd really like to be able to plot ROC curves for each of these classifiers to directly compare them using AUC. Is there a way to get data out to do this? I've tried using enrichment scores but they aren't really suited to it
Any help much appreciated,