Model Sensitivity - Part One

I got my first feature request today for the Wingman Jr. plugin -thank you, anonymous user! A user was asking for a way to change the sensitivity of detection. As it turns out, I've recently been looking at some related things from the model prediction side, so it is good timing.  I'd like to talk today a bit about how machine learning models view sensitivity vs. how humans view sensitivity. (And for you stats nerds out there - set aside the definition of "sensitivity" for just a bit.)

Let's start with how humans typically view sensitivity. Suppose a human is asked to grade how mature a movie is, perhaps using something similar to the MPAA ratings of G, PG, PG-13, R. We generally expect that a human uses more of a gradual rating, where - for example - increasingly violent content might cause the rating to go from G to R. We would expect a gradual, smooth transition as the objectionable content increases. With this in place, we can then say things like "I'd only like to watch movies that are up to PG-13". So here sensitivity might mean something like "a gradual scale that allows for a cutoff at a certain level of objectionability."

However, for machines this isn't always the case. Often times they take a more probabilistic approach. For example, a quite normal approach is to build a "binary classifier" (a program that just says "yes" or "no" for some input) and look at the percentage of "yes" and "no" that come out. So, you might be able to say about a certain model - "this catches 90% of bad images, but blocks 10% of good ones by accident." While it's not ideal that any should fail, this makes the tradeoffs easy to reason about from a statistical perspective.  So here sensitivity may be roughly defined as "a cutoff point that guarantees a certain statistical balance yes and no classes". (Strictly speaking, the statistics definition is solely concerned with the rate of true positives to true positives and false negatives but I would argue most humans have a much squishier view of the meaning.)

Unfortunately, this statistical view - while useful - often leaves the human judge quite dissatisfied. Grading does not fail gracefully. In our hypothetical example, the human might ask something like "Why did this movie with an intense scene of gore get rated G instead of R? If had at least been PG-13 it would be somewhat understandable, but this is terrible!" This type of failure, the lack of understanding and "smoothness" in grading successes and failures, is one of the key challenges facing real-world use of AI today.

For Wingman Jr., I'm planning to take an approach that hopefully captures some of the spirit of both notions of sensitivity.

A bit of background first. Currently the model is built from a selection of graded images falling into one of four categories: safe, questionable, racy, or explicit. The current strategy has been to do a fairly standard image classifier approach with these four classes, but with weighting to penalize confusions between e.g. explicit and safe more heavily than say between questionable and safe.

So first, with respect to the statistical aspect of sensitivity implementation: I'm working on distilling the prediction of the four classes into simply two: safe and not safe, as a numeric score. This allows for analysis with traditional means like the AUC ROC.

With respect to the common sense aspect of sensitivity: while I can't make the grading perfectly smooth, I can instruct the tradeoff to be sensitive to the severity of the image. So, more "explicit" images will be caught than "questionable" images at a given cutoff threshold because I've purposely over-represented "explicit" images.

I have some initial results that seem promising, but I have some work left to do before it's ready for prime time - a useful piece of software is more than just a machine learning model.

Stay tuned!

The next installment of this series is available at Model Sensitivity - Part Two