Model Sensitivity - Part Two

In part one, I talked about different ideas of what sensitivity could mean and the planned approach for the Wingman Jr. model. Today I'd like to talk about the software features around sensitivity that I'm working on for the 1.0 release.

As discussed last time, a binary classifier is simply a program that says yes or no given some input. In our case, the input is a picture and the model says "pass" or "block" as the two possible outcomes. A typical way of grading this is to consider the tradeoffs between how many images are correctly blocked vs. incorrectly blocked at different thresholds. This allows us to pick tradeoff points for different types of model behavior.

For Wingman Jr., I've picked three tradeoff points that correspond to three ways the user might wish to use the plugin, based on what "zone" they are browsing in.

1. The user trusts the zone they are browsing in - pick a tradeoff that rarely falsely flags an image as bad, but still catches well over half the bad images as a safety net.
2. The user does not trust they zone they are browsing in at all - pick a tradeoff that catches almost all bad images, but falsely flags a number of images as well.
3. The user has a neutral opinion about the visual safety of the zone they are browsing - pick a tradeoff that balances catching most of the bad images but also flags some false positives.

Being able to choose the "trusted zone" is great for situations where the user is seeing a number of a times that the model is saying something is bad when it is not. While bad images should get blocked, it is easy to get frustrated if a bunch of images are getting blocked for no good reason.

Similarly, if you know you are going to visit someplace that is a bit less safe it is a good tradeoff to block almost all the bad ones at the cost of some good ones.

I'm excited to roll out this feature but I have one more thing included that I'd like to discuss in the next post.  The conclusion of the series is now available here: Model Sensitivity - Part Three