Changes to AI Dungeon’s safety filter

AI Dungeon uses a safety filter that prevents AI models from generating sexual content involving minors. Because detecting this content can sometimes result in false positives, we included a feature that lets players report an issue and continue playing. Reporting a suspected mistake gave players a 30-minute temporary exception so they could continue playing. There was no limit on how often players could use these exceptions.

We’ve made substantial improvements to the filter, and we’re now confident enough in its accuracy to limit temporary exceptions.

Going forward, players will receive five lifetime exceptions to the safety filter (each lasting for 30 minutes, as they do today). Because we expect false positives to be rare, we believe most players won’t need all five exceptions.

Why do we have this safety filter?

Private, unpublished play on AI Dungeon gives you freedom to explore and create. One of the rare boundaries we’ve set is that we don’t allow our AI to generate sexual content involving minors, as we shared years ago with our “Walls Approach” blog post.

We enforce this boundary by putting limits on what the AI generates, not what you write yourself. You will not be suspended, banned, or punished for triggering this filter in private, unpublished play.

How were we able to improve the filter?

Thanks to players reporting false positives and the availability of better models we can use for classification, we’ve been able to improve our safety filter. We’ve improved how the filter recognizes sexual content and evaluates age-related context.

In one recent review of hundreds of user-submitted reports from players who believed the filter made a mistake, we found only one genuine false positive. This gives us confidence that the filter making mistakes will be a rare occurrence going forward.

What to do if the filter gets something wrong?

No filter is perfect, and although this new one is significantly more accurate than before, we know there will still be genuine false positives. If you encounter a false positive, please continue to report it as you have in the past. Our team will keep reviewing the reports and use them to improve our classification.

If you use up all five of your content exceptions and believe the filter has made a mistake, please reach out to our team at [email protected]. We will review the content of your reports and restore any exceptions that were used for genuine false positives.

Rollout and feedback

This change is effective immediately, and your five-window allowance will start counting now. Any past reports you’ve submitted are not counted. Everyone starts with a fresh five.

We’ll continue using your feedback to improve the detection of sexual content involving minors. Our goal is to eliminate as many false positives as possible so you can enjoy the creative freedom that’s possible on AI Dungeon.