Content Safety Classifier

Runs inside your own AWS account: your text is processed on a SageMaker endpoint you control, inside your VPC, and is never sent to a third-party API.

This SageMaker model package provides a REST API to classify English text for content safety.

Each input text is classified into one of four categories: safe, harm, nsfw, or bias, with a confidence score. Use it to moderate user-generated content, filter LLM outputs, or triage support and community messages.

The API accepts input as JSON or CSV and supports real-time endpoints and batch transform.

We welcome your feedback at [email protected]

Model and training data The model is a ModernBERT-base (answerdotai/ModernBERT-base) text classifier fine-tuned with PyTorch and Hugging Face Transformers on the public civil_comments dataset. It runs in network isolation on SageMaker, so no data leaves your account.

Known limitations - English only. Other languages are not supported. - The model was fine-tuned on inputs up to 512 tokens. Longer text is accepted, but accuracy is only characterized to that length; split long documents into passages before sending. - The model assigns a single dominant label. Text that is both harmful and NSFW returns only the stronger signal. - It is a triage aid, not a compliance control. Keep a human in the loop for enforcement decisions.

Measured performance - Accuracy 93.9 percent, macro-F1 0.621, measured on 20,000 held-out examples from civil_comments. The split is seeded (70/20/10, seed 42) and was not used in training. - The classes are heavily imbalanced, so accuracy is dominated by the majority class. Weakest classes on the same split: nsfw F1 0.35 (precision 0.23, recall 0.74); bias F1 0.49 (precision 0.39, recall 0.67). Treat low-confidence minority-class predictions as triage, and review them. - Real-time endpoint on ml.m5.large: 248 ms median per request (271 ms p95). - Throughput on the same instance: about 27 texts per second at a batch size of 32.