Formality Classifier

Runs inside your own AWS account: your text is processed on a SageMaker endpoint you control, inside your VPC, and is never sent to a third-party API.

This SageMaker model package provides a REST API to classify English text as formal or informal, with a confidence score.

Use it to route or adapt content by register: enforcing a professional tone in business communications, filtering training data for style, or powering writing assistants.

The API accepts input as JSON or CSV and supports real-time endpoints and batch transform.

We welcome your feedback at [email protected]

Model and training data The model is a ModernBERT-base (answerdotai/ModernBERT-base) text classifier fine-tuned with PyTorch and Hugging Face Transformers on the public osyvokon/pavlick-formality-scores dataset. It runs in network isolation on SageMaker, so no data leaves your account.

Known limitations - English only. Other languages are not supported. - The model was fine-tuned on inputs up to 512 tokens. Longer text is accepted, but accuracy is only characterized to that length; split long documents into passages before sending. - Binary output (formal or informal) derived from human formality ratings, which are subjective near the middle of the scale.

Measured performance - Accuracy 84.0 percent, macro-F1 0.838, measured on 2,254 held-out examples from pavlick-formality-scores. The split is seeded (70/20/10, seed 42) and was not used in training. - Real-time endpoint on ml.m5.large: 249 ms median per request (265 ms p95). - Throughput on the same instance: about 24 texts per second at a batch size of 32.