Content moderation checks conversation inputs and model outputs against a moderation service. You point the platform at an OpenAI-compatible service and choose which categories to enforce on each surface.
Where to find it#
Open Content moderation in the admin console. Only superadmins can manage the moderation service and policies. Other administrators see a note that stats are under Statistics and events are under Logs.
The page has two sections: Moderation service and Moderation policies.
Enable content moderation#
- Turn on Enable content moderation.
- Configure the service fields below.
- Select at least one moderation category.
- Click Save.
When the switch is off, moderation does not run, but the service and policy settings are kept. You must save an API key before you can enable moderation or test the connection.
Configure the moderation service#
| Field | Required | Notes |
|---|---|---|
| Enable content moderation | Yes | Turns moderation on or off. |
| Moderation service URL | Yes | An OpenAI-compatible service root, a /v1 URL, or a full /moderations URL. |
| Moderation model | Yes | The model used for text and image moderation. |
| Moderation API key | Yes | Write-only. Leave blank to keep the current key, or enter a replacement and save. |
| Moderation timeout | When enabled | Total wait for one check, an integer from 1 to 60 seconds. |
| Moderation max concurrency | When enabled | Moderation tasks that can run at once, an integer from 1 to 64. |
| Moderation queue capacity | When enabled | Maximum queued moderation tasks, an integer from 1 to 4096. |
Click Save to persist changes. If a numeric field is out of range, the error names the field and its allowed range.
Test the connection#
- Save the service settings first. The test button is disabled while there are unsaved changes.
- Click Test connection.
- Read the result. It runs a Text probe and an Image probe, each marked valid or invalid with its latency.
If the probe fails, you see "Connection test failed" with the error message. Fix the URL, model, or API key and test again.
Configure moderation policies#
Policies decide which checks run on which surface. The four surfaces are:
| Surface | What it moderates |
|---|---|
| Input text | Prompts and text sent in the current turn. |
| Input image | Uploaded images used in the current turn. |
| Output text | Text generated by the assistant in the current turn. |
| Output image | Generated images shown as reply attachments. |
For each surface:
- Open the category dropdown.
- Select the categories to enforce. Use Select all to select every category.
- Click Save.
The dropdown lists the categories reported by the moderation service. The trigger shows how many are enabled, for example "Enabled 3/12", or "Disabled" when none are selected.
Policies are disabled until content moderation is on. Hovering the disabled selector shows "Enable content moderation before configuring moderation policies." At least one category must be selected before moderation can be enabled, or you see "Select at least one moderation category before enabling content moderation."
Where results appear#
- Statistics shows moderation counts and latency by day, direction, modality, category, and result.
- Logs → Moderation events (superadmins only) shows each check with its result, direction, modality, latency, and decrypted text or isolated images.
See Logs for the event filters and detail view.
Troubleshooting#
| What you see | What to check |
|---|---|
| "An API key is required before content moderation can be enabled." | Save a moderation API key. |
| "Select at least one moderation category before enabling content moderation." | Select at least one category on any surface. |
| "{field} must be an integer from {min} to {max}." | Fix the out-of-range timeout, concurrency, or queue capacity. |
| "Connection test failed" | Check the service URL, model, and API key. Confirm the service is reachable. |
| Test button is disabled | Save the service settings first; the test runs only against saved values. |
| Policy dropdown is greyed out | Turn on content moderation first. |
| "Only superadmins can manage moderation policy and service settings." | Ask a superadmin to make the change. |