Types of content moderation

Pre-moderation checks content before anyone else can see it; post-moderation publishes first and checks right after; reactive moderation acts only on user reports. Proactive moderation looks for harm without waiting for reports, distributed moderation lets the community vote, and automated moderation uses software for some or all decisions. In practice, platforms mix them by content type and by how far they trust each user.

The six types at a glance

Pre-moderation

Nothing is published until it passes. It keeps harmful content off the platform, at the price of a wait for the poster and, where people review, a queue that must keep up. It fits profile photos, usernames, listings and spaces for young users.

With a content moderation API:

1. The user submits; your server stores the item unpublished.
2. Your server calls the API and waits, or sends the check with "async": true and shows the item as pending.
3. No flag above your line, and none you need left in unchecked: publish. Otherwise reject it or hold it for review.

An answer at once costs twice the async price, so if the poster can wait, the pending route is cheaper; async checks usually run within seconds (async requests).

Post-moderation

Content goes live at once and is checked right after. Nobody waits, which suits busy comments, reviews and chats; the cost is a window in which others can see harm.

1. Publish the item and send it with "async": true and its id in metadata.
2. Get the result by webhook: moderation.completed for every finished check, or moderation.flagged for flagged ones only.
3. On a flag above your line, hide the item and queue it for a moderator.

Reactive moderation

Moderators act only on what users report. It is cheap and uses the reporter's context, but harm stays up until someone reports it, and quiet places may never be reported. Under the EU Digital Services Act, every hosting service must let anyone report illegal content electronically (Article 16), so a platform that serves users in the EU needs a report path whatever else it runs.

1. A user reports an item.
2. Optionally, check it with the API so the queue can be sorted by flag.
3. A moderator takes one of your actions in Logs or your own tool, and your system carries it out (actions).

Proactive moderation

Software or staff look for harm without waiting for a report: an account's older posts after one bad post, or the whole archive after a rule changes. It finds what nobody reports, at the cost of checking much that is fine.

1. Send stored items as async checks, each with its id in metadata.
2. Act on the flagged results as the webhooks arrive.
3. Stay under 1,200 calls a minute per API key; on a 429, wait the seconds in Retry-After (rate limits).

Distributed moderation

Members decide: posts with enough down-votes or reports are hidden, or trusted members work through a queue. It grows with the community and reflects its norms, but it is slow on quiet content, can be gamed by organised groups, and exposes volunteers to harmful material. Voting is a feature you build; an API check can run beside it.

Automated and hybrid moderation

Automated moderation lets software make some or all decisions; hybrid moderation lets it act on clear cases and sends unclear ones to people.

1. A check raises a flag when a finding passes the flag's threshold, 0.5 by default; findings below it never come back (flags).
2. Above your own, higher line, for example 0.8 (illustrative), your system acts alone.
3. Between 0.5 and that line, a person decides.

Some rules require a person for appeals (a summary, not legal advice):

• Turkey. A regulation for social network providers ("sosyal ağ sağlayıcı": services where users create and share content to interact) applies from 1 Nov 2026 to those reachable from Turkey. Appeals about age checks, account suspensions or closures, and measures on accounts of 15-to-17-year-olds need meaningful human review, not fully automated handling (Art. 10/3).
• EU. Under the Digital Services Act, online platforms other than micro and small enterprises (Article 19) decide complaints under the supervision of qualified staff, "not solely on the basis of automated means" (Article 20(6)).

What the timing costs

In Detectivision's documented prices, an async check costs the price table and an answer given at once costs twice that. Text costs one credit per started 2,500 characters async; images are priced per image.

Errors and partial results cost nothing.

Mixing them: by content type and by user trust

Trust moves users between types: pre-moderate new or reported accounts, post-moderate users with a clean history, and put the user's id in metadata to match each result to a trust level. Keep reports open to everyone. Examples by industry: online communities and marketplaces.

What this does not do

Detectivision checks content; it is not a moderation team or a publishing system.

• It does not publish, hide or delete anything. Your system does, when a result or a moderation.action webhook arrives.
• It has no report button, voting or reviewers. Reactive and distributed moderation are features you build, and the people in a hybrid setup are yours.
• Flags judge the whole text or image. No word positions, boxes or censored copies come back.
• Text results are reliable in English, Spanish, Portuguese, French, German, Italian, Turkish, Arabic, Russian and Hindi only (text).
• Video, audio and live stream moderation are coming soon.

Read the docs