Text moderation

Text moderation that says which rule a text breaks.

Send a comment, a chat message or a review exactly as it was written. Detectivision checks it for your flags and answers with each flag the text raised and how sure it is. Every flag judges the whole text. In 12 languages.

What it finds

What your rules forbid

Your flags depend on your plan, and some can be set up for your account alone. The usual ones:

  • Name-calling and personal attacks aimed at someone.

  • Attacks on people for who they are: their origin, religion, gender and the like.

  • Profanity

    Swear words and vulgar slang, even when they are not aimed at anyone. The Basic plan has this flag.

  • Sexual language and explicit descriptions.

  • Talk of hurting oneself, so that someone can step in.

  • Bad habits

    Drugs, drinking, smoking and gambling, talked up.

How it works
  1. Send the text

    Up to 10,000 characters as input, as it was written. Nothing is trimmed or translated first.

  2. It is checked for your flags

    Every flag of your account, or only those you name in flags. A narrower request answers faster.

  3. Act on the answer

    Each flag raised comes with its confidence, from 0 to 1. Reject the text, hold it for a moderator, or let it through: your app decides.

In code

One request, one answer

Request
curl https://api.detectivision.ai/api/v1/moderation \
  -H "Api-Key: $DETECTIVISION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "text_moderation",
    "input": "Sen salak birisin, senden nefret ediyorum.",
    "mode": "detect",
    "flags": ["insult", "hate", "sexual", "self_harm"]
  }'
Response
{
  "id": 1050,
  "type": "text_moderation",
  "mode": "detect",
  "status": "completed",
  "flagged": true,
  "flags": [{ "flag": "insult", "confidence": 0.96 }],
  "checked": ["hate", "insult", "self_harm", "sexual"],
  "unchecked": [],
  "tookMs": 620,
  "usage": { "charged": 2, "remaining": 9998 }
}

The answer says the text raised insult and how sure the check is, not which word did.

Questions

Frequently asked

Which languages does it understand?

English, Spanish, Portuguese, French, German, Italian, Turkish, Arabic, Russian, Hindi, Chinese and Japanese. Text in other languages is checked too, but the results are not reliable.

How long can a text be?

1 to 10,000 characters, counted as Unicode code points, so an emoji is one. Split a longer text and check the parts.

How much does a check cost?

1 credit per started 2,500 characters, in every mode, when the answer comes later by webhook. An answer given at once, the default, costs twice that. Errors cost nothing.

Can I check only some flags?

Yes. Name them in flags, from the list your account gets from GET /api/v1/flags. The others are not checked, and the answer comes faster.

Does the answer show which words broke the rules?

It shows which flags the text raised and how sure the check is. Every flag judges the whole text, so the answer holds no word positions and no masked text, whatever mode you ask for. Hide or hold a flagged text in your own app.