Toxicity detection

A toxicity detection API that says what kind.

Toxic is not one thing. Detectivision checks a comment or a chat message for insults, hate, racism, sexism, sexual harassment and self-harm, and answers with each flag it raised and how sure it is, in Turkish and English.

What it finds

Seven flags instead of one score

Every account has these text flags. Ask for all of them, or only those your rules need.

  • Insult

    Insults and personal attacks on someone.

  • Hate

    Hate against a group of people for who they are.

  • Racism

    Racist or ethnic hate speech.

  • Sexism

    Sexist statements.

  • Sexual

    Sexual language, or sexual harassment of someone.

  • Self-harm

    Talk of suicide or self-harm, so someone can step in.

  • Bad habits

    Drug use, drinking, smoking or gambling, talked up.

In code

Check a chat message for everything at once

Request
curl https://api.detectivision.ai/api/v1/moderation \
  -H "Api-Key: $DETECTIVISION_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "text_moderation",
    "input": "Shut up, nobody wants you here, loser.",
    "mode": "detect"
  }'
Response
{
  "id": 3310,
  "type": "text_moderation",
  "mode": "detect",
  "status": "completed",
  "flagged": true,
  "flags": [{ "flag": "insult", "confidence": 0.92 }],
  "checked": [
    "bad_habits",
    "hate",
    "insult",
    "racism",
    "self_harm",
    "sexism",
    "sexual"
  ],
  "unchecked": [],
  "tookMs": 710,
  "usage": { "charged": 2, "remaining": 9998 }
}

flagged is the verdict: true when any flag was raised. checked lists every flag that was looked for; one that could not be is in unchecked, and then the check is free.

How it works
  1. Check before it shows

    Send a message or a comment as it was written; nothing is trimmed. Get the answer at once, or later by webhook with "async": true, at half the price.

  2. Act per flag

    Hide hate at once, hold insults for a moderator, and send self-harm to the people who can help. Each flag is its own decision.

  3. Review in Logs

    Requests show in the dashboard’s Logs, where your moderators take your own actions. Your system gets each one as a webhook.

Related
Questions

Frequently asked

Do I get a single toxicity score?

No. You get flagged, true when any flag was raised, and each raised flag with its confidence from 0 to 1. Use flagged as the overall verdict and the flags to decide what to do.

Which languages are supported?

Turkish and English. Text in other languages is checked, but the results are not reliable.

What does a check cost?

A text costs 1 credit per started 2,500 characters when the answer comes by webhook, and twice that when it comes at once. Asking for fewer flags never costs more.

Can moderators review flagged messages?

Every request shows in the dashboard’s Logs. You define your own actions, such as Approve or Ban; when a moderator takes one, your system gets a moderation.action webhook and carries it out. Detectivision never acts on your platform itself.