[{"data":1,"prerenderedAt":164},["ShallowReactive",2],{"content:\u002Fguides\u002Fdetect-toxic-comments":3,"ArticleBody_qEzmkoHscSHmb2nOvbptXEffLAVl1nkisJ1fwkFKAM":159},{"kind":4,"path":5,"locale":6,"title":7,"htmlTitle":8,"description":9,"navTitle":10,"published":11,"updated":11,"alternates":12,"breadcrumbs":13,"indexable":21,"twin":22,"collection":23,"slug":24,"file":25,"content":26,"hub":19,"titleEm":28,"eyebrow":29,"lead":30,"cta":31,"faq":34,"faqJsonLd":53,"sources":60,"related":135,"topic":155,"jsonLdType":156,"blocks":157,"shortDefinition":158,"feed":21},"article","\u002Fguides\u002Fdetect-toxic-comments","en","How to detect toxic comments in an app or community","How to Detect Toxic Comments in Your App · Detectivision","How to detect toxic comments: what counts as toxic, the three approaches, what to do with a score, and a working Python and Node.js example.","Detect toxic comments","2026-10-17T01:24:41+03:00",[],[14,17,20],{"name":15,"path":16},"Home","\u002F",{"name":18,"path":19},"Guides","\u002Fguides",{"name":10,"path":5},true,"\u002Fguides\u002Fdetect-toxic-comments.md","guides","detect-toxic-comments","content\u002Fen\u002Fguides\u002Fdetect-toxic-comments.md",{"collection":27,"path":5},"guides_en","detect toxic comments","Guide","Check each comment before others see it. A word list finds the words you list. A classifier or an API reads the whole comment and says which kind of abuse it finds and how sure it is. Then you set your lines: hide above one, send the middle to a person, and let authors appeal. A working example is included.",{"label":32,"to":33},"Toxicity detection API","\u002Ftoxicity-detection-api",[35,38,41,44,47,50],{"q":36,"a":37},"What is toxic comment detection?","Software that reads a comment and says whether it contains abuse such as insults, harassment, hate speech or threats, and often which kind and how sure it is. It can use a word list, a trained classifier or a moderation API. People still decide the cases it is unsure about.",{"q":39,"a":40},"How do I detect toxic comments automatically?","Send each comment to a classifier or a moderation API when it is written. Read which kinds of abuse it flags and how sure it is, then act on lines you set: hide above a high line, send the middle to a person and publish the rest. A word list alone finds only the words on it.",{"q":42,"a":43},"What score should I use as the line for a toxic comment?","There is no standard line. Detectivision raises a flag from a confidence of 0.5, so your line sits above that, and each flag needs its own: 0.8 for hate and 0.8 for insult do not mean the same thing. Sort your own comments by confidence and set each line where harmless comments start to appear.",{"q":45,"a":46},"Should toxic comments be deleted automatically?","Only above a high line, and only for the kinds you are sure about. Hide rather than delete, so an appeal can restore the comment. Hold the middle band for a moderator, tell authors why a comment was hidden, and give them a way to appeal.",{"q":48,"a":49},"Can I detect toxic comments in languages other than English?","Yes. Detectivision's text check is reliable in English, Spanish, Portuguese, French, German, Italian, Turkish, Arabic, Russian and Hindi. Text in other languages is checked, but the results are not reliable. Test comments in each language your community writes.",{"q":51,"a":52},"Is a word list enough to detect toxic comments?","It finds listed words in listed spellings. It passes abuse that uses no listed word, such as a plain insult, and it can block harmless words that contain a listed one. Use it for strings you always refuse, next to a classifier or an API.",[54,55,56,57,58,59],{"q":36,"a":37},{"q":39,"a":40},{"q":42,"a":43},{"q":45,"a":46},{"q":48,"a":49},{"q":51,"a":52},[61,66,70,74,78,82,86,90,94,98,101,105,108,111,114,117,120,123,126,129,132],{"title":62,"url":63,"publisher":64,"accessed":65},"Analyze Comment Toxicity with Perspective API (Firestore extension made by Jigsaw: the TOXICITY definition)","https:\u002F\u002Fextensions.dev\u002Fextensions\u002Fjigsaw\u002Ffirestore-perspective-toxicity","Jigsaw, on Firebase Extensions","2026-10-17",{"title":67,"url":68,"publisher":69,"accessed":65},"Perspective API sunset announcement and transition period","https:\u002F\u002Fperspectiveapi.com\u002F","Jigsaw",{"title":71,"url":72,"publisher":73,"accessed":65},"Detoxify: trained models and code to predict toxic comments (README: labels, training data, limitations)","https:\u002F\u002Fgithub.com\u002Funitaryai\u002Fdetoxify","Unitary, on GitHub",{"title":75,"url":76,"publisher":77,"accessed":65},"Moderation guide (categories: harassment, harassment\u002Fthreatening, hate and others)","https:\u002F\u002Fdevelopers.openai.com\u002Fapi\u002Fdocs\u002Fguides\u002Fmoderation","OpenAI",{"title":79,"url":80,"publisher":81,"accessed":65},"What is hate speech? (the United Nations' definition)","https:\u002F\u002Fwww.un.org\u002Fen\u002Fhate-speech\u002Funderstanding-hate-speech\u002Fwhat-is-hate-speech","United Nations",{"title":83,"url":84,"publisher":85,"accessed":65},"Santa Clara Principles on Transparency and Accountability in Content Moderation, 2.0 (Notice and Appeal)","https:\u002F\u002Fsantaclaraprinciples.org\u002F","Santa Clara Principles coalition",{"title":87,"url":88,"publisher":89,"accessed":65},"obscenity (README: a profanity filter for Node.js, MIT licence)","https:\u002F\u002Fgithub.com\u002Fjo3-l\u002Fobscenity","obscenity, on GitHub",{"title":91,"url":92,"publisher":93,"accessed":65},"bad-words on the npm registry (version 4.1.5, MIT)","https:\u002F\u002Fregistry.npmjs.org\u002Fbad-words","npm",{"title":95,"url":96,"publisher":97,"accessed":65},"Detectivision API status (GET \u002Fapi\u002Fv1\u002Fstatus)","https:\u002F\u002Fapi.detectivision.ai\u002Fapi\u002Fv1\u002Fstatus","Detectivision",{"title":99,"url":100,"publisher":97,"accessed":65},"Toxicity detection API (the text flags and what each finds)","https:\u002F\u002Fdetectivision.ai\u002Ftoxicity-detection-api",{"title":102,"url":103,"publisher":104,"accessed":65},"Quickstart","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fquickstart","Detectivision docs",{"title":106,"url":107,"publisher":104,"accessed":65},"Authentication","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fauthentication",{"title":109,"url":110,"publisher":104,"accessed":65},"Text moderation","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Ftext",{"title":112,"url":113,"publisher":104,"accessed":65},"Flags","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fflags",{"title":115,"url":116,"publisher":104,"accessed":65},"Responses","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fresponses",{"title":118,"url":119,"publisher":104,"accessed":65},"Errors","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Ferrors",{"title":121,"url":122,"publisher":104,"accessed":65},"Async requests","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fasync",{"title":124,"url":125,"publisher":104,"accessed":65},"Webhooks","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fwebhooks",{"title":127,"url":128,"publisher":104,"accessed":65},"Metadata and actions","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Factions",{"title":130,"url":131,"publisher":104,"accessed":65},"Credits and billing","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fcredits",{"title":133,"url":134,"publisher":97,"accessed":65},"Detectivision pricing","https:\u002F\u002Fdetectivision.ai\u002Fpricing",[136,139,143,147,151],{"path":33,"title":137,"description":138},"Toxicity detection","Check comments and chat messages for insults, hate, harassment and self-harm in ten languages, with a flag for each kind of toxicity.",{"path":140,"title":141,"description":142},"\u002Fguides\u002Fprofanity-filter-vs-ai-moderation","Profanity filter vs AI moderation: what each one catches","Profanity filter vs AI moderation: what each catches and misses, from leetspeak and Scunthorpe to context and languages, plus cost and when to use both.",{"path":144,"title":145,"description":146},"\u002Fguides\u002Fai-hate-speech-moderation","Can AI moderate text for hate speech and harassment?","Yes, for most of it: a model reads every message and says which kind of abuse it finds and how sure it is. What it does well, where people decide.",{"path":148,"title":149,"description":150},"\u002Fguides\u002Fcontent-moderation-system-design","How to design a content moderation system: checks, rules and queues","A reference design for moderating user content: where to check, when to block or hold, how to route doubtful cases to people, and how decisions reach your app.",{"path":152,"title":153,"description":154},"\u002Falternatives\u002Fperspective-api","Perspective API alternative","Perspective API shuts down after 31 December 2026. See how its attributes map to Detectivision flags, with the code before and after the move.","building","TechArticle",false,null,["Island",160],{"key":161,"result":162},"ArticleBody_qEzmkoHscSHmb2nOvbptXEffLAVl1nkisJ1fwkFKAM",{"head":163},{},1792193478394]