[{"data":1,"prerenderedAt":128},["ShallowReactive",2],{"content:\u002Fguides\u002Fcontent-moderation-system-design":3,"ArticleBody_56KsMbPdPrmVcJP0s4HlSu1VNtA36zCPIdZjd6O1M":123},{"kind":4,"path":5,"locale":6,"title":7,"htmlTitle":8,"description":9,"navTitle":7,"published":10,"updated":10,"alternates":11,"breadcrumbs":12,"indexable":20,"twin":21,"collection":22,"slug":23,"file":24,"content":25,"hub":18,"titleEm":27,"eyebrow":28,"lead":29,"cta":27,"faq":30,"faqJsonLd":46,"sources":52,"related":100,"topic":27,"jsonLdType":121,"blocks":122,"shortDefinition":27,"feed":20},"article","\u002Fguides\u002Fcontent-moderation-system-design","en","How to design a content moderation system: checks, rules and queues","Content Moderation System Design Reference · Detectivision","A reference design for moderating user content: where to check, when to block or hold, how to route doubtful cases to people, and how decisions reach your app.","2026-10-16T18:00:00+03:00",[],[13,16,19],{"name":14,"path":15},"Home","\u002F",{"name":17,"path":18},"Guides","\u002Fguides",{"name":7,"path":5},true,"\u002Fguides\u002Fcontent-moderation-system-design.md","guides","content-moderation-system-design","content\u002Fen\u002Fguides\u002Fcontent-moderation-system-design.md",{"collection":26,"path":5},"guides_en",null,"Guide","Where checks run, when they answer, how scores become publish, hold or block, and how a moderator's decision reaches your app, with the API's real endpoints and events.",[31,34,37,40,43],{"q":32,"a":33},"What should a content moderation system design interview answer cover?","Start with requirements: content types, languages, volume and how long each kind of content may wait. Then cover where checks run, whether they answer at once or in the background, how scores become publish, hold or block with a line per category, the review queue and actions, what happens when checks fail, the decision record and appeals, and how you measure precision and recall on a labelled sample. The interview checklist on this page follows that order.",{"q":35,"a":36},"Do I have to check old content again when my rules change?","Not when you only move a line. Keep each result's flags and confidence in your own database and run the new rules over the stored results: lines sit at 0.5 or above, so every finding they need is already stored. Check again only for flags a partial result could not check, or for a flag your requests did not ask for: a flag left out of a request is never looked for. A new check is charged like the first, and an image is fetched again from its URL.",{"q":38,"a":39},"How do I stop a webhook from being processed twice?","Deliveries can arrive more than once and out of order. Each message carries a webhook-id that stays the same on every retry: store it under a unique key and skip ids you have already handled, and use the message's timestamp to apply events in order. Answer with a 2xx within 10 seconds and do the work afterwards, so slow processing does not cause retries.",{"q":41,"a":42},"Should automated and human moderation decisions be recorded differently?","Keep them in one record, with a field that says who decided: your rules or a person. Some laws ask about the difference. For hosting services offered in the EU, the Digital Services Act requires the statement of reasons to include, where applicable, information on the use of automated means (Art. 17(3)(c)). For social networks reachable from Turkey, a regulation applying from 1 November 2026 says appeals about age checks, suspended or closed accounts, and measures in the restricted service for users aged 15 to 17 cannot be examined by fully automated systems (Art. 10\u002F1 and 10\u002F3).",{"q":44,"a":45},"Can one moderation pipeline handle both text and images?","Yes. With Detectivision both go to the same endpoint and come back in the same shape; only the type and the input differ. Text is sent as it is, and an image as a URL that must stay reachable until the check runs. The image is not kept after the check, so keep your own copy for reviewers. Video, audio and live streams are coming soon.",[47,48,49,50,51],{"q":32,"a":33},{"q":35,"a":36},{"q":38,"a":39},{"q":41,"a":42},{"q":44,"a":45},[53,58,62,65,68,71,74,77,80,83,86,89,92,96],{"title":54,"url":55,"publisher":56,"accessed":57},"Detectivision API status (GET \u002Fapi\u002Fv1\u002Fstatus)","https:\u002F\u002Fapi.detectivision.ai\u002Fapi\u002Fv1\u002Fstatus","Detectivision","2026-10-16",{"title":59,"url":60,"publisher":61,"accessed":57},"Async requests","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fasync","Detectivision docs",{"title":63,"url":64,"publisher":61,"accessed":57},"Webhooks","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fwebhooks",{"title":66,"url":67,"publisher":61,"accessed":57},"Metadata and actions","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Factions",{"title":69,"url":70,"publisher":61,"accessed":57},"Flags","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fflags",{"title":72,"url":73,"publisher":61,"accessed":57},"Responses","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fresponses",{"title":75,"url":76,"publisher":61,"accessed":57},"Credits and billing","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fcredits",{"title":78,"url":79,"publisher":61,"accessed":57},"Errors","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Ferrors",{"title":81,"url":82,"publisher":61,"accessed":57},"Image moderation","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fimage",{"title":84,"url":85,"publisher":61,"accessed":57},"Text moderation","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Ftext",{"title":87,"url":88,"publisher":61,"accessed":57},"Service status","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fstatus",{"title":90,"url":91,"publisher":61,"accessed":57},"API reference: Moderate content (POST \u002Fapi\u002Fv1\u002Fmoderation)","https:\u002F\u002Fdetectivision.ai\u002Fdocs\u002Fapi\u002FcreateModeration",{"title":93,"url":94,"publisher":95,"accessed":57},"Sosyal Ağ Sağlayıcı Tarafından Çocuklara Özgü Ayrıştırılmış Hizmet Sunulmasına ve Yaş Doğrulamaya İlişkin Yönetmelik (Resmî Gazete, 10 October 2026, issue 33396)","https:\u002F\u002Fwww.resmigazete.gov.tr\u002Feskiler\u002F2026\u002F10\u002F20261010-15.pdf","Resmî Gazete",{"title":97,"url":98,"publisher":99,"accessed":57},"Regulation (EU) 2022\u002F2065 (Digital Services Act)","https:\u002F\u002Feur-lex.europa.eu\u002Flegal-content\u002FEN\u002FTXT\u002F?uri=CELEX:32022R2065","EUR-Lex, Publications Office of the European Union",[101,105,109,113,117],{"path":102,"title":103,"description":104},"\u002Fglossary\u002Ftypes-of-content-moderation","Types of content moderation","Six ways to moderate user content, from checking before anything is posted to acting on reports: how each works, what it costs in delay, and when it fits.",{"path":106,"title":107,"description":108},"\u002Fglossary\u002Fhuman-in-the-loop-moderation","Human-in-the-loop moderation","Human-in-the-loop moderation sends what software is unsure about to people. How a review queue works, what to record, and where the law requires a person.",{"path":110,"title":111,"description":112},"\u002Fguides\u002Fai-hate-speech-moderation","Can AI moderate text for hate speech and harassment?","Yes, for most of it: a model reads every message and says which kind of abuse it finds and how sure it is. What it does well, where people decide.",{"path":114,"title":115,"description":116},"\u002Fguides\u002Fpay-per-request-content-moderation","The benefits of pay-per-request content moderation for SaaS.","Why a SaaS product pays per check for moderation: costs follow usage, nothing to run, failed checks are free. And when another way fits better.",{"path":118,"title":119,"description":120},"\u002Fuse-cases\u002Fonline-communities","Online communities","Moderate forum posts, comments, usernames and avatars for insults, hate, sexual content and self-harm, in ten languages, with one API.","TechArticle",false,["Island",124],{"key":125,"result":126},"ArticleBody_56KsMbPdPrmVcJP0s4HlSu1VNtA36zCPIdZjd6O1M",{"head":127},{},1792166153975]