[{"data":1,"prerenderedAt":221},["ShallowReactive",2],{"\u002Fai\u002Fmultiforum-llm-moderation-part-1":3},{"id":4,"title":5,"body":6,"createdAt":211,"description":212,"extension":213,"meta":214,"navigation":215,"order":216,"path":217,"seo":218,"stem":219,"__hash__":220},"aiIntegrations\u002Fai\u002Fmultiforum-llm-moderation-part-1.md","An LLM Moderation Plugin, End to End (Part 1)",{"type":7,"value":8,"toc":199},"minimark",[9,13,22,27,39,53,56,60,63,69,72,75,79,82,85,92,96,99,105,109,112,118,121,124,128,131,137,140,144,147,150,153,156,160,183],[10,11,12],"p",{},"Multiforum runs plugins in response to events like new comments and discussions. One plugin sends a comment and the forum's rules to a large language model and asks whether the comment breaks a rule. This post walks through running that plugin end to end on the live site, and what I found.",[10,14,15,16,21],{},"I wrote about the plugin system itself in ",[17,18,20],"a",{"href":19},"\u002Fpost\u002Fdesigning-a-plugin-system-that-can-fail-safely","an earlier post",". This one is about a single plugin and the model behind it.",[23,24,26],"h2",{"id":25},"what-the-plugin-does","What the plugin does",[10,28,29,30,34,35,38],{},"The plugin listens for content events: ",[31,32,33],"code",{},"comment.created",", ",[31,36,37],{},"comment.edited",", and the discussion equivalents. When one fires, it collects the submitted content and the forum's rules and sends them in one request to a separate moderation service.",[10,40,41,42,34,45,48,49,52],{},"The service is a FastAPI app on Cloud Run. It holds the provider credentials, limits its own request rate, retries transient failures, and asks the model for a fixed response shape rather than free text: which rules were violated, a recommendation (",[31,43,44],{},"edit",[31,46,47],{},"remove",", or ",[31,50,51],{},"none","), a confidence score, and a short rationale. If the response is valid and the confidence is above the forum's threshold, the plugin files one report for the forum's moderators, authored by a bot account.",[10,54,55],{},"Nothing is removed automatically. The model flags and explains; a moderator decides.",[23,57,59],{"id":58},"the-test-forum","The test forum",[10,61,62],{},"I tested this in a forum called Bad Advice, whose only rule is inverted:",[64,65,66],"blockquote",{},[10,67,68],{},"Good advice is prohibited.",[10,70,71],{},"It is a forum for deliberately bad suggestions, so the moderator's job is inverted: catch anything genuinely helpful. The model has to judge whether advice is actually good, not scan for banned words.",[10,73,74],{},"One account posted a question about a gaming laptop with a loud fan that overheats. A second account replied with a list of three fixes, written to slip something useful past the moderator. The first two are deliberately bad: put the laptop in the freezer, and tape over the vents so you cannot hear the fan. The third is real, useful advice: clean the dust out of the vents with compressed air and keep the laptop on a hard, flat surface so air can flow. The good tip is buried at the bottom, after two obvious jokes. Will the model catch it?",[23,76,78],{"id":77},"step-1-the-model-flags-the-comment","Step 1: the model flags the comment",[10,80,81],{},"It did. The bot filed a report within a few seconds, catching the one useful tip buried under the two bad ones.",[10,83,84],{},"The screenshot below is what any forum member sees in the issue's activity feed: read-only unless you are a moderator, with the bot labeled as a bot. The report names the rule it matched, recommends removing or archiving the comment, gives a confidence of 0.90, and explains its reasoning. People who are moderated can see exactly what the automated system said.",[10,86,87],{},[88,89],"img",{"alt":90,"src":91},"The moderation bot's report in the issue activity feed, labeled as a bot, with 0.90 confidence and a rationale","\u002Fimages\u002Fllm-moderation\u002F01-ai-report-feed.jpg",[23,93,95],{"id":94},"step-2-a-moderator-reviews-it","Step 2: a moderator reviews it",[10,97,98],{},"A moderator opening the issue sees the recommendation presented as advice, not a decision. The screenshot below shows the review panel: the model's suggestion to remove or archive, labeled 90% confidence, then a decision point (\"is there a rule violation?\") with separate paths for \"no violation\" and \"violation.\" The moderator makes the call.",[10,100,101],{},[88,102],{"alt":103,"src":104},"The moderator review panel showing the recommendation at 90% confidence and separate decision paths","\u002Fimages\u002Fllm-moderation\u002F02-mod-review-recommendation.jpg",[23,106,108],{"id":107},"step-3-edit-with-history","Step 3: edit, with history",[10,110,111],{},"Instead of removing the comment, a moderator can edit it. I changed the one useful tip into more bad advice, swapping the compressed-air-and-airflow fix for blowing warm air into the vents with a hairdryer and resting the laptop on a blanket. The screenshot below shows the activity feed recording the change as a before-and-after diff: the removed text on the left, the new text on the right.",[10,113,114],{},[88,115],{"alt":116,"src":117},"The activity feed showing a before and after diff of the edited comment","\u002Fimages\u002Fllm-moderation\u002F03-comment-edit-diff.jpg",[10,119,120],{},"The edit also re-triggered the bot, and here the model made a mistake. The comment was now entirely bad advice, with nothing genuinely useful left, so the expected result was no report at all. Instead the bot filed a second report claiming the comment still contained good advice. That is a false positive: it flagged a rule violation that was not there. Solving that over-flagging is the subject of the next post.",[10,122,123],{},"Each edit is a revision. Content history and moderation actions share one timeline, so the record of what happened to a comment stays auditable.",[23,125,127],{"id":126},"step-4-close-the-issue","Step 4: close the issue",[10,129,130],{},"With no useful advice left, the moderator closes the issue as no violation. The screenshot below shows the issue marked Closed, with the history preserved: the original report, the edit, and the close.",[10,132,133],{},[88,134],{"alt":135,"src":136},"The issue closed as no violation","\u002Fimages\u002Fllm-moderation\u002F04-issue-closed.jpg",[10,138,139],{},"End to end, on the live site: the model flags a comment, a moderator reviews the report, edits or resolves the content, and the history is recorded.",[23,141,143],{"id":142},"what-i-found","What I found",[10,145,146],{},"Getting this far took several fixes. The report was not firing at all until I traced the request through the pipeline: a pipeline that queued work but never ran it, rules that were dropped before reaching the model, a response shape the model was not told to produce, and a permission gap that made any bot-enabled forum's pages fail to load. Each was a small fix.",[10,148,149],{},"The model itself has a clearer problem. After I edited the comment down to only bad advice, the model flagged it again as containing good advice, with 0.90 confidence, on content that had none. I checked that the pipeline sent the edited text and not an earlier version, and I reproduced the same result by sending that text to the model directly. So it is the model, not the pipeline.",[10,151,152],{},"The cause is the setup: a one-line inverted rule (\"good advice is prohibited\") and a small, fast model. The model sees advice and flags it. That is a real problem. A moderator who has to clear false positives will stop trusting the tool.",[10,154,155],{},"The plugin works end to end, and the model over-flags. That raises a question: can we fix this by being more specific in the prompt or in the forum's rules, or do we need a pricier model? The next post works out whether being more specific is enough, or whether upgrading the model is really necessary.",[23,157,159],{"id":158},"engineering-decisions","Engineering decisions",[161,162,163,171,177],"ul",{},[164,165,166,170],"li",{},[167,168,169],"strong",{},"The main tradeoff."," The model runs as a separate FastAPI service called through a plugin pipeline, and it produces a report for a human rather than acting on content itself, so a wrong verdict creates review work instead of removing a comment.",[164,172,173,176],{},[167,174,175],{},"An alternative I rejected."," Calling the model inline from the backend, or letting it remove content directly, would spread provider credentials and a probabilistic decision through the application; a service boundary and a human checkpoint keep the blast radius of a bad call small.",[164,178,179,182],{},[167,180,181],{},"How I validated it."," I traced a report that never fired through the pipeline to four separate defects, then confirmed the remaining over-flag was the model and not stale data by reproducing it with the exact edited text sent straight to the service.",[10,184,185,186,192,193,198],{},"The moderation plugin lives in the ",[17,187,191],{"href":188,"rel":189},"https:\u002F\u002Fgithub.com\u002Fgennit-project\u002Fmultiforum-backend",[190],"nofollow","Multiforum backend",", and the model-facing service it calls is a separate ",[17,194,197],{"href":195,"rel":196},"https:\u002F\u002Fgithub.com\u002Fgennit-project\u002Fmultiforum-plugin-llm-moderation-service",[190],"FastAPI app on GitHub",".",{"title":200,"searchDepth":201,"depth":201,"links":202},"",2,[203,204,205,206,207,208,209,210],{"id":25,"depth":201,"text":26},{"id":58,"depth":201,"text":59},{"id":77,"depth":201,"text":78},{"id":94,"depth":201,"text":95},{"id":107,"depth":201,"text":108},{"id":126,"depth":201,"text":127},{"id":142,"depth":201,"text":143},{"id":158,"depth":201,"text":159},"2026-11-03","Running Multiforum's LLM moderation plugin against the live site: how a comment gets flagged, reviewed, and resolved, and where the model still over-flags.","md",{},true,3,"\u002Fai\u002Fmultiforum-llm-moderation-part-1",{"title":5,"description":212},"ai\u002Fmultiforum-llm-moderation-part-1","AUuEL0_HgIQoF4ozDPg1f8a-0z65TNEFt8Qgmwvv_vc",1793696720114]