Artificial intelligence is taking on a growing role in social media moderation.
It reviews posts, identifies potentially harmful content, recommends what users see, flags misinformation and determines which content deserves more or less visibility. Platforms are increasingly relying on AI to perform tasks that once required human judgment.
That shift has an obvious benefit: scale.
It also creates a governance problem.
The question should not simply be whether AI can moderate content. It should be who is ultimately responsible for the decisions AI makes, what objective the system is optimizing for and where humans remain in control.
For Isvari Maranwe, an AI policy analyst, attorney, and founder of Yuvoice, the answer is not to remove AI from social media. It is to fundamentally change its role.
AI should help humans moderate, not replace them
Content moderation is one area where AI can provide enormous value.
Human moderators may encounter deeply disturbing material, including graphic or exploitative content, as part of their work. The psychological consequences of that work can be severe, including trauma and PTSD.
AI can provide an important first layer of review, identifying potentially harmful material before it reaches a human moderator.
The same principle applies to misinformation.
AI can serve as a first-line fact checker, flagging questionable claims, and sending them to people for further review rather than requiring humans to manually identify every potential problem.
“The right role for AI is often to take the first pass on the things humans should not have to endure or manually process at scale,” Maranwe says. “AI can identify patterns, flag potentially harmful content, and surface misinformation, but humans should remain responsible for the judgment that follows.”
That distinction is important because AI is increasingly capable of understanding context.
It can recognize that a post is discussing a particular political issue even when the user avoids obvious keywords. It can identify coded language, images, and references that a traditional keyword-based moderation system might miss.
But greater contextual understanding can also mean greater censorship.
When AI understands what you meant
Users have already adapted to social platforms by changing the words they use.
Some users have adopted alternative wording when discussing sensitive subjects, partly in response to platform moderation practices. Users have substituted imagery and coded references for explicit language. Yet as AI becomes better at interpreting context, those workarounds become less effective.
An AI system does not necessarily need a prohibited word to understand what a post means.
It can infer meaning from the surrounding language, images, and cultural context.
That creates a new question: when does effective moderation become over-moderation?
AI can also influence what users see through recommendation and content-ranking systems. Systems that give certain content greater visibility may also reinforce existing biases or repeatedly expose users to similar viewpoints.
The problem is not necessarily that the AI is malfunctioning.
It may be doing exactly what it was designed to do.
That brings the governance discussion back to objectives.
Social media giants are AI-led, human-backed
Traditional social media governance has often placed humans in charge of policy while using technology to enforce those policies.
Maranwe believes the balance is increasingly shifting.
“Today's social media is becoming AI-led and human-backed, which is deeply problematic,” she says. “We need to move toward human-led, AI-backed systems where people establish the values and objectives, and AI helps us execute them consistently and at scale.”
That philosophy is central to Yuvoice.
Rather than allowing algorithms to determine what users should see primarily according to engagement-driven objectives, Yuvoice is developing an AI-backed system that rewards users for contributions to real-world projects.
The distinction matters because algorithms do not independently determine what is valuable.
People do.
If a system is optimized for engagement, it will pursue engagement. If it is optimized for outrage, it can reward outrage. If it is optimized for time spent, it will identify ways to keep users on the platform.
The objective is a design decision.
And that decision deserves governance.
Humans can expand the objective
There is another limitation that deserves more attention.
AI systems learn from existing data. Humans, by contrast, can deliberately challenge existing assumptions and introduce ideas that are not represented in the training data.
That makes human judgment particularly important when designing the objectives an algorithm should pursue.
“AI is very good at finding patterns in what already exists,” Maranwe says. “Human beings have the ability to question those patterns and decide that we want something different. Governance should preserve that human capacity rather than allowing algorithms to simply reinforce the world they were trained on.”
This is particularly important in social media, where the algorithm is not merely predicting behavior. It can influence behavior.
A recommendation system can influence which voices receive attention. A moderation system can affect which conversations are restricted. A ranking system can influence which perspectives become culturally visible.
Those are not purely technical decisions.
They are societal decisions implemented through technology.
AI can also help govern AI
That does not mean AI should be excluded from moderation.
Quite the opposite.
AI can be useful for identifying synthetic content, detecting coordinated manipulation, recognizing patterns of misinformation, and reducing the amount of harmful material that reaches human reviewers.
The governance model should therefore not be AI versus humans.
It should be AI supporting human judgment.
The difference may sound semantic, but it changes who is accountable.
A human-led system establishes the values, defines the boundaries and determines what outcomes are acceptable. AI then performs the repetitive, high-volume work that machines are better equipped to handle.
That creates an additional layer of accountability: organizations can document not only what the AI system did, but why it was designed to do it.
The future of social media governance
Social platforms will continue to need AI. The volume of online content makes moderation difficult to manage through human review alone.
But greater AI involvement should not mean less human responsibility.
The objective should be to build systems where AI handles the volume while humans retain control over the values.
That means documenting what an algorithm is optimizing, testing for unintended consequences, monitoring whether recommendation systems amplify harmful patterns, and maintaining meaningful human oversight over consequential decisions.
Most importantly, companies should ask a question that goes beyond whether their AI is accurate:
Is it optimizing for the intended outcome?
“The future should not be about making AI the final decision-maker,” Maranwe says. “It should be about using AI to give humans better information, better tools, and better scale while keeping humans responsible for the decisions that shape our digital communities. The future needs to be humans, not bots.”
The technology to moderate the internet is becoming more powerful.
The real governance challenge is deciding who gets to define what that technology is trying to accomplish.
VentureBeat newsroom and editorial staff were not involved in the creation of this content.