Roblox open-sources Sentinel AI to detect harmful chat messages

• Built to spot grooming and child endangerment through chat pattern analysis.
• Sentinel scans over six billion messages daily in real-time.
• Trained on past violations of Roblox’s child-endangerment policy.
• Helped identify 1,200 potential child exploitation cases in 2025.

Roblox is open sourcing its AI-based Sentinel tech designed to detect grooming or child endangerment by identifying chat patterns.

The firm said Sentinel works by analysing over six billion chat messages daily in real time, using one-minute snapshots to flag potentially harmful content and track patterns for further investigation.

The system flags messages by comparing them to patterns learned from past reports of violations against Roblox’s child-endangerment policy.

A proactive approach 

Roblox claimed that so far in 2025, Sentinel has helped identify 1,200 potential child exploitation cases reported to the National Centre for Missing and Exploited Children. 

“In the first half of 2025, 35% of the cases we’ve detected are due to this proactive approach, in many cases catching them before an abuse report could be filed," Roblox said in a post. 

“When combined with our other moderation systems, Sentinel expands the arsenal of tools we have to detect and act on these potentially serious violations."

The company added: “Bad actors often use subtle, indirect, or coded language - purposely making patterns hard to detect, even for human reviewers. 

“Therefore, our detection systems continually evolve to keep pace with new ways bad actors attempt to evade our systems."

Last month, Roblox introduced new safety and privacy features including Trusted Connections for users aged 13 and up to chat with known contacts, as well as improved tools to manage notifications, screen time, and privacy.