GAMESLASHWhat 508,192 Steam reviews can teach AI game creators
By GameSlash
A newly published study highlights quality, disclosure and player value, with important limits on comparing AI-disclosed games and procedural generation.
For teams using generative AI to build games, a large Steam-review study offers a useful checklist: inspect the quality of generated content, explain what AI actually does, and test whether it adds something players value. The paper by Northeastern researchers Mahsa Bazzaz and Seth Cooper was published in Proceedings of the ACM on Human-Computer Interaction on November 3, 2026. An author preprint has been available since August 12.
The research, Player Perceptions of Generative AI in Games: A Steam Review Analysis, examines 508,192 English-language reviews of games released from 2010 through 2025. It compares games carrying developers’ AI-content disclosures with games tagged by the community as using procedural content generation (PCG), such as rule-based generation of levels or other content.
A reception gap, with important limits
The PCG group contained 341,447 reviews across 5,186 games; the disclosed-GenAI group contained 166,745 reviews across 5,970 games. In the paper’s overall review comparison, recommendation rates were 86.3% for PCG and 68.4% for disclosed GenAI, a gap of 17.9 percentage points.
These groups are not a controlled experiment. They differ in characteristics such as pricing, genres and release status, and the study uses different signals to identify each group. The result does not establish that using AI caused a lower score, or predict how a particular AI-assisted game will perform.
Price provides one example of why context matters. In the study’s breakdown, recommendation rates for paid games were 86.9% for PCG and 78.9% for disclosed GenAI. Among free games, the rates were 83.1% and 50.1%. These are comparisons within the observed sample, not evidence that charging for a game improves its reception.
What creators can take into their workflow
The researchers also closely read 600 reviews mentioning AI. They intentionally sampled 70% not-recommended and 30% recommended reviews to examine criticism in depth, using one highly upvoted review per selected game. These themes should not be read as percentages of all Steam players.
Quality control came through strongly: reviewers noticed inconsistent art, awkward anatomy, robotic voice work and poor translation. Other criticism concerned ethics or a perception that the developer had invested too little care. Some reviews accepted AI when its use was proportionate or brought meaningful novelty. Mismatches between a disclosure and the AI use players believed they saw also raised trust concerns.
For a small development team, these findings suggest three practical checks before release:
Review generated assets together in the actual game. Check visual consistency, text, voices and translations in context, rather than approving each output in isolation.
Explain AI use accurately and keep the disclosure consistent with the shipped build. Describe the relevant role without claiming that a label alone establishes quality.
Playtest the value players receive. Ask whether an AI-driven feature improves the experience, and fix ordinary gameplay and presentation problems before treating faster production as a player benefit.
Use the findings as a checklist
This is evidence about a particular set of English-language Steam reviews and declared or tagged games. Undisclosed AI use, missing PCG tags and opinions that never mention AI can fall outside parts of the analysis. The authors also note that release dates can change and attitudes may evolve.
For GameSlash creators, the useful lesson is to make quality checks and accurate disclosure part of the production process. The research gives reasons to examine those decisions, not a verdict that every game made with AI will be rejected.
Published paper · Author preprint and license · Open full text and figures