SWE-Game

Purpose

SWE-Game provides tasks and runtime evaluation for coding agents building, completing, repairing and porting games. It checks mechanics and playability through game execution, with optional visual assessment.

Getting started

Follow the repository setup guide on Linux x86-64, configure your model provider, select a fixed task package and run an agent or evaluate an existing submission. Keep the evaluator-owned hidden material separate from the agent’s visible task package.

Requirements and limits

The documented setup uses Ubuntu 22.04/24.04, root or sudo, user namespaces and Godot 4.5.1. Unity porting requires licensed Unity 6000.3.23f1 in the documented Docker workflow. Model calls can incur fees. The v1.0.0 release catalog has 246 tasks; the paper’s reported evaluation used 247, so record the exact task inventory and scoring configuration. Benchmark code and documentation are Apache-2.0, but game assets retain component-specific licenses and some permission records remain unresolved. No benchmark run was performed for this listing.

GitHub / SWE-Game