Google、UCLA 与 CMU 联合发布 GenXR-Bench,评测 AI 编码工具构建 XR 应用的能力
Google、UCLA 与 CMU 的论文提出 GenXR-Bench,用 112 个专家编写的任务评测 AI 编码工具构建 XR 功能、修复缺陷和开发完整应用的能力。
New Google + UCLA + CMU paper finds that AI coding tools often fail to build virtual and mixed reality apps because they lose track of where things are in 3D space.
Claude Opus 5 led at 50.9%, and no model topped 29.2% on SDK feature tasks. Coordinate-frame mistakes appeared in 64.4% of failed runs, and startup race conditions in 44.9%.
XR apps run in 3D space and react to head motion, hands, and room sensing. Normal coding benchmarks use unit tests or browser clicks, which cannot check whether a virtual button follows your palm.
They built GenXR-Bench, 112 expert-written tasks covering XR feature work, bug fixes, and full app builds. Its tests simulate hands and rooms, and matched blinded XR experts using real headsets 96.5% of the time.
If you use agents for XR apps, test the result with simulated hand and room input, since agent-written unit tests often miss runtime bugs.
来源:rohanpaul_ai · x.com