deedydas 评述 LLM 可解释性研究：Qwen-32B 中 90% 神经元只负责一件事
deedydas 转述并高度评价一项 LLM 可解释性研究结果，称其为 AI 领域最重要的成果之一：神经网络并非纠缠的结，多义性（polysemanticity）观点是错的，可以直接研究神经元本身而无需用第二个模型展开，Qwen-32B 中 90% 的神经元只负责一项工作。
This is one of the most important results in AI, period.
TLDR: neural networks are not tangled knots (polysemanticity is wrong). You can study their own neurons directly, without first "unfolding" them with a second model. 90% of neurons in Qwen-32B have one job.
Nick Cammarata started OpenAI’s interpretability team and is at Goodfire now. He’s been studying how LLMs work for a decade.
The stakes for understanding and aligning model behavior are the highest it’s ever been. Models have achieved superhuman results in math, can break production software systems, and changes the nature of white collar work. This work may finally allow us to intentionally design LLMs and shape how they behave.
Its a long, phenomenal read.
来源：deedydas · x.com