LLMs can now program neural networks by directly editing their weights, but this capability is limited to small networks; frontier models like Claude Opus 3.5 score only 0.535 on the benchmark, revealing significant gaps in weight-space programming.
WeightBench is a benchmark for evaluating how well large language models can directly manipulate neural network weights—reading what networks learned, fixing broken ones, writing new networks, and compressing them—without using gradient descent. Testing frontier models on 42 tasks shows they can handle small networks but struggle with larger language models.