LLMs show uneven performance on embedded software tasks: they need local code context to generate code effectively, can model state machines but fail at implementing them, and struggle with certain OS-to-OS translations, suggesting developers should choose models carefully and provide surrounding...
RealEmbedBench is a benchmark with 5,142 real-world embedded software tasks testing LLMs on code generation, state machine modeling, and cross-OS translation.