Coordinating GPU resource allocation, cache-aware scheduling, and environment execution can significantly speed up batch processing of agent workloads without sacrificing accuracy.
This paper optimizes batch execution of LLM agent tasks by coordinating inference and environment interaction. It introduces resource allocation strategies for hybrid LLMs, trajectory-aware routing for cache reuse, and Chain-Advancing Action Scheduling to handle environment bottlenecks—achieving 10-65% makespan reductions on real benchmarks.