Definition
Throughput is the amount of defined work a system completes during a unit of time. Depending on the layer, the unit may be requests per second, generated tokens per second, agent runs per hour, merged changes per day, or accepted durable outcomes per month.
The unit determines what the number means. Token throughput measures serving capacity, not useful work. Task throughput can rise if tasks get smaller. Pull-request throughput can rise while review and incidents accumulate downstream. Factory reporting should pair raw production with acceptance, quality, durability, cost, and work-in-progress measures.
Capacity and bottlenecks
Throughput is constrained by the slowest effective stage, not the fastest generator. More parallel agents may increase implementation volume until evaluation, integration, review, deployment, or human judgment becomes the bottleneck. Past that point, concurrency grows queues and coordination cost rather than completed outcomes.
Distinguish it from nearby terms
Latency describes elapsed time for one item. Throughput describes aggregate completion rate. Concurrency is work in progress. Utilization is the share of capacity in use. Batching can raise throughput while increasing latency, so a single metric cannot describe the system.
Check your understanding
An agent platform doubles patches produced per hour, but the number merged and stable in production does not change. Did factory throughput improve? Generator throughput did. Accepted-outcome throughput did not. Name both and locate the queue between them.