2.3 KiB
| type | domain | description | confidence | source | created |
|---|---|---|---|---|---|
| claim | internet-finance | Optimal server provisioning formula derived from queueing theory for systems operating in the QED regime | proven | Ward Whitt, What You Should Know About Queueing Models (2019) | 2026-03-11 |
Square-root staffing rule provisions servers as base load plus beta times square root of base load where beta controls service quality
The square-root staffing principle states that optimal server count = R + β√R, where R is the base load required at full utilization and β is a quality-of-service parameter (typically 1-2). This formula emerges from queueing theory analysis of multi-server systems operating in the Halfin-Whitt (QED) regime.
The rule captures a fundamental efficiency: larger systems need proportionally fewer excess servers because variance scales with √n while capacity scales with n. A system with 100 base servers needs only 10-20 safety margin servers (10-20% overhead), while a system with 10,000 base servers needs only 100-200 safety margin servers (1-2% overhead).
The β parameter controls the service level tradeoff. Higher β means more servers, lower utilization, shorter queues. Lower β means fewer servers, higher utilization, longer queues. The optimal β depends on the cost of servers versus the cost of delays.
Evidence
Whitt's paper presents this as the core practical result from decades of queueing theory research. The Erlang C formula provides the computational foundation for determining exact server counts given arrival rates, service times, and target service levels.
The Halfin-Whitt regime (also called QED regime) is the theoretical framework: systems operate near full utilization (ρ → 1) while keeping delays manageable by approaching full utilization at rate Θ(1/√n) as server count n grows.
Practical Application
For pipeline systems processing asynchronous work (like Teleo's source processing), this means:
- Don't provision for peak load (wasteful)
- Don't provision for average load (queue explosion)
- Provision for average + β√average where β ≈ 1-2
For a system with R=8 base workers (full utilization), optimal provisioning is 8 + 1.5√8 ≈ 12-13 workers.
Relevant Notes:
- domains/internet-finance/_map
Topics:
- core/mechanisms/_map