39 lines
No EOL
2.3 KiB
Markdown
39 lines
No EOL
2.3 KiB
Markdown
---
|
||
type: claim
|
||
domain: internet-finance
|
||
description: "Optimal server provisioning formula derived from queueing theory for systems operating in the QED regime"
|
||
confidence: proven
|
||
source: "Ward Whitt, What You Should Know About Queueing Models (2019)"
|
||
created: 2026-03-11
|
||
---
|
||
|
||
# Square-root staffing rule provisions servers as base load plus beta times square root of base load where beta controls service quality
|
||
|
||
The square-root staffing principle states that optimal server count = R + β√R, where R is the base load required at full utilization and β is a quality-of-service parameter (typically 1-2). This formula emerges from queueing theory analysis of multi-server systems operating in the Halfin-Whitt (QED) regime.
|
||
|
||
The rule captures a fundamental efficiency: larger systems need proportionally fewer excess servers because variance scales with √n while capacity scales with n. A system with 100 base servers needs only 10-20 safety margin servers (10-20% overhead), while a system with 10,000 base servers needs only 100-200 safety margin servers (1-2% overhead).
|
||
|
||
The β parameter controls the service level tradeoff. Higher β means more servers, lower utilization, shorter queues. Lower β means fewer servers, higher utilization, longer queues. The optimal β depends on the cost of servers versus the cost of delays.
|
||
|
||
## Evidence
|
||
|
||
Whitt's paper presents this as the core practical result from decades of queueing theory research. The Erlang C formula provides the computational foundation for determining exact server counts given arrival rates, service times, and target service levels.
|
||
|
||
The Halfin-Whitt regime (also called QED regime) is the theoretical framework: systems operate near full utilization (ρ → 1) while keeping delays manageable by approaching full utilization at rate Θ(1/√n) as server count n grows.
|
||
|
||
## Practical Application
|
||
|
||
For pipeline systems processing asynchronous work (like Teleo's source processing), this means:
|
||
- Don't provision for peak load (wasteful)
|
||
- Don't provision for average load (queue explosion)
|
||
- Provision for average + β√average where β ≈ 1-2
|
||
|
||
For a system with R=8 base workers (full utilization), optimal provisioning is 8 + 1.5√8 ≈ 12-13 workers.
|
||
|
||
---
|
||
|
||
Relevant Notes:
|
||
- domains/internet-finance/_map
|
||
|
||
Topics:
|
||
- core/mechanisms/_map |