teleo-codex/inbox/archive/2018-00-00-siam-economies-of-scale-halfin-whitt-regime.md

type	title	author	url	date	domain	format	status	tags	processed_by	processed_date	claims_extracted	extraction_model	extraction_notes
source	Economies-of-Scale in Many-Server Queueing Systems: Tutorial and Partial Review of the QED Halfin-Whitt Heavy-Traffic Regime	Johan van Leeuwaarden, Britt Mathijsen, Jaron Sanders (SIAM Review)	https://epubs.siam.org/doi/10.1137/17M1133944	2018-01-01	internet-finance	paper	processed	pipeline-architecture operations-research queueing-theory Halfin-Whitt economies-of-scale square-root-staffing	rio	2026-03-11	square-root-staffing-principle-achieves-economies-of-scale-in-queueing-systems-by-operating-near-full-utilization-with-manageable-delays.md moderate-scale-queueing-systems-benefit-from-simple-threshold-policies-over-sophisticated-algorithms-because-square-root-staffing-captures-most-efficiency-gains.md	anthropic/claude-sonnet-4.5	Extracted two claims about queueing theory and economies of scale. The source is a mathematical tutorial with proven results (SIAM Review), so confidence is 'proven' for the core mathematical claim and 'likely' for the practical application claim. No entities to extract (academic paper, no companies/products/decisions). The relevance to Teleo is in pipeline architecture optimization, which is noted in the source's 'Relevance to Teleo Pipeline' section.

Economies-of-Scale in Many-Server Queueing Systems

SIAM Review tutorial on the QED (Quality-and-Efficiency-Driven) Halfin-Whitt heavy-traffic regime — the mathematical foundation for understanding when and how multi-server systems achieve economies of scale.

Key Content

• The QED regime: operate near full utilization while keeping delays manageable
• As server count n grows, utilization approaches 1 at rate Θ(1/√n) — the "square root staffing" principle
• Economies of scale: larger systems need proportionally fewer excess servers for the same service quality
• The regime applies to systems ranging from tens to thousands of servers
• Square-root safety staffing works empirically even for moderate-sized systems (5-20 servers)
• Tutorial connects abstract queueing theory to practical staffing decisions

Relevance to Teleo Pipeline

At our scale (5-6 workers), we're in the "moderate system" range where square-root staffing still provides useful guidance. The key takeaway: we don't need sophisticated algorithms for a system this small. Simple threshold policies informed by queueing theory will capture most of the benefit. The economies-of-scale result also tells us that if we grow to 20+ workers, the marginal value of each additional worker decreases — important for cost optimization.

Key Facts

• Halfin-Whitt QED regime: utilization approaches 1 at rate Θ(1/√n)
• Square-root staffing validated empirically for systems as small as 5-20 servers
• 100-server system needs ~10 excess servers; 400-server system needs ~20 (not 40) for same quality