---
type: source
title: "What You Should Know About Queueing Models"
author: "Ward Whitt (Columbia University)"
url: https://www.columbia.edu/~ww2040/shorter041907.pdf
date: 2019-04-19
domain: internet-finance
format: paper
status: processed
tags: [pipeline-architecture, operations-research, queueing-theory, square-root-staffing, Halfin-Whitt]
processed_by: rio
processed_date: 2026-03-11
claims_extracted: ["square-root-staffing-rule-provisions-servers-as-base-load-plus-beta-times-square-root-of-base-load-where-beta-controls-service-quality.md", "halfin-whitt-qed-regime-enables-high-utilization-with-bounded-delays-by-approaching-full-load-at-rate-proportional-to-inverse-square-root-of-server-count.md", "erlang-c-formula-is-the-computational-workhorse-for-multi-server-queue-staffing-calculations-given-arrival-rates-and-service-level-targets.md"]
extraction_model: "anthropic/claude-sonnet-4.5"
extraction_notes: "Extracted three foundational queueing theory claims with direct applicability to pipeline architecture and worker provisioning. Source is authoritative (Ward Whitt is a field founder) and presents proven mathematical results. All claims are at 'proven' confidence level as they represent established queueing theory with decades of validation. No entities to extract (academic paper, no companies/products/decisions). No enrichments identified as these are novel claims to the KB focused on operations research rather than existing internet-finance mechanisms."
---

# What You Should Know About Queueing Models

Practitioner-oriented guide by Ward Whitt (Columbia), one of the founders of modern queueing theory for service systems. Covers the essential queueing models practitioners need and introduces the Halfin-Whitt heavy-traffic regime.

## Key Content

- Square-root staffing principle: optimal server count = base load + β√(base load), where β is a quality-of-service parameter
- The Halfin-Whitt (QED) regime: systems operate near full utilization while keeping delays manageable — utilization approaches 1 at rate Θ(1/√n) as servers n grow
- Economies of scale in multi-server systems: larger systems need proportionally fewer excess servers
- Practical formulas for determining server counts given arrival rates and service level targets
- Erlang C formula as the workhorse for staffing calculations

## Relevance to Teleo Pipeline

The square-root staffing rule is directly applicable: if our base load requires R workers at full utilization, we should provision R + β√R workers where β ≈ 1-2 depending on target service level. For our scale (~8 sources/cycle, ~5 min service time), this gives concrete worker count guidance.

Critical insight: you don't need to match peak load with workers. The square-root safety margin handles variance efficiently. Over-provisioning for peak is wasteful; under-provisioning for average causes queue explosion. The sweet spot is the QED regime.


## Key Facts
- Ward Whitt is a founder of modern queueing theory for service systems at Columbia University
- Paper published 2019-04-19 as practitioner-oriented guide
- QED regime stands for Quality-and-Efficiency-Driven
- Halfin-Whitt regime is also called QED regime
- Typical β values for square-root staffing are 1-2 depending on service quality targets