teleo-codex/domains/internet-finance/optimal-queue-control-policies-have-threshold-structure-for-single-server-systems.md

type	domain	description	confidence	source	created
claim	internet-finance	Classic MDP results show admission and service policies converge on simple threshold rules rather than complex state-dependent strategies	proven	Li et al., 'An Overview for Markov Decision Processes in Queues and Networks' (2019), surveying 60+ years of queueing MDP research	2026-03-11

Optimal queue control policies have threshold structure for single-server systems

For single-server queueing systems modeled as continuous-time MDPs, optimal admission and service policies typically take the form of simple threshold rules: "serve if queue > K, idle if queue < K" or "admit if queue < N, reject if queue ≥ N." This threshold structure emerges from dynamic programming solutions across a wide range of cost functions and arrival/service distributions.

The threshold property means that despite the theoretical complexity of MDP state spaces, optimal policies can be characterized by a small number of parameters. This has two practical implications:

1. Policy search simplification: Instead of solving the full MDP via value iteration across all states, practitioners can search over threshold parameters, reducing computational complexity from exponential to linear in state space size.

2. Near-optimality of simple heuristics: Well-tuned threshold policies achieve near-optimal performance even when derived heuristically rather than through formal optimization, because the structure of the optimal solution is known in advance.

For multi-server systems, the threshold structure extends to policies like "join-shortest-queue" with queue-length-dependent routing thresholds. The survey covers applications from the 1960s (admission control in telephone networks) through modern cloud computing resource allocation.

Evidence

Li et al. (2019) provide a comprehensive 42-page survey of MDP applications in queueing theory, synthesizing results from 60+ years of operations research. The paper documents that across diverse queueing models—M/M/1, M/G/1, networks of queues—optimal policies derived through dynamic programming consistently exhibit threshold structure when the objective is cost minimization or throughput maximization.

Key theoretical results cited:

• Monotone policies: When the cost function is convex in queue length, optimal policies are monotone increasing (serve more aggressively as queue grows)
• Structural results: For admission control with holding costs, the optimal policy is a threshold on queue length
• Multi-server extensions: Join-shortest-queue is optimal for homogeneous servers; threshold-based routing is optimal for heterogeneous servers

The survey notes that these structural results break down in two cases:

1. High-dimensional state spaces: Networks with many queues suffer curse of dimensionality, requiring approximate methods
2. Non-convex costs: When costs have discontinuities or non-monotone structure, threshold policies may be suboptimal

Relevance to Teleo Pipeline

The Teleo pipeline architecture has a manageable state space: queue depths across 3 stages (inbox, analysis, synthesis), worker counts per stage, and time-of-day context. This falls well within the regime where exact MDP solutions are tractable via value iteration.

The threshold structure result means that even without solving the full MDP, a heuristic policy of the form "if inbox_queue > X and active_workers < Y, spawn worker" will be near-optimal if X and Y are well-tuned. The survey confirms this approach is theoretically grounded for systems of this scale.

For future work: if the pipeline scales to 10+ stages or incorporates complex dependencies between stages, approximate dynamic programming or reinforcement learning methods (covered in the survey's final section) become necessary.

---

Relevant Notes:

• core/mechanisms/_map

Topics:

• domains/internet-finance/_map