m3taversal
be8ff41bfe
Wrote sourced_from: into 414 claim files pointing back to their origin source. Backfilled claims_extracted: into 252 source files that were processed but missing this field. Matching uses author+title overlap against claim source: field, validated against 296 known-good pairs from existing claims_extracted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
38 lines
No EOL
2.2 KiB
Markdown
38 lines
No EOL
2.2 KiB
Markdown
---
|
|
type: claim
|
|
domain: internet-finance
|
|
description: "MDP research shows threshold policies are provably optimal for most queueing systems"
|
|
confidence: proven
|
|
source: "Li et al., 'An Overview for Markov Decision Processes in Queues and Networks' (2019)"
|
|
created: 2026-03-11
|
|
sourced_from:
|
|
- inbox/archive/internet-finance/2019-07-00-li-overview-mdp-queues-networks.md
|
|
---
|
|
|
|
# Optimal queue policies have threshold structure making simple rules near-optimal
|
|
|
|
Six decades of operations research on Markov Decision Processes applied to queueing systems consistently shows that optimal policies have threshold structure: "serve if queue > K, idle if queue < K" or "spawn worker if queue > X and workers < Y." This means even without solving the full MDP, well-tuned threshold policies achieve near-optimal performance.
|
|
|
|
For multi-server systems, optimal admission and routing policies follow similar patterns: join-shortest-queue, threshold-based admission control. The structural simplicity emerges from the mathematical properties of the value function in continuous-time MDPs where decisions happen at state transitions (arrivals, departures).
|
|
|
|
This has direct implications for pipeline architecture: systems with manageable state spaces (queue depths across stages, worker counts, time-of-day) can use exact MDP solution via value iteration, but even approximate threshold policies will perform near-optimally due to the underlying structure.
|
|
|
|
## Evidence
|
|
|
|
Li et al. survey 60+ years of MDP research in queueing theory (1960s to 2019), covering:
|
|
- Continuous-time MDPs for queue management with decisions at state transitions
|
|
- Classic results showing threshold structure in optimal policies
|
|
- Multi-server systems where optimal policies are simple (join-shortest-queue, threshold-based)
|
|
- Dynamic programming and stochastic optimization methods for deriving optimal policies
|
|
|
|
The key challenge identified is curse of dimensionality: state space explodes with multiple queues/stages. Practical approaches include approximate dynamic programming and reinforcement learning for large state spaces.
|
|
|
|
Emerging direction: deep RL for queue management in networks and cloud computing.
|
|
|
|
---
|
|
|
|
Relevant Notes:
|
|
- domains/internet-finance/_map
|
|
|
|
Topics:
|
|
- domains/internet-finance/_map |