teleo-codex/domains/internet-finance/mdp-based-autoscaling-with-hysteresis-outperforms-simple-threshold-heuristics-for-cloud-resource-allocation.md

type	domain	description	confidence	source	created
claim	internet-finance	Structured MDP algorithms incorporating hysteresis properties achieve better execution time and accuracy than simple threshold heuristics in cloud VM provisioning	likely	Tournaire, Castel-Taleb, Hyon (2021), 'Optimal Control Policies for Resource Allocation in the Cloud'	2026-03-11

MDP-based autoscaling with hysteresis outperforms simple threshold heuristics for cloud resource allocation

Markov Decision Process formulations that incorporate hysteresis properties (different thresholds for scaling up versus scaling down) outperform simple threshold heuristics in both execution time and accuracy for cloud auto-scaling problems. The MDP approach treats VM provisioning as a sequential decision problem where states represent queue lengths plus active VMs, actions are add/remove VMs, and rewards are negative cost (energy plus SLA violations).

The key insight is that hysteresis—using different thresholds for scaling up (e.g., queue=10) versus scaling down (e.g., queue=3)—prevents oscillation in resource allocation. MDP algorithms using value iteration and policy iteration can discover optimal hysteresis thresholds automatically, whereas simple heuristics require manual threshold tuning.

This finding is particularly relevant for pipeline architectures where worker pools must respond to variable queue depths. The structured MDP approach provides a principled framework for determining when to spawn versus kill workers based on queue occupation, minimizing the combined cost of compute resources and processing delays.

Evidence

• Tournaire et al. (2021) demonstrate that structured MDP algorithms incorporating hysteresis properties outperform heuristics in both execution time and accuracy for cloud auto-scaling
• The paper shows MDP algorithms find optimal hysteresis thresholds automatically through value iteration and policy iteration
• Simple threshold policies (scale up at queue=N, scale down at queue=M where M < N) perform reasonably well but are suboptimal compared to MDP-derived policies

Relevance to Teleo

This directly applies to the Teleo pipeline architecture where states = (unprocessed queue, in-flight extractions, open PRs, active workers), actions = (spawn worker, kill worker, wait), and cost = (Claude compute cost per worker-minute + delay cost per queued source). The hysteresis insight suggests we should have different thresholds for spinning up versus spinning down workers to prevent oscillation.

---

Relevant Notes:

• domains/internet-finance/_map

Topics:

• domains/internet-finance/_map