teleo-codex/domains/internet-finance/mdp-based-autoscaling-with-hysteresis-outperforms-simple-threshold-heuristics-for-cloud-resource-allocation.md

type	domain	description	confidence	source	created
claim	internet-finance	Structured MDP algorithms that incorporate hysteresis properties achieve better performance and faster execution than simple threshold heuristics in cloud VM provisioning	likely	Tournaire et al., 'Optimal Control Policies for Resource Allocation in the Cloud' (2021)	2026-03-11

MDP-based autoscaling with hysteresis outperforms simple threshold heuristics for cloud resource allocation

Markov Decision Process formulations that incorporate hysteresis properties (different thresholds for scaling up versus scaling down) outperform simple threshold heuristics in both execution time and accuracy for cloud auto-scaling problems. The MDP approach automatically discovers optimal hysteresis thresholds rather than requiring manual tuning.

The problem formulation treats VM provisioning as a sequential decision problem where:

• States = queue lengths + active VMs
• Actions = add/remove VMs
• Rewards = negative cost (energy + SLA violations)

Value iteration and policy iteration algorithms find optimal threshold policies that prevent oscillation by using different thresholds for scaling up (e.g., queue=10) versus scaling down (e.g., queue=3).

Tournaire et al. (2021) demonstrate that structured MDP algorithms incorporating hysteresis properties outperform heuristic approaches in both execution time and accuracy. The key insight is that hysteresis—different thresholds for scaling up versus scaling down—prevents oscillation, and MDP algorithms can discover these optimal thresholds automatically rather than through manual tuning.

Relevance to Pipeline Architecture

This formulation maps directly to autonomous pipeline management:

• States = (unprocessed queue, in-flight extractions, open PRs, active workers)
• Actions = (spawn worker, kill worker, wait)
• Cost = (Claude compute cost per worker-minute + delay cost per queued source)

The hysteresis insight is particularly valuable for preventing worker thrashing in variable-load scenarios. Simple threshold policies (scale up at queue=N, scale down at queue=M where M < N) provide reasonable baseline performance, but MDP optimization can find better thresholds given cost structure and workload patterns.

---

Relevant Notes:

• domains/internet-finance/_map

Topics:

• domains/internet-finance/_map