34 lines
No EOL
2.6 KiB
Markdown
34 lines
No EOL
2.6 KiB
Markdown
---
|
|
type: claim
|
|
domain: internet-finance
|
|
description: "Structured MDP algorithms incorporating hysteresis properties achieve better execution time and accuracy than simple threshold heuristics in cloud VM provisioning"
|
|
confidence: likely
|
|
source: "Tournaire, Castel-Taleb, Hyon (2021), 'Optimal Control Policies for Resource Allocation in the Cloud'"
|
|
created: 2026-03-11
|
|
---
|
|
|
|
# MDP-based autoscaling with hysteresis outperforms simple threshold heuristics for cloud resource allocation
|
|
|
|
Markov Decision Process formulations that incorporate hysteresis properties (different thresholds for scaling up versus scaling down) outperform simple threshold heuristics in both execution time and accuracy for cloud auto-scaling problems. The MDP approach treats VM provisioning as a sequential decision problem where states represent queue lengths plus active VMs, actions are add/remove VMs, and rewards are negative cost (energy plus SLA violations).
|
|
|
|
The key insight is that hysteresis—using different thresholds for scaling up (e.g., queue=10) versus scaling down (e.g., queue=3)—prevents oscillation in resource allocation. MDP algorithms using value iteration and policy iteration can discover optimal hysteresis thresholds automatically, whereas simple heuristics require manual threshold tuning.
|
|
|
|
This finding is particularly relevant for pipeline architectures where worker pools must respond to variable queue depths. The structured MDP approach provides a principled framework for determining when to spawn versus kill workers based on queue occupation, minimizing the combined cost of compute resources and processing delays.
|
|
|
|
## Evidence
|
|
|
|
- Tournaire et al. (2021) demonstrate that structured MDP algorithms incorporating hysteresis properties outperform heuristics in both execution time and accuracy for cloud auto-scaling
|
|
- The paper shows MDP algorithms find optimal hysteresis thresholds automatically through value iteration and policy iteration
|
|
- Simple threshold policies (scale up at queue=N, scale down at queue=M where M < N) perform reasonably well but are suboptimal compared to MDP-derived policies
|
|
|
|
## Relevance to Teleo
|
|
|
|
This directly applies to the Teleo pipeline architecture where states = (unprocessed queue, in-flight extractions, open PRs, active workers), actions = (spawn worker, kill worker, wait), and cost = (Claude compute cost per worker-minute + delay cost per queued source). The hysteresis insight suggests we should have different thresholds for spinning up versus spinning down workers to prevent oscillation.
|
|
|
|
---
|
|
|
|
Relevant Notes:
|
|
- domains/internet-finance/_map
|
|
|
|
Topics:
|
|
- domains/internet-finance/_map |