43 lines
No EOL
2.4 KiB
Markdown
43 lines
No EOL
2.4 KiB
Markdown
---
|
|
type: claim
|
|
domain: internet-finance
|
|
description: "Auto-scaling problems can be formulated as MDPs where queue states and VM counts map to optimal control policies that minimize energy plus performance costs"
|
|
confidence: likely
|
|
source: "Tournaire, Castel-Taleb, Hyon (2021), 'Optimal Control Policies for Resource Allocation in the Cloud'"
|
|
created: 2026-03-11
|
|
---
|
|
|
|
# Sequential decision MDP formulation maps cloud autoscaling to optimal control theory
|
|
|
|
Cloud auto-scaling problems can be formally modeled as Markov Decision Processes where the state space consists of queue lengths and active VM counts, actions are VM provisioning decisions (add/remove/wait), and the reward function is the negative of combined energy and SLA violation costs. This formulation enables the application of optimal control theory through value iteration and policy iteration algorithms to find provably optimal scaling policies.
|
|
|
|
The MDP framework treats resource allocation as a sequential decision problem rather than a reactive heuristic, allowing the system to optimize over future expected costs rather than just current state. The key advantage is that MDP solutions automatically balance the tradeoff between energy costs (from running VMs) and performance costs (from queue delays and SLA violations).
|
|
|
|
This approach generalizes beyond cloud infrastructure to any resource allocation problem with:
|
|
1. Observable queue/workload state
|
|
2. Discrete provisioning actions with known costs
|
|
3. Stochastic arrival/service processes
|
|
4. Multi-period cost minimization objective
|
|
|
|
## Evidence
|
|
|
|
- Tournaire et al. (2021) formulate auto-scaling as MDP with states = (queue lengths, active VMs), actions = (add/remove VMs), rewards = negative cost (energy + SLA violations)
|
|
- Value iteration and policy iteration algorithms find optimal threshold policies that minimize combined energy and performance costs
|
|
- The MDP formulation enables principled optimization over future expected costs rather than reactive heuristics
|
|
|
|
## Applications
|
|
|
|
This framework applies to:
|
|
- Cloud VM provisioning (the paper's focus)
|
|
- Pipeline worker pools responding to variable queue depths
|
|
- Database connection pool sizing
|
|
- API rate limiting and capacity planning
|
|
- Any system with observable workload state and discrete resource provisioning decisions
|
|
|
|
---
|
|
|
|
Relevant Notes:
|
|
- domains/internet-finance/_map
|
|
|
|
Topics:
|
|
- domains/internet-finance/_map |