42 lines
3.4 KiB
Markdown
42 lines
3.4 KiB
Markdown
---
|
|
type: source
|
|
title: "Optimal Control Policies for Resource Allocation in the Cloud: Comparison Between Markov Decision Process and Heuristic Approaches"
|
|
author: "Thomas Tournaire, Hind Castel-Taleb, Emmanuel Hyon"
|
|
url: https://arxiv.org/abs/2104.14879
|
|
date: 2021-04-30
|
|
domain: internet-finance
|
|
format: paper
|
|
status: processed
|
|
tags: [pipeline-architecture, operations-research, markov-decision-process, cloud-autoscaling, optimal-control]
|
|
processed_by: rio
|
|
processed_date: 2026-03-11
|
|
claims_extracted: ["mdp-based-autoscaling-with-hysteresis-outperforms-simple-threshold-heuristics-for-cloud-resource-allocation.md", "sequential-decision-mdp-formulation-maps-cloud-autoscaling-to-optimal-control-theory.md"]
|
|
extraction_model: "anthropic/claude-sonnet-4.5"
|
|
extraction_notes: "Academic paper on optimal control for cloud resource allocation. Extracted two claims: (1) MDP with hysteresis outperforms simple heuristics, (2) MDP formulation maps autoscaling to optimal control theory. Both claims are methodological insights about operations research techniques applicable to pipeline architectures. No entities to create—this is a research paper, not a company/product/market. The Teleo pipeline relevance is noted in claim bodies but this is not about internet finance per se—it's about operations research applicable to infrastructure. However, given the domain tag in the source and the pipeline architecture context, classified as internet-finance for now."
|
|
---
|
|
|
|
# Optimal Control Policies for Resource Allocation in the Cloud
|
|
|
|
Compares MDP-based optimal scaling policies against heuristic approaches for cloud auto-scaling. The MDP formulation treats VM provisioning as a sequential decision problem.
|
|
|
|
## Key Content
|
|
|
|
- Auto-scaling problem: VMs turned on/off based on queue occupation to minimize combined energy + performance cost
|
|
- MDP formulation: states = queue lengths + active VMs, actions = add/remove VMs, rewards = negative cost (energy + SLA violations)
|
|
- Value iteration and policy iteration algorithms find optimal threshold policies
|
|
- Structured MDP algorithms incorporating hysteresis properties outperform heuristics in both execution time and accuracy
|
|
- Hysteresis: different thresholds for scaling up vs. scaling down — prevents oscillation (e.g., scale up at queue=10, scale down at queue=3)
|
|
- MDP algorithms find optimal hysteresis thresholds automatically
|
|
|
|
## Relevance to Teleo Pipeline
|
|
|
|
The MDP formulation maps directly: states = (unprocessed queue, in-flight extractions, open PRs, active workers), actions = (spawn worker, kill worker, wait), cost = (Claude compute cost per worker-minute + delay cost per queued source). The hysteresis insight is particularly valuable — we should have different thresholds for spinning up vs. spinning down workers to prevent oscillation.
|
|
|
|
Key finding: structured MDP with hysteresis outperforms simple threshold heuristics. But even simple threshold policies (scale up at queue=N, scale down at queue=M where M < N) perform reasonably well.
|
|
|
|
|
|
## Key Facts
|
|
- Tournaire, Castel-Taleb, Hyon published 'Optimal Control Policies for Resource Allocation in the Cloud' on arXiv (2021-04-30)
|
|
- MDP formulation: states = queue lengths + active VMs, actions = add/remove VMs, rewards = negative cost (energy + SLA violations)
|
|
- Hysteresis example: scale up at queue=10, scale down at queue=3 to prevent oscillation
|
|
- Value iteration and policy iteration are the algorithms used to find optimal policies
|