extract: 2021-09-00-vlahakis-aimd-scheduling-distributed-computing

Pentagon-Agent: Ganymede <F99EBFA6-547B-4096-BEEA-1D59C3E4028A>

domains/internet-finance/aimd-congestion-control-generalizes-to-distributed-resource-allocation-because-queue-dynamics-are-structurally-identical-across-networks-and-compute-pipelines.md

@@ -0,0 +1,32 @@
+---
+type: claim
+domain: internet-finance
+description: "TCP's AIMD algorithm applies to worker scaling in distributed systems because both solve the producer-consumer rate matching problem"
+confidence: likely
+source: "Vlahakis, Athanasopoulos et al., AIMD Scheduling and Resource Allocation in Distributed Computing Systems (2021)"
+created: 2026-03-11
+---
+
+# AIMD congestion control generalizes to distributed resource allocation because queue dynamics are structurally identical across networks and compute pipelines
+
+The core insight from Vlahakis et al. (2021) is that TCP's AIMD (Additive Increase Multiplicative Decrease) congestion control algorithm, proven optimal for fair network bandwidth allocation, applies directly to distributed computing resource allocation. The paper demonstrates that scheduling incoming requests across computing nodes is mathematically equivalent to network congestion control — both are producer-consumer rate matching problems where queue state reveals system health.
+
+The AIMD policy is elegant: when queues shrink (system healthy), add workers linearly (+1 per cycle). When queues grow (system overloaded), cut workers multiplicatively (e.g., halve them). This creates self-correcting dynamics that are proven stable regardless of total node count and AIMD parameters.
+
+Key theoretical results:
+- Decentralized resource allocation using nonlinear state feedback achieves global convergence to bounded set in finite time
+- The system is stable irrespective of total node count and AIMD parameters
+- Quality of Service is calculable via Little's Law from simple local queuing time formulas
+- AIMD is proven optimal for fair allocation of shared resources among competing agents without centralized control
+
+The practical implication: distributed systems don't need to predict load or use complex ML models for autoscaling. They can react to observed queue state using a simple, proven-stable policy. When extract produces faster than eval can consume, AIMD naturally provides backpressure (slow extraction) or scale-up (more eval workers) without requiring load forecasting.
+
+This connects directly to pipeline architecture design: the "bandwidth" of a processing pipeline is its throughput capacity, and AIMD provides the control law for matching producer rate to consumer capacity.
+
+---
+
+Relevant Notes:
+- core/mechanisms/_map
+
+Topics:
+- domains/internet-finance/_map

domains/internet-finance/aimd-worker-scaling-requires-only-queue-state-observation-not-load-prediction-making-it-simpler-than-ml-based-autoscaling.md

@@ -0,0 +1,40 @@
+---
+type: claim
+domain: internet-finance
+description: "AIMD autoscaling reacts to observed queue dynamics rather than forecasting demand, eliminating prediction error and model complexity"
+confidence: experimental
+source: "Vlahakis, Athanasopoulos et al., AIMD Scheduling (2021), applied to Teleo pipeline context"
+created: 2026-03-11
+---
+
+# AIMD worker scaling requires only queue state observation not load prediction making it simpler than ML-based autoscaling
+
+Traditional autoscaling approaches attempt to predict future load and preemptively adjust capacity. This requires:
+- Historical load data and pattern recognition
+- ML models to forecast demand
+- Tuning of prediction windows and confidence thresholds
+- Handling of prediction errors and their cascading effects
+
+AIMD eliminates this entire complexity layer by operating purely on observed queue state. The control law is:
+- If queue_length is decreasing: add workers linearly (additive increase)
+- If queue_length is increasing: remove workers multiplicatively (multiplicative decrease)
+
+This reactive approach has several advantages:
+1. **No prediction error** — the system responds to actual observed state, not forecasts
+2. **No training data required** — works immediately without historical patterns
+3. **Self-correcting** — wrong adjustments are automatically reversed by subsequent queue observations
+4. **Proven stable** — mathematical guarantees from control theory, not empirical tuning
+
+The Vlahakis et al. (2021) paper proves that this decentralized approach achieves global convergence to bounded queue lengths in finite time, regardless of system size or AIMD parameters. The stability is structural, not empirical.
+
+For the Teleo pipeline specifically: when extract produces claims faster than eval can process them, the eval queue grows. AIMD detects this and scales up eval workers. When the queue shrinks below target, AIMD scales down. No load forecasting, no ML models, no hyperparameter tuning — just queue observation and a simple control law.
+
+The tradeoff: AIMD is reactive rather than predictive, so it responds to load changes rather than anticipating them. For bursty workloads with predictable patterns, ML-based prediction might provision capacity faster. But for unpredictable workloads or systems where prediction accuracy is low, AIMD's simplicity and guaranteed stability are compelling.
+
+---
+
+Relevant Notes:
+- core/mechanisms/_map
+
+Topics:
+- domains/internet-finance/_map

inbox/archive/2021-09-00-vlahakis-aimd-scheduling-distributed-computing.md

@@ -6,8 +6,13 @@ url: https://arxiv.org/abs/2109.02589
 date: 2021-09-06
 domain: internet-finance
 format: paper
-status: unprocessed
+status: processed
 tags: [pipeline-architecture, operations-research, AIMD, distributed-computing, resource-allocation, congestion-control]
+processed_by: rio
+processed_date: 2026-03-11
+claims_extracted: ["aimd-congestion-control-generalizes-to-distributed-resource-allocation-because-queue-dynamics-are-structurally-identical-across-networks-and-compute-pipelines.md", "aimd-worker-scaling-requires-only-queue-state-observation-not-load-prediction-making-it-simpler-than-ml-based-autoscaling.md"]
+extraction_model: "anthropic/claude-sonnet-4.5"
+extraction_notes: "Extracted two mechanism design claims about AIMD's generalization from network congestion control to distributed computing autoscaling. The source is a 2021 academic paper proving mathematical properties of AIMD in multi-queue distributed systems. Primary relevance is to pipeline architecture and operations research, with direct application to Teleo's extract-eval pipeline scaling problem. No entities to create (academic paper, no companies/products/decisions). No enrichments identified — these are novel mechanism insights not covered by existing claims in the KB."
 ---
 
 # AIMD Scheduling and Resource Allocation in Distributed Computing Systems
@@ -28,3 +33,10 @@ Applies TCP's AIMD (Additive Increase Multiplicative Decrease) congestion contro
 AIMD provides an elegant scaling policy: when queue is shrinking (system healthy), add workers linearly (e.g., +1 per cycle). When queue is growing (system overloaded), cut workers multiplicatively (e.g., halve them). This is self-correcting, proven stable, and doesn't require predicting load — it reacts to observed queue state.
 
 The TCP analogy is precise: our pipeline "bandwidth" is eval throughput. When extract produces faster than eval can consume, we need backpressure (slow extraction) or scale-up (more eval workers). AIMD handles this naturally.
+
+
+## Key Facts
+- AIMD (Additive Increase Multiplicative Decrease) is TCP's congestion control algorithm
+- Vlahakis et al. (2021) proved AIMD stability for distributed computing resource allocation
+- AIMD achieves global convergence to bounded set in finite time regardless of node count
+- Little's Law connects queue length to QoS metrics in AIMD systems