# Compute Arbitrage Engine

*/Opportunities/Compute_Arbitrage_Engine*

## Opportunity Overview

**Wedge**: Target batch-processing video and audio generation startups first. These asynchronous workloads tolerate variable execution times and cold starts, making them ideal for aggressive spot-instance arbitrage. Expand next into offline model fine-tuning pipelines, and finally capture synchronous low-latency API inference routing once provider reliability data matures.
**Timing**: The proliferation of specialized Tier-2 GPU clouds and standardized container APIs introduces massive price variations for identical hardware. Programmatic provisioning APIs now allow instant switching between hardware providers without manual infrastructure configuration.
**Why This I C P**: Mid-stage AI application builders burn capital on compute and lack the scale for multi-year reserved instance contracts with AWS or GCP. They require immediate cost reduction and tolerate multi-cloud infrastructure to maintain reliable uptime during GPU shortages.
**Size Of Prize**: Approximately 40,000 AI-native startups and enterprise ML teams spend an average of 50,000 dollars annually on compute orchestration tooling and spot-market broker premiums, resulting in a 2 billion dollar addressable prize.
**Gap Narrative**: AI development teams and ML researchers manually provision compute across fragmented GPU clouds. They lack an automated broker that continuously monitors spot pricing, availability, and latency across providers like RunPod, CoreWeave, and AWS to dynamically route containerized workloads.
**Defensibility**: The system builds a compounding data moat based on historical price and reliability telemetry. Every routed workload contributes to a proprietary dataset of provider uptime, hidden latency penalties, and spot market elasticity, generating routing algorithms that new entrants without volume cannot replicate.
**Why This Thesis**: A middleware software approach fits precisely because routing compute requires millisecond-level price comparisons and automated deployment execution. Software executes this multi-variable matching instantly, replacing static human DevOps provisioning.

## Opportunity Linked Thesis

**Thesis**: [Software](/Theses/Software)

## Opportunity Linked I C P

**Icp**: [Machine Learning Startup](/CompanyTypes/Machine_Learning_Startup)

## Opportunity Market Sizing

_Illustrative — target and order-of-magnitude estimate figures, not an achieved track record (this Thing is concept-stage)._

**S A M**: ~$1.5-2.5B VC-backed US and European ML startups requiring GPU clusters
**S O M**: ~$50-150M
**T A M**: ~20k global AI and ML software companies × ~$250k/yr addressable cloud compute spend ≈ $5B
**Growth Rate**: ~35-45%/yr, driven by escalating GPU cluster demands for model training and the fragmentation of specialized cloud providers
**Paid Comparable Spend**: ~$120k-180k/yr for a dedicated DevOps engineer managing spot instances manually, plus ~$10k-30k/yr for static cloud cost monitoring software

## Opportunity Incumbents

- [NetApp Spot](/Products/NetApp_Spot) — Tool
- [AWS Spot Fleet](/Products/AWS_Spot_Fleet) — Tool
- [Akash Network](/Products/Akash_Network) — Open-Source
- [Golem Network](/Products/Golem_Network) — Open-Source
- [Custom Terraform Scripts](/Products/Custom_Terraform_Scripts) — DIY
- [Manual Python Allocators](/Products/Manual_Python_Allocators) — DIY

## Opportunity Win Conditions

**Kill Thresholds**:
- Egress costs consume >40% of gross compute arbitrage savings
- Job resumption failure rate >5% after preemption
- D30 active user retention <40%
- Average integration and setup time >14 days
**Leading Metrics**:
- Egress-to-savings ratio
- Percentage of training jobs successfully resumed after preemption
- Time-to-first-arbitrage deployment
- Average net savings per GPU-hour dynamically allocated
**What Proves Right**: ML engineering teams route at least 30% of their training workloads through the engine within the first 60 days. The system automatically migrates active training jobs across disparate GPU providers without losing checkpoint progress. Customers adopt a shared-savings pricing model that yields $2k in monthly revenue while cutting their gross compute spend by 25%.
**What Proves Wrong**: Egress costs and data transfer latency between cloud providers outpace the spot instance compute savings. ML engineering teams abandon the routing layer because it disrupts their specialized container environments or complicates debugging. Startups default to purchasing reserved instances directly from primary vendors rather than tolerating multi-cloud deployment complexity.

## Opportunity Build Profile

**Hardest Part**: Migrating active, large-state memory workloads across heterogeneous GPU clusters just prior to spot termination without dropping requests or corrupting checkpoints.
**Min Viable Scope**: Confine v1 to stateless batch inference tasks on a single GPU class spanning exactly two major cloud spot markets. Explicitly exclude synchronous real-time inference, stateful ML training workloads, and long-term reserved instance brokering.
**Cold Start Problem**: The engine requires historical pricing and interruption data to accurately predict spot termination probabilities and arbitrage spreads. Break this by running synthetic benchmark workloads across target clouds to map termination vectors before onboarding the first live customer.
**Time To First Value**: 1-2 days to integrate containerized workloads and execute the first batch job
**Data Moat Available**: true
**Technical Difficulty**: High

## Neighborhood

### Where the gap lives

- [Information](/Industries/Information) — latent gap · Industries

### Incumbent in

- [Cloud FinOps Consultancies](/Products/Cloud_FinOps_Consultancies) — incumbent in · Products
- [Cast AI](/Products/Cast_AI) — incumbent in · Products
- [AWS Spot Fleet](/Products/AWS_Spot_Fleet) — incumbent in · Products
- [Akash Network](/Products/Akash_Network) — incumbent in · Products
- [Custom Terraform Scripts](/Products/Custom_Terraform_Scripts) — incumbent in · Products
- [Golem Network](/Products/Golem_Network) — incumbent in · Products
- [Manual Python Allocators](/Products/Manual_Python_Allocators) — incumbent in · Products
- [NetApp Spot](/Products/NetApp_Spot) — incumbent in · Products
- [Spot By NetApp](/Products/Spot_By_NetApp) — incumbent in · Products
- [Boutique Cloud Brokerages](/Products/Boutique_Cloud_Brokerages) — incumbent in · Products
- [Manual FinOps Spreadsheets](/Products/Manual_FinOps_Spreadsheets) — incumbent in · Products

### Applies thesis

- [Machine Learning Startup](/CompanyTypes/Machine_Learning_Startup) — applies thesis · CompanyTypes
- [Computing Infrastructure Provider](/CompanyTypes/Computing_Infrastructure_Provider) — applies thesis · CompanyTypes

### Embodies

- [Software](/Theses/Software) — embodies · Theses

### Similar Opportunities

- [Compute Arbitrage Engine](/Industries/Information/Opportunities/Compute_Arbitrage_Engine) — similar · Opportunities
- [Render Compute Orchestration](/CompanyTypes/Boutique_VFX_Studio/Opportunities/Render_Compute_Orchestration) — similar · Opportunities
- [Simulation Workload Router](/Opportunities/Simulation_Workload_Router) — similar · Opportunities
- [Capacity Routing API](/Opportunities/Capacity_Routing_API) — similar · Opportunities
- [Dynamic LLM Routing for AI Startups](/Opportunities/Dynamic_LLM_Routing_for_AI_Startups) — similar · Opportunities
- [Dynamic Workload Allocation for Cloud Providers](/Opportunities/Dynamic_Workload_Allocation_for_Cloud_Providers) — similar · Opportunities
- [Distributed Load Router](/Opportunities/Distributed_Load_Router) — similar · Opportunities
- [Resource Arbitration API](/Opportunities/Resource_Arbitration_API) — similar · Opportunities
- [CI Compute Router](/Opportunities/CI_Compute_Router) — similar · Opportunities
- [Simulation Cost Orchestrator](/Opportunities/Simulation_Cost_Orchestrator) — similar · Opportunities
- [Browser Compute Router](/Opportunities/Browser_Compute_Router) — similar · Opportunities
- [Rack Price Procurement](/Opportunities/Rack_Price_Procurement) — similar · Opportunities
- [Outage Mitigation Gateway](/Opportunities/Outage_Mitigation_Gateway) — similar · Opportunities
- [AI for AI for Software Publishers as a Service API](/Opportunities/AI_for_AI_for_Software_Publishers_as_a_Service_API) — similar · Opportunities
- [Predictive Load Balancer](/Opportunities/Predictive_Load_Balancer) — similar · Opportunities
- [Headless Genomic Pipeline](/Opportunities/Headless_Genomic_Pipeline) — similar · Opportunities
- [Vendor Rerouting Engine](/Departments/Example_One/Opportunities/Vendor_Rerouting_Engine) — similar · Opportunities
- [Provider Abstraction Gateway](/Opportunities/Provider_Abstraction_Gateway) — similar · Opportunities
- [Cloud Provisioning Optimizer](/Occupations/Computer_and_Mathematical_Occupations/Opportunities/Cloud_Provisioning_Optimizer) — similar · Opportunities
- [Staging Routing Engine](/Opportunities/Staging_Routing_Engine) — similar · Opportunities