Know which workload you have
GPU requirements split cleanly. Training needs sustained hours, a lot of memory and often multiple cards. Inference needs availability and low latency but usually far less capacity. Occasional experimentation needs neither, and is badly served by paying for either.
Match the purchase to the pattern. Renting a large training instance for intermittent inference is the most common way to waste money here.
The hyperscalers: capability and queues
AWS, Google Cloud and Azure have the widest range of accelerator types and the deepest surrounding tooling for managed training and deployment.
They are also the most expensive per hour, and capacity for the most sought-after accelerators is genuinely constrained. Quota approval is a real gate on a new account, and in some regions capacity has to be reserved rather than requested on demand.
Mid-size providers compete hard on price
- DigitalOcean offers GPU Droplets with straightforward hourly pricing and the same simple console as the rest of the platform
- Linode has GPU plans with flat pricing
- OVHcloud offers GPU instances and dedicated GPU servers, often at noticeably lower rates than the hyperscalers
- Hetzner sells dedicated GPU hardware, which is the cheapest route if you can keep it busy
What you give up is the managed training platform. If you already have your own tooling, that is not much of a loss.
Dedicated hardware versus hourly rental
A dedicated GPU server billed monthly is usually far cheaper per GPU-hour than hourly cloud rental, provided utilisation is high. The break-even point arrives sooner than people expect.
Work out roughly how many hours a month the card will actually be busy. Below a few hours a day, hourly rental wins. Above that, dedicated hardware is typically cheaper, and the gap grows.
Do not overlook the surrounding costs
- The instance bills for every hour it exists, whether or not the GPU is computing
- Datasets moved into the region generate transfer cost if storage is elsewhere
- Checkpoints and model artefacts accumulate storage charges
- Idle development instances left running overnight are the classic waste
Stopping instances when not in use is the single highest-value habit here.
Quota is the first obstacle on any new account
GPU quota starts at zero on essentially every provider. Requesting an increase requires explaining the workload and waiting, and for the most in-demand accelerators the request may not be granted immediately.
Factor that lead time into the plan rather than discovering it on the day you intend to start.
Related accounts
Accounts on the platforms discussed here: Buy AWS Accounts, Buy DigitalOcean Accounts, Buy OVHcloud Accounts, Buy Linode Accounts.
Related cloud accounts
Accounts on the platforms discussed here, with every configuration and price published up front:
- Buy AWS Accounts — from $25.00. Over 200 services, from EC2 to Bedrock.
- Buy DigitalOcean Accounts — from $25.00. Droplets, managed Kubernetes and Spaces.
- Buy OVHcloud Accounts — from $30.00. European dedicated servers and GPU instances.
- Buy Linode Accounts — from $25.00. Akamai-backed compute and SMTP-capable accounts.
Further reading
- Which Cloud to Use for AI and Machine Learning Work
- Linode vs Kamatera: Which Cloud Account Should You Buy?
- AWS Accounts: What You Get and How to Choose a Configuration
- DigitalOcean vs Kamatera: Which Cloud Account Should You Buy?
Official documentation
Provider behaviour, quotas and pricing are defined by the platform itself. These are the first-party references for the services discussed above:
- AWS documentation
- DigitalOcean documentation
- OVHcloud documentation
- Akamai Cloud Computing documentation
Frequently asked questions
GPU quota is zero by default on almost every provider. It must be requested, and approval is not instant.
Above roughly a few hours of use per day, usually yes, and the saving grows with utilisation.
Only if you want the managed training platform. For raw compute, mid-size providers are frequently cheaper.
