How to compare GPU VPS providers without getting fooled by the cheapest hourly price

Open the comparison spreadsheet for GPU VPS providers and the first column is always hourly price. RunPod $0.34/hr for an A100. Lambda $1.10/hr. Vast.ai $0.23/hr if you take a community node. Hetzner GPU $0.80/hr.

This column is the most prominent and the least useful. Two providers with identical hourly rates can produce wildly different monthly bills depending on the workload. A provider that's twice as expensive per hour can be cheaper per finished job if its setup is faster, its bandwidth is higher, or its GPU is genuinely twice as fast at your specific load.

The right comparison framework is a checklist that translates hourly price into real cost. Here's the framework, with how to apply each dimension.

Dimension 1: what is your actual workload?

Stop comparing GPUs in abstract; start comparing them on what you're going to run.

Inference of small models (≤13B params). Memory is the binding constraint, not raw compute. An A6000 with 48GB VRAM may beat an A100 with 40GB at half the price. The 40GB A100 forces quantization or split-loading; the 48GB card runs the model whole.

Inference of large models (70B+). Two A100 40GB outclass one A100 80GB only if your framework supports tensor parallelism cleanly. If not, single big-memory cards win.

Fine-tuning. Compute and memory both matter. The right balance depends on the fine-tune type (LoRA vs full). LoRA can run on smaller VRAM; full requires high memory.

Training from scratch. You probably aren't doing this on a VPS. If you are, ignore most of this guide and rent reserved capacity from a hyperscaler.

Image generation (Stable Diffusion, SDXL, FLUX). Memory matters, but throughput per dollar is the real metric. RTX 4090 community boxes often crush A100 setups on cost-per-image.

Embeddings. Sequential throughput matters more than VRAM. Older T4 or L4 cards can be the right tool.

RAG / Vector search. GPU is rarely the bottleneck; bandwidth and CPU are. You might not need a GPU VPS at all.

List your actual workload first. Only then look at GPU options.

Dimension 2: VRAM is the gating constraint for most loads

For anything LLM-related, VRAM is the constraint that decides whether your model runs at all. Performance comparisons are pointless if the model OOMs.

Rules of thumb:

• 7B model, fp16: ~14GB minimum.
• 13B model, fp16: ~26GB minimum.
• 34B model, fp16: ~68GB minimum.
• 70B model, fp16: ~140GB minimum (so multi-GPU).
• Quantization (4-bit) ~halves these.

Add 30-40% for context and KV cache during inference. Add more for fine-tuning state.

When comparing providers, lay out the VRAM column next to the price column. Sort by fits-your-model-yes/no first; sort by price within the yes group.

Dimension 3: sustained performance vs benchmark performance

Marketing benchmarks are usually short-burst. A GPU that hits 90 TFLOPS for a 30-second test may thermal-throttle to 70 over a long inference job, especially in dense rack setups.

Ways to check:

• Look for benchmark data from real users on Reddit (r/LocalLLaMA is a good source).
• Compare provider reviews mentioning thermal throttling or noisy neighbor issues.
• For community marketplaces (Vast.ai), check the host's reliability score and review history.
• For new hardware (H100, H200), check whether the provider has actually deployed the GPU you're buying or whether it's a marketing page.

Sustained performance is the metric that determines time-to-finish on a long job. Time-to-finish is what determines real cost. Hourly price × hours = bill.

Dimension 4: bandwidth and storage

GPU VPS shoppers chronically underestimate the cost of data movement.

Outbound bandwidth. Providers vary 100x on this. RunPod includes generous outbound on its newer machines; some hyperscalers charge per GB. If your workload returns large outputs (images, video, model weights), this can dominate the bill.

Storage. Some providers offer ephemeral local SSD only; some offer persistent block storage at extra cost. If you need to keep model weights between runs, persistent storage matters. Otherwise, you're re-downloading the model every session.

Network attached storage performance. If the provider's persistent storage is slow, model load times can rival actual inference times. "Cheap GPU, slow disk" produces high effective cost-per-task.

Lay out the storage and bandwidth columns explicitly. Calculate what your workload moves.

Dimension 5: region

Latency from your users to the GPU matters for interactive use cases (chat-style inference, image generation triggered by user actions).

Things to consider:

• Where are your users?
• Where is your data?
• Are you subject to data residency requirements?
• Does the provider have a region near both?

For batch workloads, region is irrelevant. For real-time, it can be the deciding factor.

EU-residency requirements can also limit choices substantially. Some US-only providers have no EU region; some have one. Note this column explicitly.

Dimension 6: setup friction

The time from "reserve a machine" to "run my code" varies wildly:

• Pre-built Docker / templates. RunPod, Lambda, and a few others ship with PyTorch/CUDA templates. Setup is minutes.
• Bare images. You install CUDA, drivers, your framework. Setup is hours, sometimes a full day with version mismatch chasing.
• Spot/community machines. Variable image quality. Sometimes everything works; sometimes you spend longer setting up than running.

For short jobs, setup friction can be the largest cost. A $0.20/hr machine that takes 90 minutes to set up costs more in your time than a $1.00/hr machine you can run a 30-minute job on immediately.

When comparing providers, estimate the setup time honestly. Include it in the cost.

Dimension 7: hidden cost amplifiers

A few items the spreadsheet usually misses:

Egress for model weights. If you're using a custom model and you tear down the machine after every session, you pay to re-upload it each time. Persistent storage often pays for itself.

Idle minutes during exploration. Interactive jobs (notebook sessions) accumulate idle minutes. The cheap hourly machine that's billed by the minute helps; the one billed by the hour doesn't.

Failed runs. Spot instances that get evicted mid-job count toward your bill but produce no output. Reliability matters.

Support latency. When the machine has a problem, how long until it's fixed? A 4-hour outage on a $1/hr machine is much worse than the price difference vs a $1.50/hr provider with same-hour support.

These five items can easily double the effective cost of the "cheap" provider.

A practical comparison sheet

When evaluating providers, the columns to include:

This sheet predicts the monthly bill way better than the hourly price column alone.

When the cheap provider really is the right answer

For batch, fault-tolerant, output-light workloads — embedding millions of documents, batch image generation that doesn't care about reorder, periodic fine-tunes — the cheap community providers (Vast.ai, RunPod Community Cloud, etc.) often genuinely win.

The characteristics that make this work:

• The job can survive eviction (it's idempotent or checkpoints regularly).
• The job doesn't produce huge outbound data.
• The job can wait for available capacity instead of needing immediate spin-up.
• The job uses a model the provider's image already supports.

When those four are true, paying 3-4x for a managed provider is overpaying.

When the expensive provider is the right answer

For interactive, latency-sensitive, output-heavy workloads — production chat APIs, real-time image gen tied to user actions, fine-tunes that need full reproducibility — the managed and well-supported providers usually win. The hourly delta is overwhelmed by:

• Faster setup → less developer time.
• Reliable instances → predictable scheduling.
• Better networking → lower egress costs and lower user-facing latency.
• Real support → faster recovery from issues.

If you can't tolerate the failure modes of cheap providers, the math says pay more.

The summary

GPU VPS comparison is not a sort-by-hourly-price problem. It's a six-axis decision: workload fit (VRAM), sustained perf, bandwidth and storage, region, setup friction, and reliability. Lay out the columns. Calculate the effective cost of finishing your specific job. The cheapest hourly often isn't cheapest monthly; sometimes it is. The framework lets you tell which case you're in.