Cloud vs Self-Hosted AI: A Practical Guide

Cloud AI means you send prompts to a vendor and pay per token or by subscription. Self-hosted AI means you run an open model on hardware you control, at home, in an office or on rented GPUs. Cloud wins on strength and simplicity. Self-hosting wins on privacy and control, and on cost only when the GPU stays busy. There is a middle route, too. Checked 31 October 2026.

The short version

The three places AI can run

1. A vendor's cloud. You call Claude, GPT, Gemini or a host that serves open models. You pay per token or by plan.

2. Your own cloud account or rented GPUs. You rent a machine from a GPU provider and run an open model on it. You pay by the hour or second.

3. Your own hardware. A laptop, a desktop with a good GPU, or a server in a rack. You pay once and for power.

The last two are both called self-hosted. The difference is whose building the machine sits in. Rented GPUs are other people's computers, which matters for privacy.

Costs

Cloud APIs

From the pricing pages on 31 October 2026, per million tokens:

Consumer plans exist too, such as Claude Pro at $20 a month. Our Anthropic vs OpenAI API page compares the two big vendors, and DeepSeek vs Claude covers the cheap open-weights option.

Rented GPUs

Runpod's list rates per hour, from its pricing page: RTX 4090 $0.34 (Community) or $0.89 (Secure), A100 PCIe $1.19 or $1.79, H100 SXM $2.69 or $3.99, H200 $3.59 or $5.29. Modal bills per second, with the H100 SXM5 near $3.95 an hour. A month on one H100 SXM is about $1,964 on Runpod Community Cloud, whether or not anyone uses it. See Runpod vs Vast.ai and Modal vs Runpod.

Your own hardware

You pay up front for the machine and then for electricity. LM Studio's docs recommend 16 GB of RAM or more and say an 8 GB Mac can run small models with modest context. We do not publish hardware prices here, since they change fast.

The break-even question

Take your monthly API spend and compare it with the monthly cost of a GPU that could do the same work. If the API bill is lower than the GPU rental, cloud is cheaper on price alone, and the GPU also has to be big enough and busy enough. Self-hosting tends to win when volume is high and steady, the tasks are simple enough for an open model, and someone on your team can run the server. It loses when usage is occasional, because an idle GPU still bills.

Privacy and control

This is the strongest reason to self-host.

With cloud AI, your prompts go to the vendor, and its terms govern retention and training. Vendors differ. OpenRouter's FAQ says it logs basic request metadata such as timestamps, model and token counts, and that prompts and completions are not logged by default. LM Studio says its cloud services are zero data retention. Ollama offers optional cloud models and a setting, OLLAMA_NO_CLOUD, to turn them off. We did not read the retention and training terms of the large API vendors for this page, so read them for your plan.

With self-hosting on your own hardware, prompts never leave the machine. With rented GPUs, they go to the GPU provider, which is a different company with its own terms and staff. For client code or regulated data, that distinction matters.

Control goes beyond privacy. Self-hosting lets you pin a model version, so results do not change overnight, choose when to upgrade, and fine-tune on your own data. Cloud APIs retire models on the vendor's schedule.

Quality

Cloud vendors ship the largest and most capable models. Self-hosted models are limited by the memory of the machine, and the best open-weights models are large. A laptop runs small models well. A single server GPU runs mid-size ones. A multi-GPU rack is needed for the biggest.

We do not quote benchmark scores, because comparing models fairly takes tests on your own tasks. Our open vs closed models page explains the licences and the trade-offs, and compare models lets you look at specs side by side.

Effort

Cloud is an API key. Self-hosting is a project.

Runtimes help. Ollama and LM Studio make a single machine easy. For many users, vLLM is built for concurrent serving. llama.cpp gives you the most control.

The middle routes

Pure cloud and pure self-hosting are the ends of a range.

• A hosted open model. You pay per token to a host that serves open weights. Your prompts go to the host, not to the lab that made the model. Gateways list many hosts; see OpenRouter alternatives.
• A private cloud endpoint. Big clouds offer model hosting inside your own account, such as AWS Bedrock, with data staying in that account under its terms.
• Rented GPUs with a serving engine. The self-hosted route without buying hardware.
• A split by task. Cloud for hard work, local for simple or private work.

For private deployments in general, see private AI.

Using agents with each route

Coding agents can point at either. Most accept an OpenAI-style or Anthropic-style endpoint, which Ollama, LM Studio and vLLM all expose. A local model on a laptop is slower and weaker at multi-step tool use than a hosted frontier model, so expect to keep a strong cloud model for hard tasks. The quickest hybrid is one agent with two model settings: a cloud model by default and a local one for sensitive repositories. See open-source agents.

A worked scenario: a ten-person team

Suppose ten developers use coding agents daily. Three scenarios, using only published list prices and example usage you should replace with your own.

Scenario A: subscriptions. Ten Claude Pro seats at $20 a month is $200 a month, with five-hour and weekly limits. Simple to buy, and some people will hit the limits.

Scenario B: API by the token. If the team's agents together use 500 million input tokens and 50 million output tokens a month on Sonnet 5.5, that is 500 times $2 plus 50 times $10, or $1,500 a month. The same volume on DeepSeek flash at off-peak rates is 500 times $0.15 plus 50 times $0.60, or $105 a month, if the quality is good enough for the task. Those token counts are an example, not a measurement. Agent loops resend context, so real totals can be much higher.

Scenario C: self-hosted on rented GPUs. One H100 SXM on Runpod Community Cloud for a month is about $1,964. That buys nothing if the model you need does not fit on one card, and it needs an engine such as vLLM and someone to run it.

The lesson is not that one scenario wins. It is that the answer swings by an order of magnitude with usage and model choice, so measure first. For a team this size, the cheapest route is often a hosted open model for bulk work and a subscription for the hard cases. Self-hosting becomes attractive when the data rules leave no other choice.

Setting up a hybrid in practice

A hybrid takes an afternoon.

1. Install a local runtime such as Ollama or LM Studio and pull a small model that fits your memory.
2. Keep your cloud agent as the default. Note where its settings take a base URL or model name.
3. For private repositories, switch to the local endpoint at http://localhost:11434/v1/ for Ollama, or LM Studio's local server.
4. Write the rule into your project instructions: which repositories may use the cloud and which must stay local.
5. For bulk tasks, add a hosted open model through a gateway, and cap its spend.
6. Review the setup each quarter, since prices and models change.

Review the first month's bills against your estimate and adjust. Keep the rule simple enough that a new teammate can follow it without asking. A rule nobody remembers is not a control.

A checklist before you decide

1. What is the data, and who is allowed to see it?
2. What is the monthly volume, measured, not guessed?
3. What quality do the tasks need, and have you tested an open model on them?
4. Who runs the servers, and what happens when they fail at 2 a.m.?
5. What do the vendor terms say about retention and training, for your plan?
6. What is the exit plan if a vendor raises prices or retires a model?
7. When will you re-run these numbers?

Limits and gotchas

• Idle GPUs bill. A forgotten rented GPU is the most common surprise.
• Local models are smaller. Do not expect cloud-level reasoning from a laptop model.
• Security is yours. An open port with no authentication is a breach waiting to happen.
• Compliance still applies. Self-hosting does not by itself satisfy a regulation. It only removes one third party.
• Prices move. Both API prices and GPU rates change. Rerun your arithmetic quarterly.
• People cost. Server upkeep takes time that does not show on a pricing page.

Who should pick which

A freelancer or solo developer

Cloud, through a subscription or API, with a small local model for private repos and offline work.

A startup

Cloud first. Move simple, high-volume tasks to a hosted open model when the bill justifies it.

A company with sensitive data

Self-hosted, or a private cloud endpoint under a contract your legal team accepts. Pilot first.

A research group

Rented GPUs for training and big experiments, local machines for daily work.

An enterprise

A mix, governed by data class: public data to cloud, regulated data to private endpoints or on-premises.

Break-even, worked through

The scenarios above give totals. This section turns Runpod's hourly rates into a threshold you can compare with your own token counts. A month is about 730 hours, and the GPU runs the whole month, which is the worst case for a rented card.

Now ask what API spend that would buy. On Sonnet 5.5 at $2 input and $10 output, a job with ten input tokens for each output token costs $30 per 10M input and 1M output. An H100's $1,964 is about 65 times that, which is roughly 650M input tokens and 65M output tokens a month. Below that volume on Sonnet-class pricing, the API is cheaper on price alone, and the GPU only wins if you use it above that line, keep it busy and the model you need fits on the card.

Two caveats move the line. An open model that is good enough for your task may cost far less per token through a hosted API than Sonnet does, which pushes the break-even higher still. Serverless GPU billing, which charges only while a request runs, removes the idle-month penalty and lowers the threshold. The honest summary is that self-hosting on price alone pays off only for steady, high volume. For most teams the reasons to self-host are privacy and control, with cost a side effect.

Sorting your data by where it may go

Decide this before you pick infrastructure. A short table, filled in with your own categories, settles most arguments.

The columns echo the earlier point: rented GPUs are other people's computers, so they do not give you the privacy that your own hardware does.

Questions people ask

Is a local model on a laptop good enough for coding? A small local model handles simple edits and private snippets. Hard multi-file work still goes to a larger cloud model for most teams, which is why the hybrid setup above exists.

What is the cheapest way to try self-hosting? Install Ollama or LM Studio on a machine you already own and pull a small model. It costs electricity and an afternoon.

Do I need an engineer to run it? For one laptop, no. For a shared server with uptime expectations, yes, because updates, monitoring and security become your job.

Our take

Start in the cloud. It is the fastest way to learn what AI can do for your work, and the price at low volume is small. Move work to self-hosted or hosted open models for three reasons only: privacy rules, steady high volume, or a need to control model versions.

We did not run benchmarks or build a cluster for this page. The prices come from vendor pages on 31 October 2026 and the arithmetic is yours to redo. The best advice is to measure your real monthly usage for a month first, then price both routes against it.

Related pages

• Open vs closed models
• Ollama vs vLLM
• Private AI
• Compare hosting providers