Private AI for Teams: Options, Costs and Rules

Private AI for a team means prompts and files go only to systems the team controls or has a signed data agreement with. There are four ways to do it: a shared machine you run, a GPU you rent and manage, a hosted open-model API with documented data terms, or a team plan from a model vendor. Costs and risks differ. Figures below were checked on 31 October 2026.

Four setups compared

None is private in an absolute sense. Each moves the trust boundary. The question to answer for each is who can read the prompt, how long they keep it, and what they may do with it. Our cloud vs self-hosted AI page covers the general trade.

Start with the data, not the tool

Teams often pick a tool and then ask what data may go in. Reverse it.

Write this down on one page. Then each tool decision is a lookup, not an argument.

Option 1: a shared machine you own

One server, one runtime, one proxy. For a team of up to about ten, Ollama is enough. Past that, or when agents are involved, use vLLM. Both expose an OpenAI-compatible API, so most tools can point at them.

Memory is the planning figure. From Ollama's library: gpt-oss 20B is 14 GB, gpt-oss 120B is 65 GB and fits one 80 GB GPU, Qwen3-Coder 30B is 19 GB. Cache grows with context and with concurrent requests, per Ollama's FAQ. See GPU requirements for LLMs.

Pros: the strongest data control, and no per-token cost. Cons: you carry the operations, and open models trail frontier models on hard tasks. The broader on-premise picture is in on-premise AI.

Option 2: a rented GPU you manage

You install the same software on a rented card and keep the weights and the logs. Prices checked 31 October 2026:

Some links go through /go/ and may earn us a fee; it never changes our order. An example with volumes we chose: a team of eight uses an A100 for 10 hours a day on 22 working days, 220 hours. At Runpod Secure's $1.79 that is about $394 a month, or about $49 a head. Always on, 720 hours, it is about $1,289.

The privacy note: it is a rented machine in someone else's data centre. Choose a tier and a provider whose data terms fit your policy, and read them. Community tiers on marketplaces are shared with other hosts' conditions. Runpod's docs say Secure Cloud runs in T3/T4 data centers while Community Cloud uses peer-to-peer providers.

Provider pages: Runpod, Lambda Cloud, Modal. Side-by-sides: Runpod vs Lambda, Modal vs Runpod.

Option 3: a hosted open-model API

Here you run nothing. A provider serves open models and charges per token.

For a team that uses 40 million input and 8 million output tokens in a month (an example), Groq's rates give $6.00 plus $4.80, about $10.80, and Cloudflare's give $14.00 plus $6.00, $20.00. These are far below a rented GPU, and that is the point: for light use, an API is cheaper by an order of magnitude. What you give up is control over where the prompt is processed and for how long it is kept.

So the data terms are the whole question. Read the provider's retention and training statements, and look for a zero-data-retention option. Groq's data page says it does not keep inference inputs or outputs by default, lets any customer turn on zero data retention in its settings, and does not say whether it trains on customer data. Cerebras's privacy policy says it does not retain inputs and outputs for its inference services, and notes it may not cover data processed under a customer agreement. Ollama's FAQ says that for its cloud models it processes prompts and responses to provide the service, does not store or log that content and never trains on it. Cloudflare's pricing page lists Workers AI as billed in neurons at $0.011 per 1,000 above 10,000 free per day. Cloudflare's Workers AI data usage page says it does not retain inputs or outputs once they are no longer needed to process the request and does not use Customer Content processed by Workers AI to train models.

Option 4: a team plan from a model vendor or platform

Team plans bring admin controls and contracts. Examples we checked:

At ten people, Hugging Face Team is $200 a month and Enterprise is $500. Ollama's Team plan is flat at $500 whatever the headcount, with $1,000 of credits shared. The better deal depends on the tokens you will really use, so measure before you buy. For managed endpoints, AWS Bedrock's documentation says model providers do not have access to its deployment accounts or to customer prompts and completions. See AWS Bedrock and Hugging Face Inference.

A one-page policy you can adapt

Keep it short enough that people read it. A workable shape:

1. What this tool is for. Name the approved tools and the use cases.
2. What may go in. Use the data classes above, with examples from your own work.
3. What may not. Credentials, customer records and anything under a contract that bars it.
4. Who owns it. A name and a contact.
5. What is logged. Say whether prompts are stored and for how long.
6. How to ask for more. A route for requesting a bigger model or a new use case, so people do not go around you.
7. Review date. Quarterly.

A policy people can follow beats a strict one they ignore. If the private tool is much slower or weaker than the public one, expect leakage, and fix the tool before you tighten the rule.

Cost per head, side by side

A comparison with figures we chose, for a team of ten with moderate use. Treat the volumes as examples.

Two things stand out. The rented GPU is expensive when idle, so schedule it or shut it down. And the API line is so low that it will win on cost at light use, which is why the discussion has to start from the data policy.

Token volume is the number to measure. A coding agent can use many times the tokens of a chat user, and a few agent users can outspend a whole team of chat users. Measure a week of real use before extrapolating.

Training people

Tools do not fix habits. A thirty-minute session covers most of it.

• Show what is private and what is not, with two examples from your own work.
• Show the approved tool doing a real task, so people see it is worth using.
• Say plainly what not to paste, and what to do if they already did.
• Explain how to report a problem, and who answers.

Repeat it for new hires, and again when you change the setup. Most leaks are habit, not malice.

If something goes wrong

Have a runbook before you need it.

1. A key leaks. Revoke it at the proxy, check usage for that key, issue a new one.
2. Data was pasted into a public tool. Record what and when, follow your incident process, and ask the vendor about deletion if their terms allow it.
3. The server is down. Say whether people wait, fall back to a hosted option, or use a local model on their laptops. Make sure the fallback respects the same data class.
4. A model change breaks a workflow. Roll back to the previous container tag or model version.
5. A provider changes terms. Review it the same week. Your policy should name who watches provider notices.

Planning an exit

Choose setups you can leave. OpenAI-compatible APIs make this easier, since the app code barely changes when the base URL changes. Keep prompts and configs in your own repository, not only inside a vendor's console. Record which model and version produced important outputs. If you adopt a team plan, check what export you get and what happens to your data when you cancel. We did not read each plan's terms.

Access and identity

A private setup with open doors is not private.

1. One credential per person or app. Keys at the proxy, revocable one at a time.
2. Tie access to your directory if your proxy allows it, so leavers lose access automatically.
3. Rate limits per key. One agent loop should not starve the rest.
4. Network placement. Keep servers off the public internet. Ollama binds to 127.0.0.1 by default and has no login, and vLLM checks only an optional API key.
5. Admin separation. The people who run the box need not read prompts.

Logging, retention and audit

Decide in advance.

• Metadata only. Who, when, which model, token counts. Enough for billing and capacity.
• Full prompts. Needed only if audit rules require them. They store your sensitive data again, so lock them down and set a deletion date.
• Provider logs. For a hosted option, ask what they keep and for how long, and get it in writing.

Use cases and the right setup

A rollout plan

Week one: policy and pilot. Write the data classes. Pick five volunteers and one use case. Stand up the simplest private option: Ollama on one machine.

Weeks two and three: measure. Track usage hours, concurrent requests, and complaints. Compare against a hosted tool with the same tasks if your policy allows.

Week four: choose. Use the table above and the numbers you measured to pick between an owned box, a rented GPU, a hosted API or a vendor plan.

Month two: harden. Add the proxy, per-user keys, limits and logging decisions. Write the runbook.

Quarterly: review. Costs, quality, model changes and who left the team.

What teams get wrong

• Treating "private" as a feature of the model. Privacy comes from where prompts go and who can read them.
• Buying hardware for a headcount, not for concurrency.
• Forgetting that a weak private tool drives people to public ones.
• No owner. The server runs until the first outage, then nobody knows who to call.
• Mixed policies. One team on a local model, another pasting into a public chatbot with the same data.

Our take

For most teams, start with the cheapest option that meets your data rule. If the rule is strict, own or rent the GPU and run vLLM or Ollama behind a proxy. If the rule is "a vendor with a good contract is fine", a hosted API or a team plan will cost less and need less work. Measure demand before you buy anything, and write the policy first. This page draws on vendor pricing and documentation pages only. We have not run a team deployment to test it. Use the hosting compare to filter providers.

Related pages

• Private AI hub and Self-hosted AI
• How to self-host an LLM
• Hosting hub