Open-source models or Claude, ChatGPT, Gemini, Grok? First ask whether your data can leave the company

Cloud models are smarter and need no hardware. Open-source models run inside your company, and your data stays put. This post compares pricing, data handling, and limits across Anthropic, OpenAI, Google, xAI, and open-source models, and tells you how to choose, or how to run both.

Johnny Yang7 min read
Open-source models or Claude, ChatGPT, Gemini, Grok? First ask whether your data can leave the company

"Claude, ChatGPT, and Gemini are all this strong now. Why would we install an open-source model ourselves? Or is open source actually better?"

There is no standard answer to this question, but there is a very clear way to decide. Many owners treat it as a contest over which one is stronger. The question you should actually be asking is a different one.

I'm Johnny Yang, founder of Brightstream Technology in Taiwan. I've led my team through system implementation for seven years, more than 252 projects across over 32 industries. Some of our systems connect to cloud models. Others run offline, with the model installed in the client's own server room.

My answer: open source isn't better, and cloud isn't better. What matters is whether your data can leave the company. If it can, cloud is usually smarter and less work. If it can't, open-source on-prem is the only way you get to use AI at all. Many companies end up running both: sensitive data goes on-prem, everyday work goes to the cloud.

Why this isn't about which one is stronger

Let me admit one thing first: on raw intelligence, top cloud models are still usually stronger at complex reasoning. This is no secret. This roundup on Hugging Face says the same.

But "stronger" only means something if you can use it. If you're a medical clinic, patient records can't be sent out. If you're a testing lab, your clients' sample data is under confidentiality agreements. If you're bidding on government contracts, the agency explicitly requires that data not leave the country. In those cases, however strong the cloud is, you can't use it.

So the first question is always whether your data can leave. Only the second question is which one is stronger or cheaper.

How do the four big cloud providers compare?

Below are the models and prices published on the four providers' official pages as of November 2026. Prices are in US dollars per million tokens, input first, output second.

ProviderFlagship modelMid-tierSmallCan it be installed in the client's server room?
AnthropicClaude Fable 5.2: 8/40; Opus 5.5: 4/20Sonnet 5.5: 2/10Haiku 5.5: 0.10/0.50No, but available through AWS, Google Cloud, and Microsoft Foundry
OpenAIgpt-6.2-astra: 8/40gpt-6.2-sol: 2/10gpt-6-luna: 0.10/0.50No
GoogleGemini 4 ArgonGemini 3.8 FlashGemini 3.1 Flash-Lite: 0.25/1.50No
xAIgrok-4.7: 2/6Not officially listedNot officially listedNo
Open-source modelsgpt-oss-120b, DeepSeek, Qwen, etc.Qwen3.8-27B, Gemma 4 31Bgpt-oss-20b, Gemma 4 12BYes

We couldn't find prices for some Google models on the official pages, so the table leaves them out. OpenAI's small-model figures don't match between its models page and pricing page; go by the latest official announcement. None of the four can be installed in the client's own server room. That is the biggest difference with open-source models.

Comparison of cloud LLMs and open-source on-prem models: intelligence, where data goes, cost, and maintenance

Where does your data actually go in the cloud?

Many owners worry: "If we use cloud AI, will our data be used for training?" That depends on what each provider says:

  • OpenAI: States that data sent to the API is not used to train or improve models. Abuse-monitoring logs are kept for up to 30 days (source).
  • Google: The paid tier is not used to improve products; the free tier is. Abuse monitoring keeps prompts and outputs for 55 days (source).
  • Anthropic: You can apply for zero data retention, and you can specify inference in the US.
  • xAI: Retains data for 30 days by default. You can turn on zero data retention yourself, but some features get disabled.

For many companies, these terms are good enough. But "kept for 30 days" or "kept for 55 days" is unacceptable for patient records, formulas, and confidentiality agreements. Then on-prem is the only option. I cover the security baseline for handing company data to AI in more detail in Is it safe to give company data to AI?

Pros and cons of open-source models

The pros are clear: data never leaves the company. You choose the model and tune it yourself. No usage-based billing, so the more you use it, the better the deal. You aren't affected when a cloud vendor changes prices, changes policies, or shuts down a service.

The cons need to be stated just as clearly: you have to buy hardware, and hardware prices are still rising in 2026. Someone has to maintain it; model updates and security patches are on you. Mid-size models are usually weaker than top cloud models at complex reasoning. Multiple people using it at once needs separate design work. I listed these one by one in What on-prem AI can't do.

One more point: the big cloud providers have stricter content policies. For example, Anthropic explicitly bans sexual content, and xAI bans pornographic content using real people's likenesses. Most of these rules are reasonable, but sometimes they refuse normal medical questions too, like a medical aesthetics clinic asking about intimate-area treatments. Open-source models can be adjusted to stop this kind of over-refusal. That's another kind of flexibility they give you.

How do you run both?

In practice, what we recommend most often is a hybrid setup:

Data typeWhere it goesExamples
Sensitive, can't leave the companyOn-prem open-source modelPatient records, test data, formulas, client contracts, detailed financials
General, can leaveCloud LLMMarketing copy, organizing public information, coding help
Can be de-identifiedDe-identify first, then cloudClassifying customer service questions, anonymized report analysis

Running both takes attention to two things. First, you need a routing mechanism so the system automatically decides what can't go out, instead of relying on employees to remember. Second, the answer style and format on both sides should be as consistent as possible, so employees don't feel like they're using two systems.

This routing mechanism is part of what we handle when we build a company's system. AI isn't a separate web page you open. It's wired into the systems and LINE your employees use every day, deciding in the background which model each request goes to. For adding AI to an existing system, see We already have an ERP. Do we still need AI?

Steps of a hybrid setup that routes data to on-prem or cloud models by sensitivity

How do the costs compare?

Finally, money. Cloud is pay-as-you-go: you pay for what you use. On-prem is a one-time hardware purchase, plus electricity and maintenance.

At low volume, cloud is cheaper. At high volume, on-prem is cheaper. One vendor estimates you need roughly 8 million to 10 million tokens a day to break even (see MAQ; these are the vendor's own assumptions). Most small and mid-size businesses use far less than that, so on cost alone, cloud usually wins.

But cost isn't the only consideration. When data can't leave, on-prem is the only option. When data can leave, then you compare costs. For hardware costs, see What hardware do you need for on-prem AI? For overall implementation costs, see How much does AI implementation cost?

FAQ

Can Claude or ChatGPT be installed on our own company servers?

No. These cloud LLMs can only be used over the internet. At most, you can choose to run inference on a specific cloud platform or in a specific region. To install a model entirely inside your company, you have to use an open-source model.

If we use cloud AI, will our data be used for training?

The major providers' paid enterprise APIs mostly state they don't train on your data, but they keep it for a period for abuse monitoring, such as 30 or 55 days. That's fine for most data. But for patient records, formulas, and confidentiality agreements, even retention usually isn't allowed.

Are open-source models cheaper than cloud?

At low volume, usually not. On-prem means buying hardware, paying for electricity, and having someone maintain it. Cloud charges for what you use. The main reason for on-prem is that data can't leave, not that it saves money.

Can we use cloud and on-prem at the same time?

Yes, and it's common. Sensitive data goes on-prem, everyday work goes to the cloud. The key is an automatic routing mechanism, not employees making the call themselves.

Conclusion: ask about the data first, then compare models

Open source and cloud aren't about one replacing the other. It comes down to whether your data can leave. If it can, cloud is smarter and less work. If it can't, on-prem is the only way. If you have both kinds, run both.

If you're not sure which side your data belongs on, take a look at how we do AI system development, or bring your most sensitive category of data and come talk to me.

Found this useful? Pass it on
30 minutes · free

Does this sound like your company?

Spend 30 minutes telling Johnny Yang how you do it now, and you get told where hours are leaking, where the risk is buried, and whether this is worth paying to fix. No pitch, no course to sell — if it is not a fit, that is fine too.

Pick a slot online · with Johnny Yang himself · 252+ projects behind it

Johnny Yang (熊董)
Johnny Yang (熊董)
Founder, Brightstream Technology

Serial founder leading a small team that builds systems into an owner's everyday work.

As a founder
7 yrs
Projects delivered
252+
Industries served
32+
About Johnny