Local AI or cloud AI: what the choice rests on
Running a model on your own hardware or calling it through an online service are both defensible options. The choice rests on the nature of the data being processed, the real workload, the skills available in-house and the reversibility you want — not on a general principle.
What we are talking about
Local AI: the model runs on hardware you control — a server on your premises, or a dedicated machine at a host. The data processed does not leave that perimeter.
Cloud AI: you call a model hosted by a provider. Your application sends the request, receives the answer, and manages no infrastructure.
The debate is often framed as an ideological choice between sovereignty and convenience. In practice four concrete criteria are enough to settle it, and they do not always point the same way.
The four criteria that decide
1. The nature of the data
This is the dominant criterion, and sometimes the only one that matters. Data covered by professional secrecy — legal files, health data, an accounting firm's client records — points towards local processing or a dedicated instance. Data that is already public or innocuous does not justify the constraint.
2. The real workload
Cloud is paid per use: a few dozen runs a day cost little. Local is paid up front: the hardware costs the same whether it runs one hour or twenty-four. The tipping point depends on volume and on the model used — it can be calculated, it should not be guessed. A useful rule: while usage is not yet stable, the cloud avoids buying hardware for a need you do not know well.
3. The skills available
A local inference server has to be monitored, updated, backed up, and restarted when it falls over. Without someone to do that — in-house or under contract — local becomes a point of fragility rather than a guarantee of control.
4. The reversibility you want
A provider can change its pricing, its terms, or withdraw a model. Local protects you from that at the price of operating it. A middle position is to isolate the layer that calls the model from the start, so you can change it without rewriting the rest of the system. That is the most prudent choice when you do not yet know.
Comparison table
| Criterion | Local AI | Cloud AI |
|---|---|---|
| Data leaving | None outside the perimeter | Yes, to the provider |
| Cost structure | Up-front investment, low per-use cost | No investment, cost per call |
| Time to start | Days to weeks | Hours |
| Available power | Limited by the hardware | The largest models available |
| Operations | Your responsibility | The provider's responsibility |
| Scaling up | Buying hardware | Immediate |
| Vendor dependence | Low | Real |
| Offline operation | Possible | Impossible |
A property management firm wants to classify maintenance requests received by email and extract the building concerned. Volume is a few dozen a day. The data contains tenant names and addresses, but nothing covered by professional secrecy.
Here a cloud service under a business contract is a defensible starting point: the volume does not justify a hardware investment, and setup takes a few hours. The useful precaution is to isolate the model call behind an internal interface, so that moving to a local solution stays possible if volume grows or if a client demands it.
The mixed approach
Nothing forces a once-and-for-all choice. A frequent and reasonable split:
- Local: anything touching client documents — extraction, filing, indexing for RAG. These are also well-defined tasks, where a smaller model is enough.
- Cloud: anything requiring reasoning on non-sensitive content — rewriting, summarising, drafting help.
This split assumes an explicit rule about what is allowed to leave. Without that rule written down, the boundary drifts as things are added, and generally in the wrong direction.
Common errors of reasoning
- Believing local means secure. An exposed, poorly maintained server is riskier than a properly contracted professional service. Security is a practice, not a location.
- Underestimating operations. A server's cost does not stop at its purchase price: it includes someone's time, every month, for years.
- Overestimating the power needed. Extracting five fields from an invoice does not call for the largest model on the market. The task dictates the size, not the other way round.
- Treating the question as final. It is a revisable architectural decision, provided it was designed to be revised.
Frequently asked questions
Is local AI cheaper?
It shifts the cost: less cost per use, but a hardware investment and operating time. It becomes interesting at sustained, regular load.
Can you start in the cloud and move local later?
Yes, provided you isolate the layer that calls the model from the start, so that you can swap it without rewriting the rest.
Is the cloud ruled out for sensitive data?
No. What matters is the contract, where processing happens, the retention policy and the non-reuse of data. Those points can be checked and written down.
Related articles
Sources
- Centre national pour la cybersécurité (NCSC) — https://www.ncsc.admin.ch/ncsc/fr/home.html
- Préposé fédéral à la protection des données et à la transparence (PFPDT) — https://www.edoeb.admin.ch/fr