On-premise or cloud AI for legal work

The question goes like this: can we not just run the model ourselves and avoid the third country question? Yes, that can be done, and it solves part of the problem at the price of others. Here is what local operation actually gives, what it costs a department of one to ten lawyers, and the three cases where I think it is justified.

Updated by Nils-Erik Jansson

What the question is really about

Local operation means the model runs on hardware you control, on your own servers or in a Swedish provider's data centre, using open models whose weights can be downloaded. The cloud means the model runs at the vendor. For Claude that is in the US, with routing to several regions [3]. There are hybrids: US models through European cloud regions, and Swedish providers that run open models for several customers.

The question is legal and financial: which risks go away, which are added, and what it costs. The table compares the two on the points covered below.

Question Local Cloud
Third country transfer for the model run None, if model and data are in Sweden For Claude, storage in the US and routing to several regions [3]
Authority requests to the vendor No US vendor that can be ordered to disclose Remaining risk for vendors in a US group [1]
Retention and deletion You decide, with no exceptions for flagged content or feedback The vendor's settings and exceptions
Vendor changes to the terms Nobody can change what applies to your model unless you switch version The vendor can change the terms
Model quality The open models trail the large vendors' latest The large vendors' latest
Hardware and operations You carry them: capacity, updates, security, monitoring The vendor carries them
Time to start A project for the company's IT Four weeks for a setup
Responsibility when something goes wrong Your own There is a vendor to point to

What local operation solves

  • The third country transfer for the model run: if the model and the data are in Sweden, no transfer tool and no transfer impact assessment is needed for that processing. The gain is real.
  • Authority requests to the vendor: there is no US vendor that can be ordered to hand over what only you hold. The guidance on external IT services from the Swedish Bar Association (Sveriges advokatsamfund) points to a remaining risk for vendors in a US group [1]. For this part, that risk goes away.
  • Retention and deletion: you decide, with no exceptions for flagged content or feedback.
  • Independence from changes to the vendor's terms: nobody can change what applies to your model unless you switch version.

What it costs

Hardware, or rented capacity, enough for a model that does the job. Operations: updates, security, monitoring, someone who answers when it stops. Interface and integrations: what the lawyers actually use is the tool around the model, and someone has to build or buy it. Weaker models: the open models trail the large vendors' latest, and the gap varies. Your own responsibility: when something goes wrong, there is no vendor to point to.

And time. A setup in the cloud takes four weeks. A local environment with interface, integrations and operations is a project for the company's IT department, outside the legal department's own work.

What it does not solve

GDPR in general. Personal data in prompts is still processing, with a legal basis, a purpose, a retention period and the data subjects' rights. Review of the answers: a local model can be wrong just as a cloud model can. The policy, the training and the list of what may be entered: you need all of them anyway. And the chain around the model: document systems, email, identity and any support are often cloud services with transfers of their own [2]. The assessment moves a little. It does not disappear.

Three cases where local operation is justified

Matter types that are otherwise kept out. A department with a list of matters that may not go into the cloud tool, where those matters are many or important, can run just those locally with a simpler model for defined tasks. That means two environments and one rule for which is used when.

Companies that already have the capacity. If the company has the infrastructure, the skills and an AI platform for other reasons, the legal department's use is a small extra cost. The question then concerns interface and playbook rather than operations.

Businesses with requirements that cannot be contracted away. Some clients, in the defence industry, the public sector or regulated sectors, set localisation requirements that no cloud vendor meets by contract. Then local operation is a condition of the work.

For most departments with one to ten lawyers the answer is the cloud, properly contracted and properly configured, with a list of what is kept out. That is my assessment, and I will review it as the open models and the Swedish hosting market move.

The question comes up in legal technology selection, where local operation is one of the options assessed under the same protocol as the cloud tools, and in the AI operating model, where two environments can be part of the model. The data protection questions are the same either way, see the checklist.

Frequently asked questions

Does local operation solve the third country question completely?
For the model run itself, yes: if the model and the data sit on your own servers in Sweden, nothing is transferred to the US for that processing. But the tools around the model, such as document storage, email, identity management and support, are often cloud services with transfers of their own. The assessment moves. It does not disappear. And the five data protection questions still apply, see the checklist.
Are local models good enough for legal work?
For defined tasks on your own documents, such as classification, summarisation and review against a playbook, they are enough. For work that needs the best model available at the time, with long reasoning and broad knowledge, the open models trail the large vendors' latest releases. The gap changes over time. Test them on your own contracts rather than on the vendor's examples.
What does local operation cost a small department?
More than the cloud licences, because hardware, operations, updates, security and IT staff time all count, and that is before anyone has built an interface the lawyers want to use. Figures depend entirely on the level of ambition, so I give none here. The cost is probably too large for a small department to carry alone. It is carried by the company, if the company has other reasons.

Sources

  1. [1]Uppdaterad vägledning om användningen av externa IT-tjänster i advokatverksamhet, Sveriges advokatsamfund (Swedish Bar Association), 15 April 2026 (in Swedish), point 4 on external access and the Cloud Act
  2. [2]GDPR-vägledningen, Sveriges advokatsamfund (Swedish Bar Association), 10 September 2026 (in Swedish), section 12 on third country transfers
  3. [3]Where are your servers located?, Privacy Center, Anthropic

Back to AI for legal

More under AI for legal