, Nils-Erik Jansson
Demo on your own contracts: a protocol for legal tech
Run the demo on your own contracts. The vendor's examples show what the tool is good at; your contracts show whether it is good at your work. Here is the protocol: which documents to choose, which tasks every vendor gets, how the answers are scored, and what the demo reveals.
Updated by Nils-Erik Jansson
Why your own contracts
The vendor's demo contracts are chosen to make the tool shine: in English, well structured, with clauses the tool recognises. Your contracts are in Swedish, sometimes scanned, with schedules that refer to each other and a playbook the tool has never seen. That is where the tool has to work, and that is where it should be tested. The LegalBenchmarks evaluation framework rests on the same principle: a staged evaluation with criteria developed by practising lawyers, which a team adapts to its own selection [1]. This is how I apply its demo stage.
The selection
Five to eight documents, chosen to cover what you actually do. A typical selection for a legal team: one of your own standard contracts, a contract on the counterparty's paper that departs from your playbook, a long contract with schedules, an older scanned contract, a contract in English, and a document that is not a contract but that you often handle, such as a procurement annex or a policy. Personal data is anonymised. The documents are the same for every vendor.
Add your playbook for one contract type, written with a standard position, a fallback and a limit. There is more in the guide on the contract review playbook. The playbook is the key to the most important task.
Six tasks
The same six for everyone, in the same order, worded the same way. My standard tasks:
- Summarise the long contract on one page with the ten most important terms and where they appear.
- Review the counterparty contract against our playbook and list the deviations with their severity and proposals drawn from the fallbacks.
- Find the notice period, renewal, limitation of liability and governing law in all the documents and set them out in a table.
- Answer a question that requires reading two documents together, for example whether the schedule changes the payment terms in the main agreement.
- Draft a reply to the counterparty on two deviations, in our tone and in Swedish.
- Ask a question whose answer is not in the documents, to see whether the tool says so or makes something up.
Two of the tasks are kept secret until the demo.
Scoring
Two assessors from the team, working independently, using the same template. For each task: right or wrong on substance, a reference to the right place in the document, Swedish that can be used without rewriting, time taken, and how much the vendor had to help. Zero to three for each point. The assessors then compare and write one sentence per task explaining why. The total per vendor is the basis for a discussion of which differences matter to you. It does not decide the choice.
What the demo reveals
- Source references. A tool that answers correctly but cannot point to where the answer is cannot be checked, and checking is the requirement; see legal privilege and AI. Task six shows whether the tool says the answer is missing or invents one.
- The Swedish. See whether summaries and drafts in your tone can be used without rewriting, because rewriting is work that comes back every day.
- The playbook. How the tool takes in your positions, whether it compares word for word or approximately, and whether it can explain a deviation to someone who is not a lawyer. This is where the difference shows between a tool built for legal work and a general tool with legal add-ons, in both directions.
- Scanned documents and schedules. See whether the tool handles the older scanned contract and the question that needs two documents, as well as clean PDFs.
Look also at the depth behind the surface. My contract management platform was not the best looking. Other CLM tools had drag and drop and a more polished interface, while mine had more depth and control. The end user's simple flow existed because the lawyers had done the thinking in advance.
Mistakes that make the demo worthless
Different documents for different vendors. Letting the vendor choose the tasks. Having only the general counsel assess. Not writing the assessment down the same day. And letting the demo replace the pilot: the demo tells you which two vendors to test in real work. Which one to buy is decided later. See the guide on pilot criteria.
How HOVR Legal works with this
The demo on your own contracts is one of the steps in legal technology selection, after the requirements list and before the pilot. HOVR Legal puts together the selection and the tasks with you, runs the demos so that they are comparable, and compiles the assessments. Every tool is assessed under the same protocol. How the legal tech fact sheets are made is set out on the page How the fact sheets are made.
Frequently asked questions
- May we share our contracts with a vendor for a demo?
- With a non-disclosure agreement and with personal data anonymised, usually yes for contracts where you are a party and the counterparty is not a client. Law firms use client contracts only with the client's consent, or anonymised so that the client cannot be identified. Better five contracts you are free to use than twenty you have to ask about. The demo can also run in your own environment, on a test licence, so that nothing leaves the company.
- How long should a demo be?
- Two hours per vendor: one hour for your tasks on your documents and one hour for questions. Longer demos turn into presentations. Shorter ones do not get through the tasks. Every vendor gets the same time, the same documents and the same tasks, in the same order, within the same two weeks.
- What if the vendor wants to prepare using the documents?
- Give them the documents a day in advance, all vendors alike. You want to see what the tool does once it is configured for you. Keep two of the tasks secret until the demo, though, so that you also see what happens without preparation. The difference between the two is a result in itself.
Sources
Insights on the topic
Back to Technology selection
More under Technology selection
- Technical review of legal tech vendors
An independent review of what a legal tech or AI tool does behind the demo. You get a report on technology, data and vendor, with a recommendation.
- Legal technology selection: from need to decision
HOVR Legal leads the selection of legal tech, CLM or AI tools: requirements, long and short list, pre-demo questions, demo on your contracts, pilot, decision.
- Legal tech implementation: from selection to daily use
HOVR Legal brings the tool you chose into your organisation: owners, templates and rules, migration, data protection, a pilot group, training and handover.
- CLM requirements for a Swedish legal team: before the demo
CLM requirements for a legal team in Sweden, before the demo: workflow, templates, signing, archive, AI, integrations, data protection, exit, price.
- Pilot criteria for legal tech: what to agree before signing
How a legal team runs a pilot of a legal tech or AI tool against fixed criteria set in advance.
- Legal AI in Sweden 2026: tools and framework deals
An overview for lawyers working with Sweden: the AI tools sold to Swedish law firms and legal teams, and what vendors say about sources and data.
- E-signatures under eIDAS, with Sweden as an example
The three eIDAS signature levels and their legal effect across the EU, what evidence rests on, and Sweden as an example: BankID and form requirements.