Buying AI systems: questions to ask before you sign

Why buying AI is different

Procurement of AI looks like buying any other software, but several things make it harder. The product may change after you buy it, as the supplier updates the underlying model. Its performance depends on your data, not just on the product. It may send your information to servers you do not control. Once staff and processes depend on it, leaving can be expensive. A contract that ignores these points can lock a department into poor value for years.

Always follow procurement rules that apply to your organisation and involve your procurement, legal and IT colleagues early. This module does not replace those rules. It lists the questions to ask within them.

Before you go to market

• Be clear about the problem. Describe the task and the outcome you need, not a particular product. This keeps competition open and stops the specification being written around one vendor.
• Decide what success looks like. Agree measures in advance: accuracy on your material, time saved, error rates and user satisfaction (module 8).
• Classify the risk. A high-risk use (module 3) needs stronger requirements in the tender and the contract.
• Check your data. If your data is not ready, the system will not work, whatever the supplier promises (module 5).

Questions about data

• Where will our data be stored and processed? In which country and by which companies, including subcontractors? Data sovereignty concerns are real for government information.
• Will our data be used to train or improve the supplier's models? The default answer should be no, in writing.
• Who can access our data? Including supplier staff, for support or monitoring.
• How is personal data protected? Ask about encryption, access controls and how the system meets data protection obligations. Check what law and department instructions apply.
• What happens to our data when the contract ends? It should be returned in a usable format and then deleted, with confirmation.

Questions about lock-in

Vendor lock-in happens when leaving a supplier becomes so costly or difficult that you stay even when the service is poor. Ask:

• Can we export our data, configurations and records in open, standard formats?
• Who owns anything built for us, such as custom prompts, workflows or trained components?
• How easily could we switch to another supplier or model?
• What are the costs of exit? Are they stated in the contract?
• Is the pricing fixed? Can it rise sharply with usage or at renewal?

Questions about the system itself

• Which model does it use and who controls updates? Ask for notice before significant changes and the right to test them.
• How does it perform on material like ours? Ask for evidence from comparable settings, not marketing figures.
• Can it explain its outputs? For decisions about citizens, explainability and an audit trail are essential.
• How does it handle Urdu and regional languages? Ask for testing, not assurances.
• What security testing has been done? Ask about known weaknesses and how incidents are reported.

Test before you buy

Never commit to a large purchase on the strength of a demonstration. Where procurement rules allow, require a structured test on your own material:

• use a realistic sample of your documents or cases, with sensitive data removed or protected
• compare the system against current practice on agreed measures
• include difficult cases: poor scans, mixed languages, unusual requests
• have staff who will use the system take part in the test
• record failures as carefully as successes

A short, honest test is far cheaper than a failed rollout.

Contract terms to look for

Ask your legal and procurement colleagues to make sure the contract covers:

• performance standards and what happens if they are not met
• notice and testing rights before model changes
• data location, use, return and deletion
• security incident reporting within a set time
• your right to audit, suspend and terminate
• clear exit arrangements and costs

Try it

Find a public tender notice or published request for proposals for any software or digital service. Using the questions in this module, list three things it addresses well and three AI-specific questions it would need to add.

Check yourself

1. Give three questions you should ask about data before buying an AI system.
2. What is vendor lock-in and how can you reduce it?
3. Why should you test on your own material before buying?

Answers: (1) Any three of: where data is stored and processed, whether it will train the supplier's models, who can access it, how personal data is protected, what happens to it at the end of the contract. (2) Being unable to leave a supplier because leaving is too costly or difficult; reduce it with open export formats, clear ownership, stated exit costs and fixed pricing terms. (3) Performance depends on your data and languages; demonstrations on other material tell you little.