Patient data and AI: the strictest care

Why patient data is different

Patients tell health workers things they may tell no one else. They do so because they trust that the information will be used to care for them and kept confidential. Health information can reveal illness, pregnancy, mental health, disability, addiction or infection. If it leaks, people can face stigma, discrimination, family conflict or loss of work. Some may stop seeking care altogether.

That trust is the foundation of a health service. As a manager, protecting it is part of your job. AI tools create new ways to break it by accident.

Never put patient data into public chatbots

Free and public AI chatbots are not designed for confidential health records. When you type or paste text into one, you may lose control of where it goes.

• The text is sent to servers you do not control, which may be outside Pakistan.
• Depending on the service and its settings, your input may be stored, reviewed by people or used to train future versions.
• You usually cannot delete it with certainty once it is sent.

This is a form of data leakage. The rule for your office should be absolute: no patient information goes into public AI tools. That includes names, record numbers, test results, scans, photographs, discharge summaries, referral letters and clinical notes. Staff should not upload a patient's report to "help explain it" either.

Watch for shadow AI: staff using personal accounts or phone apps for work because they are quicker. Make the rule clear, explain the reason and make sure people know which tools, if any, your department allows.

Removing names is not always enough

People often assume that deleting the name makes data safe. In health it often does not. A rare condition, an age, a village and a date of admission together may point to one person. This is called re-identification.

True anonymisation is harder than it looks. For AI use in administration, a safer habit is data minimisation: only use the smallest amount of information needed for the task and prefer totals and general descriptions to individual records. Most administrative tasks (a report template, a letter structure, a roster pattern) need no patient information at all.

Consent and confidentiality

Consent in health is a serious matter. A patient who agreed to treatment did not thereby agree to have their records used to test or train an AI system. If anyone proposes to use patient records with an AI tool, even an approved one, ask:

• What exactly will be shared and with whom?
• What is the legal basis and what does your department's guidance say?
• Have patients been told in words they can understand? Have they agreed where agreement is required?
• Who can see the data, where is it stored and for how long?
• How will patients be able to ask questions, object or seek redress if something goes wrong?

If no one can answer these questions clearly, the proposal is not ready. Get advice from whoever handles legal and information matters in your department before going further. Do not rely on a vendor's assurance alone.

Approved systems still need care

Even where a department provides an approved system for handling records, good practice continues:

• Give access only to staff who need it for their role.
• Keep an audit trail of who accessed what and when.
• Train staff on what they may and may not enter.
• Have a clear route for reporting a suspected leak quickly, without fear of blame for reporting.

If something goes wrong

If patient information has been entered into a public AI tool or otherwise exposed, do not hide it. Report it through your department's channels, record what was shared and when and take advice on whether patients need to be told. A culture where staff report mistakes early protects patients far better than one where mistakes are concealed.

Try it

Take a made-up discharge summary that you write yourself (do not use a real one). List every detail in it that could help identify the patient, including combinations of small details. Then rewrite it as the minimum general description you would need to ask an AI for help with, say, a template for discharge letters. Notice how little patient information the task actually needs.

Check yourself

1. Give three reasons why patient data should never be entered into a public chatbot.
2. Why is removing a patient's name not always enough to protect them?
3. Name three questions to ask before patient records are used with any AI tool.

Answers: (1) Any three of: it goes to servers you do not control, it may be stored or reviewed, it may be used for training, it may be outside Pakistan, it cannot reliably be deleted. (2) Other details in combination can re-identify a person. (3) Any three of: what is shared and with whom, the legal basis and departmental guidance, whether patients have been told and agreed, who can see it and where it is stored, how patients can object or seek redress.