What AI is and what it is not

Why start here

Most mistakes officers make with AI come from one misunderstanding: treating it as a source of facts rather than a producer of plausible text. Fix that idea first and the rest of this pathway is straightforward.

The tool you will actually use

When people in offices talk about AI today they usually mean a generative AI assistant: ChatGPT, Gemini, Copilot, Claude, Meta AI and similar. Underneath each is a large language model (LLM): a model trained on enormous amounts of text to predict which words should come next.

That one sentence explains most of its behaviour.

• It writes fluently, because it has seen a vast amount of fluent writing.
• It adapts to tone and format, because it has seen letters, reports, notices and minutes of every kind.
• It does not look facts up unless the tool is connected to search or to documents you provide. By default it produces what is likely, not what is verified.

What it does well

• Producing first drafts of routine documents in seconds
• Changing form: turning notes into a letter, a letter into a summary, English into Urdu, jargon into plain language
• Condensing: pulling the main points out of long text you give it
• Structuring: turning messy information into tables, lists and headings
• Suggesting: options, questions you may have missed, counter-arguments

Where it fails

• Hallucination. It can invent a rule, a notification number, a court judgment or a statistic and present it with complete confidence. This is the single most important risk for officers.
• Out-of-date knowledge. A model learns from data up to a certain date. Unless connected to current sources, it may not know about recent notifications, policies or appointments.
• Bias. It can reflect unfair patterns in its training data, for example assumptions about gender, region or class.
• No accountability. It cannot sign, explain itself to a court or answer to a citizen. You can. See accountability.

A useful way to think about it

Treat an AI assistant like a capable new assistant who joined your section yesterday: quick, articulate and well read, but unfamiliar with your department's rules and history and never willing to say "I do not know". You would let such a person prepare drafts. You would not let them send anything out without reading it.

AI and automation are not the same

Not every computer system is AI. Most government IT systems follow fixed rules written by people: a payroll system, an online form, a tracking database. This is automation. AI is different because it learns patterns from data and its outputs can vary. That flexibility is what makes it useful and what makes checking essential.

Try it

Using any AI assistant your department allows and only public information:

1. Ask it to explain the difference between a notification and an office memorandum in Pakistan's government practice.
2. Ask it for the number and date of a specific notification you know well.
3. Compare both answers with what you know.

Notice how the explanation is likely to be reasonable, while specific numbers and dates are where errors appear.

Check yourself

1. Why can an AI assistant write a convincing letter about a rule that does not exist?
2. Name two tasks AI does well and one it does badly.
3. Who is responsible for a letter drafted by AI and signed by you?

Answers: (1) It predicts plausible text rather than looking facts up, so it can produce convincing content that is false. (2) Well: first drafts, summaries, restructuring, translation for gist. Badly: quoting exact rules, numbers and dates without a source. (3) You are.