The paragraph that doesn't exist

IPSAS, Unfiltered

Ask a general-purpose AI tool a public-sector accounting question and you will get a confident, fluent, well-formatted answer. It will usually name a standard and a paragraph number. The formatting alone makes it look authoritative.

Then you go to check the paragraph, and it is not there.

Not “it says something a little different.” It does not exist. The standard has fewer paragraphs than the number quoted, or that paragraph is about something else, or the standard it cites was withdrawn three years ago. None of this is malice. The tool was trained on a world where the dominant accounting literature is IFRS, the private-sector standards. It has read a great deal of IFRS and comparatively little IPSAS. When it reaches the edge of what it knows about the public sector, it does what these tools do: it produces something that looks right.

In private-sector accounting that is an annoyance. In public-sector accounting it is a problem with your name on it, because the citation is the part the auditor checks.

Here is how the failure actually lands. A project accountant has an unusual transaction: a grant that closed oddly, an asset that arrived without paperwork. They ask the tool. They get a clean answer with a paragraph reference, and they paste it into a working paper, because a referenced answer is exactly what a working paper is supposed to contain. Months later the auditor pulls that paper, follows the reference, and finds nothing. Now the question is no longer whether the treatment was right. It is why the file cites authority that does not exist, and what else in the file was built the same way.

The point is that the citation was never decoration. In this work the reference is the defence. It is the difference between “we judged it this way” and “the standard requires it this way, and here is where.” A wrong paragraph does not weaken the answer a little. It removes the floor from under it.

So when I built these tools, the first rule was that they cannot do this. The answers are drawn only from the official text actually retrieved for the question, never from the model’s memory of what a standard probably says. Every citation is re-read against its source before it is shown, every time, by a separate check that cannot see how the answer was written. Where the tool is not sure, it says so. It does not reach for something that simply looks right.

That last rule runs against the grain of what these tools are built to do. A general-purpose model is rewarded for sounding fluent, and a fluent wrong answer arrives in exactly the same clean format as the right one. The invented paragraph does not hedge or stammer. The only defence is a tool built to say “I am not sure” out loud, and built to be checked at its source rather than one merely trusted at its surface.

The accounting layer is the one I can put a hard number on, and I did: I went through every standards reference in the training content one at a time. I will come back in a few weeks to what that number was, and why I published the unflattering half of it first. The discipline behind it is the same one in this essay. A citation is shown only after it has been read against the source. Nothing is trusted because it looks right.

That is the line I want to hold, in the tools and in this publication both. A tool that always has a paragraph number for you is not the reassuring one. The reassuring one is the tool that will tell you, plainly, when the paragraph is not there.