A fluent chatbot can write a policy, draft a questionnaire answer, and summarise a framework requirement in seconds. The question is not whether it can produce the output — it is whether that output is trustworthy enough to submit to an auditor or regulator. For general AI, the answer is: only if you are prepared to verify every claim.
What a General Chatbot Handles Fine
General AI chatbots are good at several things that look like compliance work. They produce fluent prose in the correct tone. They can summarise a SOC 2 criteria description or rewrite a policy paragraph in plainer language. For internal drafts where accuracy is less critical — a first-pass thought document, a brainstorming list — a chatbot saves time.
The problem starts when the task moves from "draft something that reads well" to "produce something that can be defended." That shift happens the moment the output is shared outside the team — with an auditor, a customer, a regulator.
Where It Breaks
The failure modes of general AI in compliance are not theoretical. They have been documented in US courts, where AI-generated legal filings have hallucinated case citations, invented judges, and fabricated quotes attributed to real people. As of April 2026, over 1,300 instances of AI-hallucinated legal citations have been recorded across US courts, with consequences ranging from fines to suspensions.
The same failure mode applies to compliance work. A general chatbot presented with a SOC 2 questionnaire or ISO 27001 gap assessment will:
| Failure mode | What happens | Impact |
|---|---|---|
| Invented control IDs | The chatbot writes "SOC 2 CC 7.4" — a control that does not exist | Auditor flags the error; credibility lost |
| Fabricated policy clauses | The chatbot quotes a policy provision that was never written | The clause cannot be produced; finding raised |
| Imagined evidence | The chatbot claims an access review was conducted in January | No evidence exists; gap identified |
| Plausible but wrong | The answer sounds correct but contradicts the organisation's actual control design | Misleads the reviewer; rework needed |
Each failure shares a root cause: the chatbot does not know what is in your workspace. It knows what is in its training data, which is a statistical approximation of the internet, not a representation of your policies, controls, and evidence.
The Moment Work Gets Specific
Compliance work becomes specific at three thresholds that general AI cannot cross without a workspace.
First threshold — framework knowledge. A chatbot knows the general structure of SOC 2 or ISO 27001 from its training data. It does not know which version of the framework your audit follows, which criteria the AICPA updated this year, or how your organisation interpreted a specific control requirement.
Second threshold — organisational context. A chatbot does not know your asset register, your vendor list, your risk register, or your policy repository. It cannot tell you whether a control listed in your scope statement is substantiated by evidence in your evidence folder.
Third threshold — audit defensibility. A chatbot cannot produce output that traces each claim to a source. It generates text; it does not generate citations. When an auditor asks "where is this control documented in your policies?" the chatbot's output provides no answer.
What "Grounded" Adds
A grounded compliance tool changes the relationship between the AI and the output. Instead of generating text from training data alone, it reads the organisation's actual documents — policies, evidence artifacts, risk registers, control mappings — before producing anything.
Grounding means:
- Every claim must be supported by a source — if the source does not exist in the workspace, the tool marks it as unsupported rather than inventing one
- Citations are part of the output — each paragraph, number, or statement traces to a document or evidence item
- Human approval is required — nothing leaves the workspace until a reviewer accepts or rejects it
The difference is not in fluency — a grounded tool may produce the same quality of prose as a general chatbot. The difference is in traceability. Auditors do not ask "does this read well?" They ask "where is the evidence for this claim?" A grounded tool answers that question for every claim it produces.
FAQ
Can't I just fact-check what ChatGPT writes?
You can, but fact-checking AI-generated compliance output costs more time than writing from scratch. Every control ID must be verified against the actual framework. Every policy reference must be checked against the current version. Every claimed evidence artifact must be located. The time saved in generation is spent on verification — and if the chatbot fabricated something that looks plausible, it may not be caught at all.
What if I upload my documents to ChatGPT?
Uploading documents to a general chatbot improves the context but does not add grounding. The chatbot still has no structural constraint that requires citations. It may reference your documents in its output, or it may ignore them and generate from training data. There is no enforcement mechanism.
Is there a compliance domain where general AI works well?
Yes — initial drafting of internal documents that will be thoroughly reviewed and edited. Policy outlines, meeting notes, internal summaries. The risk is low because the output is reviewed by someone who knows the subject. The risk becomes unacceptable when the output is shared externally or used as the basis for an audit submission.
How do I evaluate whether a tool is grounded?
Look for three signals: does it require citations on every claim? Does it mark unsupported answers rather than inventing them? Is there a human review step between generation and output? If the answer to any of these is no, the tool is not grounded for compliance work.
A general chatbot is a fast writer. A grounded compliance tool is a verifiable one. Compass by Truvara reads your workspace, drafts cited output from your actual evidence, and marks "Not Provided / no source found" when it cannot support a claim — every output passes through a human review gate before it becomes official. It competes on trustworthiness for trust work, not general versatility. Watch Compass work.