Why an LLM wiki beats RAG for law that amends itself
Retrieval assumes your documents sit still. Tax law does not. The structural argument behind CA_wikiIndia.
Almost every “chat with your documents” build reaches for the same architecture: chunk the corpus, embed the chunks, retrieve the nearest neighbours at query time, stuff them into context. For a static corpus this works well and I have no argument with it.
Indian tax law is not a static corpus, and the ways it isn’t are exactly the ways that break retrieval.
The three problems
1. Amendment. The text changes but the identity doesn’t.
A section of the Income Tax Act is not a document. It is a running position with a history. It is inserted, substituted, amended with effect from a date that may be retrospective, and sometimes omitted while its number is retained. The question “what does section X say” is incomplete — the real question is “what did section X say for assessment year Y.”
Chunking flattens this. You end up with several chunks that all look like section X, differing in ways an embedding does not capture well, because “with effect from 1 April 2021” and “with effect from 1 April 2023” are nearly identical strings that point at completely different legal realities. Retrieval will happily hand back the wrong vintage with total confidence.
2. Cross-reference. Provisions do not stand alone.
A section carries provisos, explanations, and exceptions that may live in a different section, a schedule, or a rule. Answering correctly frequently means assembling four or five fragments that are textually dissimilar and only related structurally.
Similarity search finds text that resembles the query. It does not follow references. The chunk that says “notwithstanding anything contained in section 10” is precisely the chunk that determines the answer, and it is precisely the chunk that does not resemble a question about section 10.
3. Absence. What the law does not say is often the answer.
Retrieval always returns its top k. If nothing relevant exists, it returns the nearest irrelevant thing, and the model — handed plausible context — will use it. In a legal setting, “there is no provision covering this” is a correct and important answer, and the architecture has no way to express it.
The alternative
Rather than chunking the corpus, build it as a structured wiki: one durable page per provision, holding the current text, the amendment history with effective dates, resolved cross-references as explicit links, and the notes a professional would keep.
Two properties follow, and they are the whole argument:
The unit of retrieval becomes the unit of law. You fetch section X — the entire, coherent, current section with its history — instead of three fragments that happen to score well. There is no reassembly step, so there is no reassembly error.
Traversal becomes possible. Because references are links rather than prose, “what modifies this” is a graph walk, not a hopeful second search. The system can follow the same path a professional follows.
What it costs
I want to be even-handed, because this is a real trade and not a free win.
- Structuring is expensive. Chunking is automatic; a wiki has to be built, and rebuilt as law changes. That is genuine ongoing work.
- It generalises badly. This design is tuned to a corpus that is hierarchical, cross-referential and versioned. Point it at a pile of unstructured PDFs and RAG will beat it comfortably.
- The graph can be wrong. A missed cross-reference is a silent failure, and it is a worse silent failure than a bad retrieval because it looks authoritative.
The general principle
The useful version of this, stripped of tax:
Match the retrieval unit to the domain’s natural unit of meaning.
For a support corpus that unit is roughly a paragraph, and chunking is fine. For legislation it is a provision with its history and its references — and if you chunk below that line, you are shredding the structure that made the corpus answerable in the first place.
Before reaching for embeddings, it is worth asking a plain question about your documents: do they amend themselves, and do they refer to each other? If both answers are yes, chunking is throwing away the two properties you most need.
Built out in CA_wikiIndia. Educational only — not tax advice, and not a substitute for a professional reading the section yourself.