alexandre.

← Notes

A legal agent doesn’t deliver text, it delivers Word revisions

A lawyer doesn’t review a rewritten paragraph: they review a diff in their own document. How to produce native, word-level OOXML tracked changes, and why Google Docs suggestions take a detour.

  • legal
  • OOXML
  • Word
  • adoption

An agent reviewing a contract has no trouble producing a good paragraph: better worded, well argued, aligned with the internal playbook. It hands it back as rewritten text. And the lawyer doesn’t use it.

The reason isn’t the quality of the proposal. It’s that the lawyer doesn’t work on text: they work on their document, in track changes mode, where every changed word is visible, attributed, and can be accepted or rejected in one click. A rewritten paragraph forces them to redo the diff by hand, then carry the changes across. At that price, they might as well write it themselves.

Building a contract review agent for an in-house legal department, I reached a clear conclusion: the deliverable isn’t an answer, it’s a .docx with native tracked changes. If you’re plugging an LLM into working documents, the same rule applies.

What the format imposes

Word splits each paragraph into runs, chunks of text with uniform formatting; each run’s w:rPr carries that formatting. A tracked change isn’t a diff layer laid alongside the document: it’s inline markup around those runs. w:ins wraps inserted runs, w:del deleted runs, and inside a w:del the text is carried by w:delText, never by w:t. Every mark carries a w:id, a w:author and a w:date. Schematically, a bold amount being replaced:

<w:del w:id="12" w:author="Agent" w:date="2026-07-02T12:00:00Z">
  <w:r><w:rPr><w:b/></w:rPr><w:delText>10 000 €</w:delText></w:r>
</w:del>
<w:ins w:id="13" w:author="Agent" w:date="2026-07-02T12:00:00Z">
  <w:r><w:rPr><w:b/></w:rPr><w:t>15 000 €</w:t></w:r>
</w:ins>

Three constraints follow:

  • You write inside the runs, not on top of them. A range to replace often spans several runs. Each deleted piece must keep the w:rPr of its original run, otherwise the revision reformats the contract while annotating it.
  • Identifiers have to coexist. A contract you receive already contains the other party’s revisions: new w:id values start from the highest one observed, not from 1.
  • Whitespace is content. xml:space="preserve" as soon as a piece starts or ends with a blank, otherwise spaces vanish at revision boundaries.

Rendering word by word

‘Word by word’ is a business requirement before it’s a technique: a lawyer wants to see which words change. When the code computes the diff, it works on tokens that are either a word or a run of whitespace, never a mix. The property that matters is a reconstruction invariant: kept + deleted gives back the original character for character, kept + inserted gives back the proposal. Without it, a ‘smart’ diff drops a space or an apostrophe and nobody notices.

Two changed zones separated by at most two unchanged words are merged into a single replacement: a revision chopped up word by word reads worse than a block replaced in one piece. Then the writing step splits the range at run boundaries:

# A revision is inline markup inside the runs, not a layer laid alongside.
current_rpr = style_rpr

for segment in mapped_segments:
    if segment.kind == "insert":
        ins_el = revision_element("w:ins", next_id(), author, date)
        ins_el.append(run_with_rpr(current_rpr, "w:t", segment.text))
        elements.append(ins_el)
        continue

    # Split at run boundaries: each piece keeps the w:rPr of its run.
    pieces = pieces_by_run(runs, segment.start, segment.end)

    if segment.kind == "delete":
        del_el = revision_element("w:del", next_id(), author, date)
        for rpr, piece in pieces:
            # Inside a w:del, text goes in w:delText, never in w:t.
            del_el.append(run_with_rpr(rpr, "w:delText", piece))
        elements.append(del_el)
    else:
        # keep: unchanged text stays outside any revision mark.
        for rpr, piece in pieces:
            elements.append(run_with_rpr(rpr, "w:t", piece))

    # Inserted text inherits the style of the text it replaces (after a delete)
    # or of the adjacent kept text (after a keep): never the default style.
    current_rpr = pieces[0][0] if segment.kind == "delete" else pieces[-1][0]

Simplified excerpt. The engine’s entry point is a pure function: no network call, no LLM, input never mutated; with the date fixed, two identical calls produce the same XML. Regressions are therefore testable.

The model proposes, the code marks up

Who produces the word-level changes? There are two ways to ask the model. Either it marks its own deletions and additions, and the code checks that those marks land exactly on the contract passage. Or it returns clean text, and the code computes the word-level changes with a deterministic diff. If marks exist but don’t land on the passage, there’s no fallback to the diff: the code replaces the whole range, one deletion and one insertion.

Which one to ask for by default? I settled it with an internal A/B evaluation, on 29 July 2026, over 2 contracts and 28 judged clauses: clean text 17, model marks 10, one tie. Two contracts aren’t statistical proof, but it’s enough to set a default and keep the rollback behind an environment variable.

One guardrail governs the rest: on the native-revisions path, a revision is only written if the passage the model quotes is found exactly, after normalisation, in the exported document. On that path, an approximate match serves as a diagnostic, never as a range to modify. No source passage, no revision, and nothing reaches the document without the lawyer’s click.

What went wrong

The similarity threshold. The engine refused to mark word by word when the two texts were too dissimilar: below 0.5 similarity, it fell back to a block replacement. The intent was to avoid unreadable diffs. The result was the opposite: a legitimate rewrite at 0.46 was rendered as a block and rejected by a lawyer, who wanted to see the words. I removed the threshold: two texts with no word in common degenerate on their own into a deletion / insertion pair, without blocking the cases in between.

Google Docs suggestions. The Google Docs API can’t create suggestions. documents.batchUpdate writes to the document, it doesn’t propose; you can read existing suggestions (suggestedInsertionIds / suggestedDeletionIds via documents.get), not write them. For a review agent, that’s the worst possible gap between intent and effect: believing you’ve proposed when you’ve actually modified.

The path that works is a detour: export the Doc to DOCX, write the w:ins / w:del into it, upload it back to Drive with conversion. It’s Google’s conversion that turns the revisions into native suggestions. The cost: Google then attributes the suggestions to the OAuth account that did the upload, whatever the w:author in the OOXML. Fine-grained attribution survives in Word, not in Docs. The detour has become a tested invariant: in the pipeline’s contract test, any call to documents.batchUpdate fails.

The wrong suspect. On a real document, the top margin came back squashed, the headers had disappeared, page numbers were reappearing in the body. The revision engine was the obvious culprit. It did have a defect, the header one described below, but once that was fixed, it was in the clear: the tests prove it carries over w:sectPr, margins and references unchanged. The cause was the PDF → DOCX conversion upstream: a PDF has no notion of a header, and the converter rebuilds a plausible geometry. The response: an explicit warning and the instruction to start again from the original .docx.

The traps, in the order they bite

  • Headers and footers. The nastiest one: read header.paragraphs on a section that has none, and python-docx silently creates a word/headerN.xml part and a w:headerReference. Inspecting the document changes its layout. Test is_linked_to_previous before reading.
  • Numbering. Contracts converted from PDF have neither w:pStyle nor w:numPr: there’s no way to number an added clause from Word’s numbering. It has to be inferred from the text: next number = highest number among clauses at the same level + 1, not their count + 1 (sequences have gaps), with inserted clauses numbered bis / ter.
  • Fields. A PAGE field in a footer must stay a field: the regression test checks that no w:instrText appears in the body after writing.
  • Tables. Paragraphs in cells aren’t in the body: unless you iterate over rows then cells, entire clauses stay invisible.
  • Sentence boundaries. When the model proposes a full sentence but quotes only part of the source sentence, the range is extended to sentence boundaries. If the boundary is ambiguous (a full stop followed by a lowercase letter, as in ‘au sens de l’art. précité’), the revision is dropped rather than guessed.

Reusable checklist

  1. The deliverable is the document, not the answer. If users have to carry the changes across by hand, the agent won’t be used.
  2. Write in the format, not alongside it. In Word: inline markup, w:delText in deletions, one w:rPr per piece.
  3. The model proposes, the code marks up. Its output is checked against the real document before being written.
  4. A diff that rebuilds the original character for character, and a test that proves it.
  5. An exact anchor or nothing on the path that writes into the client’s document.
  6. A test proves nothing else moves: layout, headers, styles and fields come out identical.
  7. Check that an API does what its name promises: batchUpdate modifies, it doesn’t suggest.
  8. Name what is lost (author attribution, layout): an honest warning beats a half-promise.

The deliverable’s format is an architecture decision

Writing in a format users already open isn’t a finishing touch you add after the model: it’s what decides whether the agent gets used, and it has to be designed in from the start.

If you have a well-scoped use case and users ready to use it, that’s what an Agent Sprint is for: I first agree the deliverable’s format with them, then build the agent, its evaluation set and its guardrails, all the way to production in four to six weeks.