Loop engineering for document agents

A long-document agent does not succeed merely because it can call retrieval repeatedly. The critical question is whether each call changes the state of the investigation. If the agent retrieves a covenant definition, reasons about it, then begins the next turn without retaining the finding, it has not advanced. It has performed an expensive reread.

This distinction matters in credit documentation. A question about restricted payments may require the agent to move among the covenant, defined terms, builder-basket mechanics, permitted investment provisions, ratio definitions, amendment history and guarantee structure. The relevant evidence is distributed. Some provisions qualify others. A broad keyword search may find the obvious clause while missing the definition that changes its operation.

The loop therefore needs an explicit design. It must decide what evidence to retrieve, what state to carry forward, how to select the next question, when to stop and how to preserve unfinished work. These are system decisions, not prompt-writing details.

What is the loop actually trying to complete?

The loop should operate against an investigation plan, not the vague instruction to “analyse the document.” The plan converts the assignment into claims that can be supported, rejected or left unresolved.

For a covenant analysis, the initial plan might include:

• Identify the operative covenant and relevant exceptions.
• Resolve every defined term used in the applicable test.
• Trace amounts through baskets, ratios and reclassification provisions.
• Check whether amendments or supplements modify the language.
• Record ambiguities, conflicts and missing documents.
• Produce a conclusion linked to source evidence.

Each item becomes a work unit with a status such as open, in progress, supported, contradicted or blocked. That status gives the loop direction. The next retrieval is chosen because it advances an open work unit, not simply because another iteration is available.

This also makes incompleteness visible. “No prohibition found” is not equivalent to “the transaction is permitted.” The first statement may only mean that the retrieval path has not yet reached the governing exception, condition or cross-reference.

How granular should retrieval be?

Retrieval granularity should match the reasoning task. A clause-sized passage is often enough to identify a candidate provision but too small to interpret it. A full agreement preserves context, and many will now fit in a single window, but placing everything in context makes evidence selection less reliable as competing passages accumulate.

A useful design supports several retrieval levels:

The first pass should usually construct a map: document types, section hierarchy, defined-term locations, schedules, exhibits and amendment relationships. Later passes can retrieve bounded passages from that map.

Retrieval should also follow links in the drafting. If a restricted payment exception depends on an Available Amount, the agent should queue the Available Amount definition and every component necessary to calculate or qualify it. If that definition imports a consolidated-net-income concept, the dependency becomes another work item. The loop is traversing a graph, even if the documents present the provisions linearly.

Each retrieved passage should include stable provenance: document identity, heading or clause label, page where available, and enough surrounding text to verify the interpretation. A conclusion without retrievable provenance is not a completed finding.

What belongs in the working ledger?

A transcript is not a working ledger. Transcripts contain repeated instructions, abandoned hypotheses and lengthy passages. As they grow, important evidence becomes harder to distinguish from conversational residue. They may also be truncated or compressed.

The ledger should be structured and persistent. At minimum, it should record:

• The proposition under review.
• The current status of that proposition.
• Supporting and contrary evidence.
• Source locations and document versions.
• Definitions and cross-references already resolved.
• Queries and sections already examined.
• Unresolved dependencies.
• Conflicts or interpretive uncertainty.
• The next proposed action.

Suppose the agent finds a general restriction, an exception based on a basket and a condition tied to no continuing default. Those are separate ledger entries. The basket’s capacity may remain unresolved because the necessary financial inputs are unavailable. The no-default condition may require evidence outside the agreement. The correct state is not a single confident conclusion. It is a set of supported legal mechanics with clearly identified factual dependencies.

The ledger also prevents circular searching. Before executing a query, the agent can compare it with prior searches, retrieved provisions and unresolved issues. Reopening the same section is justified only if a new dependency or contradiction requires reconsideration.

How should the agent choose the next step?

The next-step policy should favour information gain. A simple priority order is:

1. Resolve dependencies that block several conclusions.
2. Investigate evidence that could contradict the current view.
3. Follow explicit definitions and cross-references.
4. Fill provenance gaps for otherwise supported findings.
5. Broaden retrieval only when targeted paths are exhausted.

This policy is better than always following the latest passage. Document analysis creates tempting side paths. A definition may refer to another concept that is technically related but immaterial to the assigned question. The investigation plan and ledger provide the test: will resolving this item change a required conclusion or its confidence?

The loop should periodically synthesise rather than retrieve. After several evidence-gathering steps, it should update the claim statuses, merge duplicates and identify contradictions. Without synthesis, retrieval becomes accumulation. The agent may possess the relevant provisions yet fail to assemble them into an answer.

A compact loop can be expressed as:

load investigation plan and ledger
while stopping condition is not met:
    select highest-priority unresolved item
    retrieve the smallest sufficient evidence bundle
    extract findings and provenance
    update claims, dependencies and conflicts
    propose the next action
persist checkpoint
render completed and unresolved results

The persistence step should occur throughout the loop, not only after it.

When should the loop stop?

A fixed iteration count is easy to implement but poorly aligned with open-ended discovery. Ten rounds may be excessive for a direct definition question and inadequate for tracing a covenant across an agreement, amendments and incorporated schedules.

A better primary condition is saturation: stop when nothing materially new is found for K consecutive rounds. This depends on the retrieval tools reporting completeness. A capped tool that silently returns the same top results on every call produces no new evidence by construction, drives the counter straight to K and yields a confident stop that is really an interface failure. Saturation is only meaningful once the tools can tell the loop whether more remains. “Materially new” needs an operational definition. It may include:

• A new provision relevant to an open claim.
• A resolved definition or cross-reference.
• Evidence that changes a claim’s status.
• A new contradiction or qualification.
• A newly identified missing document or factual dependency.

Rephrasing an existing finding does not reset the counter. Nor does retrieving a duplicate passage from another chunk.

K should be large enough to tolerate an unproductive search path without allowing indefinite exploration. The suitable value depends on document structure and retrieval quality, so it is a configurable system parameter rather than a universal constant.

Saturation should coexist with safety limits: maximum runtime, retrieval volume, cost or tool failures. Those limits answer “when must execution cease?” Saturation answers “when is further searching unlikely to advance the investigation?” The termination record should distinguish the two.

What must survive an early termination?

Every loop ends eventually, and some end before the investigation is complete. A timeout, unavailable exhibit, malformed file or retrieval failure must not erase work already performed.

Intermediate results should be written as recoverable checkpoints after each meaningful state change. A checkpoint should contain the ledger, evidence references, search history, unresolved queue, stopping counters and termination reason. Large source passages can remain in an evidence store, with the ledger holding stable references.

The output produced after early termination should separate three categories:

• Findings supported by cited evidence.
• Provisional conclusions dependent on unresolved items.
• Questions that were not reached or could not be resolved.

This separation is particularly important for professional review. A lawyer or analyst can use a partial analysis if its boundaries are explicit. They cannot safely use a polished answer that conceals incomplete retrieval.

Checkpointing also permits resumption. A restarted agent can load the last valid ledger, confirm that source versions have not changed and continue from the unresolved queue. It need not reconstruct the investigation from a transcript or repeat every search.

What does a production-ready loop look like?

A production loop combines bounded retrieval, an explicit claim plan, persistent structured state, evidence-linked synthesis and saturation-based stopping. It treats hard iteration limits as guardrails. It treats intermediate findings as durable work product.

The central design test is simple: after any iteration, can the system explain what it learned, which source supports it, what remains unresolved and why the next action is worth taking? If not, the loop is generating activity rather than progressing an analysis.

For credit-document work, that distinction determines whether the agent can follow drafting dependencies across a long instrument without repeatedly losing its place. The model matters, but the loop decides whether its reasoning compounds from one step to the next.