The Flattened Block - Carrying Metadata Into the Prompt Instead of Just Text

The problem

An HR policy assistant I built for a 2,000-person company retrieved from three sources: the official employee handbook, an archive of individual managers' Slack and email guidance, and a benefits vendor portal export. An employee asked it a simple question, whether unused PTO carries over into next year. The handbook said no. A manager had told their team, two years earlier on Slack, "sure, just ask me," casual guidance that expired the moment the policy changed and was never corrected anywhere the manager or the employee would ever see again.

Here's what made this failure interesting to me, compared to the more obvious version of this problem. Retrieval actually worked. It pulled the correct chunks, both the current handbook language and the stale Slack comment, and every one of those chunks already carried authority_status, sensitivity, and validity-window fields at the data layer. The information needed to resolve the conflict was sitting right there in the metadata. My assembly function threw all of it away anyway, concatenating only the raw chunk text with newlines before handing the block to the model. The model saw two passages that looked equally current and equally official, because nothing told it otherwise.

I measured how often that failure actually happened. Across a 25-question eval set, each built around a deliberate expired-guidance trap, the flattened-text version reproduced the stale Slack answer instead of current policy on 35 percent of questions or worse. That's not a retrieval failure. Retrieval did its job. It's an assembly failure, the step between retrieval and generation quietly discarding information nothing else in the pipeline could recover.

The pattern

The fix keeps everything about retrieval exactly as it was. It replaces only the assembly function, because that's genuinely the only place the failure lives. Instead of flattening each chunk down to plain text, I build one structured record per retrieved chunk: text, source system, authority status, sensitivity, valid-from and valid-until dates, all carried forward together rather than stripped down to a string. Alongside that, the prompt template explicitly walks the model through weighing those fields, instructing it to treat anything expired or non-authoritative as context only, never as the answer, and to name a conflict out loud when two records disagree instead of quietly picking one.

That last instruction turned out to matter more than I expected going in. On the same 25-question eval set, the stale-guidance reproduction rate dropped from 35 percent to under 10 once the model could actually see which record was current and which had expired. But the more useful number, to me, was a second metric I started tracking alongside it: how often the model surfaced the conflict explicitly in its answer, rather than silently resolving it one way or the other. On the flattened-text version that number was close to zero, because nothing in that version was ever built to surface anything. Once the structured metadata was visible, the model started naming the tension directly, telling the employee that current policy says no carryover while flagging that a manager's older comment said otherwise. That's a materially different answer than a confident wrong one, even when the underlying facts are identical.

What I like about this fix is how little it costs to build relative to what it buys. The retrieval architecture doesn't change. The underlying data doesn't change. The only thing that changes is refusing to throw away information at the one step in the pipeline that had it in hand.

Design considerations

The limit on this one didn't show up where I expected. I went looking for edge cases in my own conflict-detection logic and found the real constraint sitting somewhere else entirely: how much a model's reliability degrades as the input it's reading gets longer, even when everything in that input is genuinely relevant to the question. That finding lands directly on this fix's own design. The richer the structured metadata I attach to each chunk, source, authority, sensitivity, two separate timestamps, the more that per-chunk information competes for attention inside a context budget that doesn't degrade gracefully as it fills up. Attach five fields to every one of forty retrieved chunks and I've built a genuinely more informative prompt and a genuinely longer one in the same move, and the second fact starts working against the first once enough chunks accumulate.

That's not a flaw I can fix inside this pattern's own design, and I've stopped trying to. It's a real ceiling on how much metadata is worth attaching per chunk before the volume itself starts costing more attention than the individual fields are worth. In practice that's pushed me toward being more selective about which fields actually earn a place in the structured record for a given use case, rather than defaulting to attaching every field I have just because I can. A validity window matters enormously for a policy assistant where guidance goes stale. It matters much less for a static reference corpus where nothing expires. Knowing which fields actually change the model's answer, for this specific retrieval task, turned out to be a better filter than "attach everything and let the model sort it out."

The other calibration question I keep coming back to is where the line sits between "flag the conflict" and "resolve the conflict." I designed this fix to explicitly not resolve, on purpose, because a policy assistant confidently picking a winner between two disagreeing sources is exactly the failure I was trying to get away from in the first place. But there are situations where a user genuinely wants a single clean answer and doesn't want to be handed two conflicting statements to sort out themselves. I've found that works fine for policy questions, where the cost of an unresolved flag is a slightly longer answer, and works much worse for something like a support macro that needs to fire a single action. The pattern earns its place specifically where the reader benefits from seeing the disagreement, and it's the wrong tool where the system needs to commit to one outcome regardless of what the metadata says.