Building the prompt and citing sources

Your retrieval is good: across 30 reference questions, the right passage appears 28 times in the top five results. Yet users report inaccurate answers. You inspect one case: the correct passage was in fourth position, a neighboring passage about a different site was in first, and the model answered from the first one while citing the fourth. The citation was wrong, and therefore unverifiable, and therefore worse than none: it makes an erroneous answer look like a sourced one.

Prompt assembly is not cosmetic. It determines what the model sees first, what it is allowed to do when information is missing, and whether its citation can be verified by a program or is merely plausible to a human. Three decisions structure everything else: the order of the passages, the citation format, and what to do when the information is absent.

Passage order is not neutral

Language models make better use of information at the beginning and end of their context than information in the middle. The effect is measurable and grows as the context gets longer. If you place ten passages in descending order of relevance, passages 5 through 8 are the ones the model is most likely to ignore.