RAG solves only one part of the problem
Retrieval-augmented generation gives a model relevant documents before it answers. That helps when knowledge changes quickly or should not be packed into a prompt. RAG does not turn documents into truth, and it does not prevent a query from crossing a data boundary. The hard part is still the boundary design.
Set four boundaries first
- Which data may be indexed, and who may discover it?
- How much time and compute may one request consume?
- What happens when the evidence is not strong enough?
- How can a document be replaced or removed without its old copy resurfacing?
Put the cheap, predictable work at the edge
An edge Worker is a good place for predictable steps: checking request size, normalising locale, filtering access, reading a cache and assigning a trace id. Run them before the model call. They reduce wasted work and create one stable observation point even when the AI provider changes.
Keep the index small on purpose
There is no need to index everything on day one. Choose documents with a clear owner and lifecycle, keep effective dates as metadata and test retrieval with questions real users ask. A small, maintained corpus is usually more trustworthy than a large collection no one owns.
- Partition documents by workspace and access scope.
- Keep source, version and effective time with every chunk.
- Check retrieval for permission scope, not only similarity.
- Have a re-index path when a document is replaced or deleted.
Cost is a product property
On a free plan, every unnecessary call reduces room for real users. Measure and cap the basics early: input length, retrieved chunks, retries and waiting time. When a request crosses a limit, the interface should ask for a narrower question instead of silently sending more calls.
A safe answer can be ‘I do not know’
If retrieved passages are not relevant enough, ask for clarification or say that the corpus has no answer. Attach sources to the response, separate inference from quotation and never let the model invent an internal link. Refusing at the right time is part of the UX, not a failure to hide.
Trustworthy RAG does not promise to answer everything; it knows what it may say and when to stop.
Conclusion
A sensible edge-first architecture can start small: authorise and validate in the Worker, give data a lifecycle in storage, retrieve within limits and call a model only when evidence is sufficient. That keeps cost, privacy and provider choice in the team’s hands as the system grows.