Which workflow fits a private, changing local corpus?#

Reviewed 29 September 2026. This report compares architectures, not measured implementations. “Private” is a policy decision about where source files, derived indexes, retrieved passages, prompts, and outputs may be stored or processed.

Current finding#

The first decision is which document content, if any, may cross the organization’s boundary. A hosted file-search service needs uploaded files and maintains a searchable index; keeping retrieval local avoids that upload, but a hosted generator still receives the passages placed in its prompt. [1] [2] If no document content may reach an external service, both retrieval and generation have to run inside the approved environment; locally served models make that architecture possible, although this report has not compared their answer quality with hosted frontier models. [3] [4]

For a changing corpus, the second decision is how quickly edits, deletions, and permission changes must take effect. OWASP advises checking access at retrieval time and removing derived chunks, embeddings, and caches when a source is deleted or loses permission. [3] OpenAI cautions that removal from its vector store is eventually consistent, so search may briefly return content from a removed file. [2] These properties matter more than whether the retriever uses vectors or keywords when immediate revocation is required. [2] [3]

When the whole corpus may be hosted#

Managed retrieval is the simplest starting workflow if uploading the documents, provider application-state retention, and the provider’s deletion timing are acceptable. OpenAI File Search, as one concrete example, uploads files to vector stores and handles chunking, embedding, indexing, and retrieval; its search combines semantic and keyword methods. [1] [2] New and changed files still need an ingestion process, and indexing operations are asynchronous. [2] OpenAI’s data-control table lists files and vector stores as application state retained until deletion and as ineligible for Zero Data Retention (ZDR). [5]

This workflow therefore suits a corpus whose owners permit a provider copy and can tolerate the interval between a source change and a completed index update. [2] [5] It is a poor fit when a revoked document must stop appearing in search immediately, because OpenAI explicitly describes removal as eventually consistent. [2] A separate source of truth should track file versions and permissions so that an answer can be withheld while a provider index catches up; this is an architectural inference from that consistency limit and OWASP’s retrieval-time access-control guidance. [2] [3]

When the index must stay local but selected passages may leave#

Keep parsing, search, and any embeddings inside the approved environment; check each user’s current access before sending a small set of retrieved passages and the question to a hosted model. OWASP warns that embeddings can leak source information and recommends treating them as sensitive data, while its access-control guidance says permission checks belong at retrieval time. [3] SQLite FTS5 demonstrates a local keyword index whose updates and deletions can be synchronized with source-table changes by triggers. [6] A local embedding model can add semantic retrieval without sending the source text to an embedding API; llama.cpp, for example, offers an embeddings endpoint as well as local model serving. [4]

This split preserves local control over the corpus and index, but it does not keep every document passage local: selected text and the user query are processed by the hosted generator. [1] [5] If the boundary allows transient processing but forbids provider retention, verify the exact API endpoint and account controls. OpenAI says ordinary API requests can generate abuse-monitoring logs retained for up to 30 days, while ZDR requires approval and has endpoint-specific limits; files and vector stores remain ineligible. [5] Setting store: false on a response is therefore not equivalent to having ZDR for the whole workflow. [5]

For frequent changes, this is the most controllable hosted-model option: a local manifest can detect changed or removed files, replace their indexed passages, and check current permissions before each retrieval. This recommendation follows from SQLite’s explicit index-maintenance model and OWASP’s requirements to propagate deletions and permission changes through derived data. [3] [6] The application must also invalidate cached answers and record which source version each citation refers to. [3]

When no document content may leave#

Run parsing, indexing, embedding, retrieval, and generation inside the approved environment. llama.cpp provides local inference and embedding APIs that can serve as components of such a pipeline. [4] A hosted frontier model cannot be used for answers containing private passages under a strict no-egress rule, because supplying those passages to its API crosses that rule by definition; this is an inference about the stated boundary, not a claim about model capability. [1] [4]

Local execution does not remove the need to control access, deletion, or prompt injection. OWASP advises preserving permissions on chunks, enforcing them before content reaches the model, and deleting derived data when source access changes. [3] It also treats retrieved text as an attack surface for instructions embedded in documents. [3] A local pipeline offers control over those steps, but the organization must operate and evaluate them itself. [3] [4]

A practical comparison before choosing#

For a small corpus that fits the prompt and may be sent to a provider, a long-context request is a useful baseline because it avoids index synchronization. Google recommends trying full context first for some Gemini tasks, while noting input cost as a constraint. [7] Each request still sends the included content to the model, so it does not satisfy a local-content boundary. [5] [7]

For a larger or frequently queried corpus, compare local keyword retrieval with a hybrid of keyword and semantic search using real questions and recently edited documents. SQLite supplies the local keyword baseline; Anthropic reports lower retrieval failure in its own experiments when contextual embeddings and keyword search were combined, but that result does not establish the same gain for this corpus. [6] [8] The evaluation should include newly added material, changed permissions, deleted documents, and citation checks, because those cases exercise the privacy and freshness requirements that an average answer-quality score can miss. [2] [3]

References#

  1. OpenAI: File search
  2. OpenAI: Retrieval and vector-store operations
  3. OWASP: RAG Security Cheat Sheet
  4. llama.cpp: HTTP server
  5. OpenAI: Data controls in the OpenAI platform
  6. SQLite: FTS5 Extension
  7. Google Cloud: Long context
  8. Anthropic: Contextual Retrieval