The Trump administration made that clear: federal agencies must begin rolling out AI systems as soon as possible. In April, the Office of Management and Buldge (OMB) issued new guidance requiring non-state security agencies to identify AI use cases and begin implementing them. National security agencies could continue.
But the familiar fog surrounds decision makers: What is legal? Do agency general advice confidently illuminate AI systems that interact with sensitive data against Greenlight AI systems, especially when it concerns us?
The answer is yes. A method called the searched high generation (RAG) provides a simple but powerful solution for federal agencies. RAG allows AI systems to interact with sensitive data without actually storing or modifying the underlying AI model, avoiding the legal grey zone that often causes hesitation. This means that agencies can start using AI today. You use tools you already have and data that are already legally retained. And in a world that accelerates global competition for AI leadership, they must. The enemy is not waiting for it to become completely clear – they are moving fast. US agents can’t afford to delay.
Many federal agencies are unsure whether internal or sensitive data can be used safely to improve AI systems. Confusion focuses on two general techniques. Training is teaching a new model by feeding large amounts of data from scratch. Additionally, fine tuning is used to adapt existing models using agency-specific data to better perform specialized tasks.
Training and fine-tuning raises open legal questions. Will improving AI models using sensitive internal data trigger a new obligation under the 1974 Privacy Act? Will it count as a new “collection” under EO 12333, or will it involve recent National Security Council (NSC) guidance on the “modified” model? These ambiguities naturally created risk aversion, but they should not paralyze the institutions all at once.
While training and fine-tuning models may require more legal clarity, Rag offers a clean, compliant alternative. This allows agencies to leverage the power of AI without being entangled in uncertain legal spheres. And the US can’t afford to be late as the enemy moves fast. RAG provides a way to build AI systems today, using data agencies that already legally own within the existing legal framework.
This is not a theoretical workaround. This is a practical and technically mature interim solution. General’s advice and the highest AI officers should act on this now. If your agent can store and query the database, you can use RAG. If a human analyst can read the document, then an AI model can also be used.
How do the rugs work?
In short, RAG systems connect AI models to the agency’s database. The model can query and search. When a user asks a question to the model, the RAG system searches the attached database and returns the material based on the user’s query. These documents are temporarily provided to the model as additional context. The model uses only this input to generate a response and then discards it. In other words, agencies have strict control over their data and can revoke access to their data at any time.
The internal parameters of the model (known as weights) have not been changed. No changes, training or tweaks.
Agents can run RAG systems for their own use with their own infrastructure. There is no need to connect to the internet or share it with anyone other than your agency. This allows bureaucrats to have strict control over system access, network configuration, and physical hardware security. The underlying vector databases that store these documents can be protected using encryption, access control, and audit logs.
Consider two examples. Federal records officers at the Department of Veterans Affairs are to use the RAG system to answer questions about eligibility criteria. The system takes the most relevant excerpts from the internal policy manual and archived guidance, and feeds them into the model to generate a complete and accurate summary.
Or, using intelligence analysts with clearance, accessing a classified document database is very large and complex, and cannot be analyzed quickly by humans. When investigating a potential threat, analysts use the RAG system to query the classified database. The system only obtains relevant and certified documents, and the AI model synthesizes them. This covers insights from sources that are too vast to be manually reviewed. This model uses only those specific documents for responses and discards them after the session – no memory, storage, no learning. Analysts can see the quote, validate the source, and trust the output.
Importantly, this does not require the need to connect the classified data to an open internet or untrusted tool. ICs can be run in a completely local or secure environment, with the choice of a reliable foundation model. These types of tools have already gained traction. Meta, for example, created Llama that can be used by US national security agencies and defense contractors, and companies such as Scale AI and Lockheed Martin have built national security tools on top of the base. National security agencies can similarly deploy trustworthy basic models in air-rated classified environments and use lags to extract insights from sensitive data without tweaking or modifying the model itself.
When AI models access information in the US, they do not have a feature that is different from cleared human analysts reading the notes. Based on EO 12333 and relevant guidance from the National Security Agency to read our personal information, it counts as a “collect” and may be subject to certain procedural requirements. But Rag doesn’t avoid these rules – it fits within them. If documents are already legally retained and AI simply accesses them to answer questions, the legal framework remains intact.
Because models do not absorb information into their internal structures (i.e. they do not change the weights of the model), RAG may work within existing legal frameworks. RAG keeps models unchanged, uses agency documents only at runtime, maintains legal accountability and prevents untraceable data retention. This design makes RAG legally clean, fundamentally different from traditional AI systems trained or fine-tuned with sensitive data.
In contrast, fine-tuning the model of sensitive data changes the internal parameters. Once data is incorporated into model weights, it becomes difficult to audit, delete, or control, and it makes it difficult to pose serious compliance challenges under statutes such as privacy laws and agency-specific retention rules.
The federal government must clarify how existing legal obligations apply to training and fine-tuning, especially as agencies employ increasingly sophisticated AI tools. In the meantime, RAG will provide legally prudent passes. This allows for powerful, document-conscious AI performance while maintaining auditability, control and compliance.
What Rag offers is structure. Cleanly separate the model from data subject to an agency’s existing legal obligations, such as retention schedules, access controls, and audit logs. Rules that require the removal of unrelated US personal information (such as minimization) still apply. It also accesses logs and purpose restrictions. It’s not new. It is a familiar legal task that applies to the new interface.
Still, there are some risks remaining. To manage access, agents can implement role-based access control (RBAC), relationship-based access control (REBAC), or fine-tuned approval (FGA). These approaches restrict access based on contextual factors such as user roles, relationships with data, and time and location of access, thereby limiting unnecessary exposure and matching long-standing data governance practices.
Encryption is also the key to the secure deployment of RAGs. Advanced techniques such as homogeneous encryption allow calculations to be directly generated on encrypted data without the need for decryption. Some agencies may be concerned about new technical concerns, such as embedding leaks that could result in partial reconstruction of vector-encoded information. This risk is unique to modern AI pipelines, but shares familiar data security challenges and characteristics. Access control, encryption, and adversarial testing strategies can help mitigate these risks.
How agents currently deploy AI systems
So, what should an institution do to use RAG? First, agents must choose a trustworthy foundation model and run it locally on a secure agency infrastructure. Without changing the internal structure of the model, agents can connect AI to internal data via the RAG pipeline and begin answering real operational questions. An open source framework can guide agencies to practical prototypes in just a few days.
The only real barrier remaining is institutional inertia. Of course, agents should continue to block and tackle. Protect your data, control access, disable unnecessary logging, and enable only authorized users to query sensitive material. However, these are familiar obligations under the privacy laws, EO 12333, and internal data processing protocols. Rather than removing liability, RAG restores legal awareness.
It’s an excuse to wait for the rag to delete. Failure to deploy AI tools that can support mission-critical decisions means transferring status to enemies who are not hesitant to deploy first. No new National Security Council memos, congressional orders or executive orders are required. Technology exists. The same goes for use cases. What’s missing is the development.
Rag is not a loophole or a workaround. A design choice that fits straight into an established legal framework. Agents do not need new doctrines to deploy it. This is just the perception that this architecture works within older buildings. While more complicated questions about fine-tuning and model training are waiting to be resolved, RAG offers a compliant, shocking interim solution.
As the Trump administration’s recent AI guidance has emerged, US federal agencies are poised to adopt AI more broadly and fully. AI is a strategic asset, and deploying it now is not just an opportunity, it is necessary.
Featured Images: US Flag and Computer Chip (via Getty Images)
