It’s Tuesday, November 17th: Welcome to another edition of The Byte.
In this piece, Karthik Chandrasekaran, Principal Engineering Manager at Microsoft, argues that enterprise AI safety is decided at the retrieval layer, where permissions and provenance are settled before a model writes a word. Most public debate focuses on alignment, but in the systems he builds, which feed AI assistants used across Fortune 500 companies, safety and compliance are won or lost upstream of the model.
A model can't enforce a policy it never sees. If the retrieval pipeline hands it a document the user was never allowed to open, even a well-aligned model will summarize it faithfully. Karthik walks through where those leaks happen: permission drift, chunks that cross trust boundaries, and prompt injection through retrieved content.
His answer is defense in depth. Enforce identity at query time, bind access controls to every chunk, keep untrusted text separate from instructions, and log provenance so every answer can be audited. He is candid about the tradeoff, since these controls add latency and can reduce recall, but argues that a slightly less helpful answer is far cheaper than a leaked one.
He also makes the case that regulation is looking at the wrong layer, focused on the model while the retrieval infrastructure around it goes ungoverned. His closing message is direct: the retrieval pipeline, not the frontier model, is what stands between your users and your most sensitive data.
Event Spotlight: Tonight in SF 👀

This is a super exclusive drone show party happening tonight in San Francisco.
Some crazy stuff is about to drop today from the Cognition team, and you're going to want to be celebrating at this party. That's all we can say for right now because we have to be hush-hush, but AI Collective members are getting the inside scoop.
Tonight at 8:45 PM PT, above the Ferry Building in San Francisco.
The Governance Layer Nobody Regulates

KEY TAKEAWAYS
Safety and compliance in enterprise AI are decided at the retrieval and grounding layer, before the model generates a single token.
The common leaks are permission drift, chunks that cross trust boundaries, and prompt injection through retrieved content.
The fix is defense in depth: enforce identity at query time, bind access controls to every chunk, keep untrusted text separate from instructions, and log provenance.
Most of the public argument about AI safety is happening one layer too high.
Read the policy papers, the model cards, and the statements from frontier lab safety teams, and you would think the whole game is alignment: teach the model to refuse the wrong requests, avoid toxic output, and generally behave. That work matters. But it is not where enterprise deployments actually fail. In the systems I have spent the last few years building, high scale retrieval and grounding infrastructure that feeds AI assistants used across Fortune 500 companies, safety and compliance are decided before the model generates a single token. They are decided at the retrieval and grounding layer. And almost no regulatory framework is looking there.
Here is the claim I want to defend. A frontier model cannot enforce your corporate compliance policy, because the model never sees your policy. It sees the context your retrieval pipeline hands it. If that pipeline returns a document the user was never allowed to open, the most carefully aligned model in the world will summarize it faithfully and hand the secret over. The failure was not the model's judgment. It was an access decision made three steps upstream.
The model inherits every upstream decision

Figure 1. Enterprise AI grounding pipeline. Trust flows downhill; by the time generation runs, permission and provenance decisions are already locked in.
To see why, follow a single query through an enterprise grounding pipeline. A user asks a question. The system authenticates them, retrieves candidate passages from a vector and lexical index, ranks and assembles those passages into a context window, and only then calls the model to generate an answer grounded in that context. This is retrieval-augmented generation, and some version of it now sits behind most serious enterprise AI.
The important property is directional. Trust flows downhill. Every decision about what the user is allowed to see is made in the retrieval and grounding stages. By the time the model runs, those decisions are locked. The model has no independent way to know that passage seven came from an HR file the user has no rights to. It cannot re-derive permissions it was never given. It works with what it is fed.
That is why I treat the grounding layer as the real security surface of enterprise AI. It is the last place where identity, permissions, and provenance are still legible. Once content crosses into the context window it has been laundered of its origin. It is just text, and text is persuasive.
Where the leaks actually happen
The failure modes are not exotic. They are mundane, which is exactly why they are dangerous.
The most common is permission drift. Enterprises trim search results to a user's access rights, but the enforcement point matters enormously. If you copy access control lists into the index at ingestion time and then never reconcile them, every reorganization, offboarding, and permission change quietly desynchronizes your security model from reality. A document shared broadly for one week can stay retrievable for a user who lost access months ago. The index remembers what the source has forgotten. The correct answer, in my experience, is to enforce identity at query time, trimming to the caller's live permissions on every request, and to treat any control that lives only in the prompt as decorative.
The second is the trust boundary that a chunk quietly crosses. Retrieval systems break documents into passages for relevance and cost. But a passage carries no memory of the wall it came from. A single high scoring chunk pulled from a confidential contract can land in the context for a user who should only see the public summary. The ranking model did its job and found relevant text. Relevance and authorization are different questions, and a system that optimizes only for the first will cheerfully violate the second.
The third is injection. Once you ground on retrieved content, you have invited untrusted text into the same context as your instructions. A document can contain language that reads, to the model, like a command. This is the core of what the OWASP Top 10 for LLM Applications calls prompt injection and sensitive information disclosure, the first two entries on that list for good reason. Your retrieved corpus is now part of your attack surface, and it grows every time someone uploads a file.
None of these are alignment problems. You cannot fine tune your way out of a stale access control list.
What good governance looks like at the grounding layer
Part of why this persists is that regulation has treated the model as the unit of risk. The EU AI Act reasons about model capability and purpose, and most published safety work evaluates models in isolation, on benchmarks, with model cards. In an enterprise system, though, the model is a commodity. The differentiated and under governed part is the retrieval infrastructure wrapped around it, and teams are left to build that governance ad hoc, usually after an incident. The good news is that it is a tractable engineering problem, and the discipline it needs is one our field already knows well: defense in depth.
Worth crediting: the NIST AI Risk Management Framework and its Generative AI Profile (NIST AI 600-1) explicitly address data provenance, information integrity, and access, and MITRE ATLAS catalogs real adversarial techniques against AI systems, including retrieval and data poisoning. These are the right instincts, but they remain advisory, and are dwarfed by the attention paid to frontier alignment.

Figure 2. Defense in depth for the grounding layer. Governance is a set of controls spread across the life of a query, not a single gate.
Concretely, that means enforcing controls at every stage of a query's life. At ingestion, capture the source's access controls and sensitivity labels and bind them to every chunk, so provenance survives the act of chunking. At query time, trim retrieval to the caller's live identity, and enforce those permissions inside the retrieval service rather than trusting the prompt to behave. At grounding, keep untrusted retrieved text structurally separated from system instructions, and require that every claim in the answer cite a source the user is actually permitted to see. After generation, scan output for sensitive data patterns and log the full provenance chain so the whole path can be audited and red-teamed later.
The principle underneath all of it is simple. Identity and trust boundaries are enforced by the infrastructure, before the model is ever involved. The model is the last mile, not the checkpoint.
There is a real tradeoff here, and it is worth naming. Every one of these controls costs latency and recall. Query-time permission trimming is more expensive than trusting a stale copy. Isolating untrusted context and demanding citations narrows what the model can draw on, and sometimes it will decline to answer a question it could technically have answered. When I have shipped these controls, the honest framing to leadership was never that safety is free. It is that a slightly less helpful answer is vastly cheaper than a leaked one, and that measuring only relevance, with metrics like NDCG, while ignoring authorization is how you build an impressively wrong system.
What changes now
The uncomfortable implication for anyone deploying enterprise AI is that your governance posture is only as strong as your retrieval stack, and most organizations have never audited it as a security system. They bought a model, wrapped it in a search index, and assumed the safety conversation was happening somewhere upstream at the lab. It was not. The lab aligned the model against generic harms. It could not, and did not, encode your permission model.
So the work is ours. Treat the grounding layer as a first class security surface. Put identity enforcement at query time. Make provenance survive chunking. Test for data leakage the way you already test for relevance. And when the next wave of AI regulation is drafted, argue for it to reach past the model and into the infrastructure, because that is where enterprise data safety is actually settled.
The frontier model is not the thing standing between your users and your most sensitive data. Your retrieval pipeline is. Build it like you know that.
Sources and further reading
OWASP Top 10 for LLM Applications (OWASP GenAI Security Project). https://genai.owasp.org/llm-top-10/
NIST AI Risk Management Framework (AI RMF 1.0). https://www.nist.gov/itl/ai-risk-management-framework
NIST Generative AI Profile (NIST AI 600-1). https://doi.org/10.6028/NIST.AI.600-1
MITRE ATLAS: Adversarial Threat Landscape for AI Systems. https://atlas.mitre.org/
EU Artificial Intelligence Act, Regulation (EU) 2024/1689. https://eur-lex.europa.eu/eli/reg/2024/1689/oj
Figures 1 and 2 are original diagrams created by the author for this article.
MISSED LAST WEEK'S BYTE?
The AI Collective is built by volunteers across 180+ chapters in 40 countries.
Thank you to the thousands of volunteers around the world who make this work possible. We truly could not do this without you.
🧑💻 About the Author & the Editorial Team

About Karthik Chandrasekaran
Karthik Chandrasekaran is a Principal Engineering Manager at Microsoft, where he leads the retrieval stack behind M365 Copilot's grounding and semantic search. Over 18 years he has built enterprise scale identity, FinTech, and AI systems at Microsoft, Apple, and Capital One, and is a GIAC certified DevSecOps practitioner based in Austin, Texas.

About Josh Evans
Josh is a Managing Editor at The AI Collective Newsletter and leads content for The Byte. Outside of AIC, Josh works in Content Protection at Spotify.


