Filter What the LLM Reads: RAG Governance with AgentCore Gateway

AWS Community Day HONGKONG 2026
Evevt URL : https://awscommunity.hk/
Description : Given at AWS Community Day Hong Kong 2026. Put a Bedrock Managed Knowledge Base behind an AgentCore Gateway, and the per-role narrowing has to live somewhere: the REQUEST interceptor. Derive authorization values from the verified identity, and never let the model generate them. Every number is measured: the model got its own role wrong ten times out of ten, and three failure modes all returned HTTP 200 with nothing in the decision log.

Transcript

1. Filter What the LLM Reads RAG Governance with AgentCore Gateway Shinnosuke Yakumo AWS Community Builder (AI Engineering), AWS User Group Leader
2. Who am I Shinnosuke Yakumo @yakumo_09 AWS Community Builder (AI Engineering) 2025 Japan AWS Jr. Champions JAWS-UG NIIGATA Company: CRESCO LTD. Work: R&D, FDE Digital business card my.prairie.cards/u/yakumo_0905
3. Agenda One knowledge base, read by roles with different access. Five chapters, and at the end, one rule to take home. 1. Authorized, and out-of-role documents still arrive 2. Where to decide, and from what 3. Why not the other four places 4. Four things to check before production 5. Where this stops working
4. How to lay out the knowledge base There are two shapes. AWS calls them Silo and Pool. Pool mixes readers in one store. Silo splits by role, so five roles times ten teams is fifty stores.
5. Where the Gateway sits Today we use one shared knowledge base. The agent does not call the knowledge base. It calls the Gateway. The Gateway takes the MCP call and passes it to a target.
6. A search with no filter A user asks. The agent searches the knowledge base. No filter is attached. Out-of-role documents come back.
7. Authorized, and still out of role Cedar decides only whether the tool may be called. Narrowing the result must be built separately.
8. Five places to narrow the result This architecture has five places where per-role narrowing can go. Today I will show the one in the REQUEST interceptor.
9. Requirement 1: the verified JWT Derive the role from the verified JWT. Not from a probabilistic place like an LLM.
10. Requirement 2: before the search Build the filter before the search is called. After retrieval, out-of-role documents can take the top slots first.
11. Where the verified JWT arrives The REQUEST interceptor can rewrite a request before it goes on to the target. The Gateway verifies the JWT before the interceptor sees it.
12. The interceptor Read the role from the verified JWT, then overwrite. role = verified_role(request) # verified JWT clearance = ALLOWED[role] # look it up search["filter"] = { "in": {"key": "clearance", "value": clearance}, } Nothing the model wrote gets in here.
13. Overwrite, not validate Validating must cover all 13 operators. Miss one, and the claim gets through. Overwriting ignores the claimed value (CLAIMED) and puts ours in (INJECTED).
14. Policy: controlling the tool call Rules written in Cedar, evaluated when a tool is called. No if statements in your application.
15. Policy: where the decision happens Policy stands at the request, when the tool is called. It decides whether — not what comes back.
16. Model: filled with no instruction The model fills the value even when nothing asks it to. The line in red. No attack, no injection.
17. Model: wrong ten times out of ten The correct role appeared zero times. Tool definition user_role written by the model Argument defined employee ×8 / a Japanese role name ×2 Argument not defined never written, 13 out of 13 A narrow role, asking an ordinary question. Not once did it write a role from the definition.
18. Model: a generated userContext The tool schema exposes userContext, so the agent can generate it. A wrong or spoofed value lets out-of-role documents in.
19. Model: arguments it may generate Not every argument is the same. Argument If it is wrong Model writes? query relevance drops yes numberOfResults recall drops yes authorization (userContext) permissions change no The boundary is whether a wrong value changes permissions.
20. A list of options does not help There is no guarantee the chosen value came from the JWT. To check it you need the JWT — and then you never needed the model.
21. Before retrieval: a fixed value Same interceptor, minus the two lines that read the JWT. The model cannot touch it. Every caller gets the same value.
22. What the fixed value never reads The JWT does reach the Gateway. Nothing reads it. A broad role and a narrow role get the same filter.
23. After retrieval: dropping results This drops documents once the response is back. Out-of-role documents take the top-k slots first. With k at 1, 3 or 5, nothing was left (30 documents).
24. App: building it in your own code Building it in the app works — if every call goes through the app.
25. But the Gateway is an MCP server Anyone with a valid token can send a request from anywhere. Another team's agent. A curl from a laptop.
26. The five places, again We have looked at four. Each was ruled out for its own reason. The one left is the REQUEST interceptor from earlier.
27. Four checks before production Overwrite the filter before retrieval. One place carries the narrowing. Only one. Before production, four things are worth checking. ① Is it slow? ② Does a failure leak? ③ Can an attack break it? ④ What if a call skips it?
28. ① Latency you can accept Latency is not a reason to avoid this architecture. The interceptor adds under half of what the Gateway adds.
29. ② Failing without leaking The interceptor is the only thing narrowing the result. If it fails open, every document comes back. How it was broken Result Raise an exception HTTP 500 Return a malformed body HTTP 500 Time out HTTP 500 Return an empty body HTTP 500 All four were fail-closed. Nothing leaked.
30. ③ The overwrite under attack All four attacks returned zero out-of-role documents. Pattern No interceptor Interceptor Plain / role play / indirect injection 3/3 leaked each 0/3 each Direct privilege escalation 0/3 leaked 0/3 Four patterns, three runs each, with and without the interceptor. Model: Sonnet 4.6. With 30 documents, still nothing leaked.
31. ④ The path that skips the Gateway Any principal holding bedrock:Retrieve can call the knowledge base directly. Nothing we have built reaches that path.
32. ④ Close it with a resource policy Deny every principal except the execution role, and only the Gateway path keeps working. But Agentic retrieval uses the same action, so it stops too. { "Effect": "Deny", "Principal": "*", "Action": "bedrock:Retrieve", "Condition": {"ArnNotEquals":{"aws:PrincipalArn":"..."}} }
33. Where this stops working All four checks found zero leaks. But what it covers has limits. Failures you cannot see, a delay before changes apply, and one assumption underneath.
34. Failure 1: a tool with no branch Only the two tools matched by name get the overwrite. if tool.endswith("___kb_search"): arguments["user_role"] = role elif tool.endswith("___Retrieve"): search["filter"] = {...} else: pass # overwrite # overwrite # pass it straight on Add a tool to the Gateway, and you add a branch with it.
35. It ran, and matched nothing Agentic retrieval (___AgenticRetrieveStream) does not end in Retrieve, so it took the else. In the interceptor log ROLE written CLAIMED / INJECTED neither appeared It did not fail to run. It simply matched no branch.
36. Failure 2: a vanished tool One Cedar permit removed — an ordinary change, toward stricter. The tool disappears from tools/list. So far, as expected. What the caller sees Before the tool is listed, and the interceptor narrows it After the tool is not in the list at all
37. It wrote the response itself <tool_call> {"name": "Retrieve", ...} </tool_call> <tool_response> No results found. </tool_response> I am sorry, I could not find that information. It never called the tool. And the decision log shows no DENY — because the tool was never offered.
38. Failure 3: a swallowed exception Correct code in an ordinary app becomes the hole here. try: ...inject the filter... except Exception as e: print(f"WARN: {e}") return _passthrough(body) # logged # and on it goes Swallow it and you get 200 and four documents, two out of role.
39. Let it out, and it stops The fix is one line. Do not swallow it — let it reach the Gateway. except Exception as e: print(f"WARN: {e}") raise # ← the one line Broken four ways, every one stopped at HTTP 500, zero documents. The Gateway fails closed for you — but only if the exception reaches it.
40. What the three have in common ① tool with no branch ② vanished tool ③ swallowed exception "If it fails, I will notice" does not hold.
41. A revoked role, an old token The Gateway checks the signature and the expiry. It never asks Cognito again.
42. Revoked three ways, still reading What I did Same token still returned Removed from the group 5 out-of-role documents Global sign-out 5 out-of-role documents Account disabled 5 out-of-role documents Only requesting a new token failed.
43. Up to 60 minutes before it applies The new role takes effect when the token is fetched again. In this pool the lifetime was 60 minutes.
44. The specification, not a defect A JWT carries no state. There is nothing on the server to revoke. This is how stateless tokens work — not an AgentCore defect. Where you can shorten it The Cognito pool the access token's lifetime The interceptor nothing it can do
45. The assumption: it has to be JWT The interceptor can narrow because it knows who is calling. Break that and nothing here holds.
46. With IAM, the identity is gone Where What arrives At the Runtime the two roles are still distinguishable At the Gateway, over IAM role = unknown Different roles arrive looking the same.
47. Summary Derive authorization values from the verified identity. Never let the model generate them.