Upgrade to Pro — share decks privately, control downloads, hide ads and more …

From Vibe to Secure Code

Sponsored · SiteGround - Reliable hosting with speed, security, and support you can count on.

From Vibe to Secure Code

Mitigating Risks in AI-Accelerated Software Development

Avatar for Lothar Schulz

Lothar Schulz

June 25, 2026

More Decks by Lothar Schulz

Other Decks in Programming

Transcript

  1. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software Development
  2. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Lothar Schulz lotharschulz.info lotharschulz.info linkedin.com/in/lotharschulz github.com/lotharschulz speakerdeck.com/lothar
  3. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software Development
  4. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://x.com/karpathy/status/1886192184808149383
  5. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://www.youtube.com/watch?v=8lF7HmQ_RgY
  6. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://simonwillison.net/2025/Oct/7/vibe-engineering/
  7. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://simonwillison.net/2025/Oct/7/vibe-engineering/
  8. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://www.lotharschulz.info/2025/07/07/let-the-agents-vibe-introducing-flow-coding/ https://cloud.google.com/blog/products/data-analytics/vibe-querying-with-comments-to-sql-in-bigquery/
  9. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://simonwillison.net/2026/May/6/vibe-coding-and-agentic-engineering/ “Vibe coding and agentic engineering have started to converge in my own work” “ … provided you understand when it can be used and when it can’t.”
  10. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Focus Shift https://www.chrisgregori.dev/opinion/code-is-cheap-now-software-isnt
  11. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Focus Shift - Orchestrator The value of an engineer is shifting away from the "how" of syntax and toward the "what" and "why" of systems. Real engineering lies in the abstractions and the architecture. It’s about knowing how to structure a system that lasts, understanding why a specific rate-limiting strategy is necessary, knowing how to manage a distributed cache, and knowing exactly where not to store your environment variables.
  12. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Orchestrator How much do you invest into monitoring & managing the AI accelerated code artefacts?
  13. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Hallucinations LLMs cannot learn all the computable functions and will therefore inevitably hallucinate if used as general problem solvers https://arxiv.org/abs/2401.11817
  14. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Hallucinations https://www.youtube.com/watch?v=wjZofJX0v4M&t=1434s - LLMs generate text on a "token-by-token" basis by repeatedly sampling from a probability distribution for each next token - At every such step, the model computes so-called logits for all tokens in the vocabulary - are then transformed into a probability distribution via a softmax function - so-called temperature is part of the softmax function calculation. A model's temperature setting thus allows you to control how "confident" or how "creative" the model should be when producing each next token. The lower the temperature, the more confident; the higher the temperature, the more often the model can choose the somewhat less certain or more creative next token.
  15. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://www.youtube.com/watch?v=wjZofJX0v4M&t=1434s Temperature
  16. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Hallucinations & Temperature https://arxiv.org/pdf/2406.10279
  17. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development https://docs.cloud.google.com/vertex-ai/generative-ai/docs/start/gemini-3-prompting-guide#temperature-tuning Temperature
  18. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Slopsquatting https://mastodon.social/@andrewnez/114302875075999244 Slop Typosquatting
  19. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Slopsquatting https://arxiv.org/pdf/2406.10279
  20. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Slopsquatting https://arxiv.org/pdf/2406.10279
  21. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Slopsquatting https://www.hiddenlayer.com/research/malware-found-in-trending-hugging-face-repository-open-oss-privacy-filter The repository had typosquatted OpenAI's legitimate Privacy Filter release, copied its model card nearly verbatim, and shipped a loader.py file that fetches and executes infostealer malware on Windows machines. May 2026
  22. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Image Prompt Injection https://pub.towardsai.net/how-hackers-hide-malicious-prompts-in-images-to-exploit-google-gemini-ai-62d4fd4a8417
  23. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risks - Role Probes https://role-confusion.github.io LLMs don't actually rely on the official tags to identify roles Writing style is used to identify roles This insight enabled a novel attack: inject fake reasoning-style text into a user prompt. This enables injecting fake reasoning-style text into a user prompt. The model treats it as its own already-reached conclusion and acts on it without scrutiny.
  24. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Follow Up Issues https://github.com/matplotlib/matplotlib/pull/31132
  25. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Follow Up Issues https://theshamblog.com/an-ai-agent-published-a-hit-piece-on-me/
  26. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Follow Up Issues https://news.ycombinator.com/item?id=46990729
  27. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development The lethal trifecta for AI agents https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
  28. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Promptware Kill Chain https://www.lawfaremedia.org/article/the-promptware-kill-chain
  29. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation https://gist.github.com/lotharschulz/371dd560ec4bc9c1959ad8a8b9484587
  30. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Global Instructions
  31. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Taylored Instructions
  32. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Sandbox https://www.lotharschulz.info/2025/06/25/getting-started-with-google-gemini-cli-complete-setup-guide-and-rust-testing-experience/#sandbox
  33. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Lockdown Mode https://www.lotharschulz.info/2026/06/06/open-ai-lockdown-mode/
  34. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Nono Sandbox https://gist.github.com/lotharschulz/30605d3d492599914f3c494e569b7fc5 https://www.lotharschulz.info/2026/05/29/nono-sandbox-for-claude-code-revisited-fixing-the-curl-install-breakage/ https://www.lotharschulz.info/2026/05/02/secure-your-terminal-a-trust-but-verify-sandbox-for-claude-code/
  35. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Sandbox https://www.luiscardoso.dev/blog/sandboxes-for-ai
  36. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - OpenSandbox https://www.lotharschulz.info/2026/06/06/mapping-opensandbox-to-the-luis-cardoso-ai-sandboxing-model/ Container By default, OpenSandbox supports standard Docker and Kubernetes runtimes. gVISOR The OpenSandbox documentation explicitly lists support for secure container runtimes like gVisor. MicroVM OpenSandbox also supports Firecracker and Kata Containers.
  37. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Input https://github.com/lotharschulz/pii-redaction-guard
  38. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Output guardrails https://github.com/lotharschulz/llm-output-guardrails
  39. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - Role Probes There are two ways to resist injection: 1. memorizing known attacks (brittle — humans can rephrase) or 2. genuine role perception as training-time property; ignore commands regardless of phrasing or assumed role Mitigations (authors are fairly pessimistic) - Treat Data as Code - Reject Forged Reasoning - Detect Task Divergence https://role-confusion.github.io/ https://gist.github.com/lotharschulz/371dd560ec4bc9c1959ad8a8b9484587 https://www.lotharschulz.info/2026/06/25/prompt-injection-as-role-confusion-explained-why-llms-confuse-style-with-roles/
  40. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Risk Mitigation - “Classic” tools https://github.com/lotharschulz/llm-output-guardrails
  41. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development MCP https://www.lotharschulz.info/2025/04/09/rust-mcp-local-server-bridging-rust-logic-with-ai-frontends/ https://genai.owasp.org/resource/a-practical-guide-for-secure-mcp-server-development/ https://modelcontextprotocol.io/docs/tutorials/security/authorization#when-should-you-use-authorization Use Cases (at least) - user-specific data (emails, documents, databases) - audit who performed which actions - access to APIs that require user consent - enterprise environments with strict access controls - rate limiting or usage tracking per user
  42. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Pipeline modernisation https://cloud.google.com/blog/topics/developers-practitioners/from-vibe-checks-to-continuous-evaluation-engineering-reliable-ai-agents/
  43. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Editor Loop “The fastest way to make your agents better at your codebase is a tighter feedback loop with scripts you already have.” “Beyond the basics with Claude Code”, Code w/ Claude 2026 verify.sh hooked to your agent - lint - compile - test - build Claude Code understands CLIs by default https://www.youtube.com/watch?v=tuY2ChJIx48
  44. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Abstract Syntax Tree https://www.researchgate.net/publication/400118489_Detecting_and_Correcting_Hallucinations_in_LLM-Generated_Code_via_Deterministic_AST_Analysis
  45. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Abstract Syntax Tree https://github.com/yvgude/lean-ctx
  46. From Vibe to Secure Code Mitigating Risks in AI-Accelerated Software

    Development Swiss Cheese Model https://en.wikipedia.org/wiki/Swiss_cheese_model