Beyond the hype: The reality of AI security

Artificial Intelligence (AI) has rapidly evolved, promising transformative advancements. However, as AI systems become increasingly sophisticated and integrated into our daily lives, so do the security risks associated with them. This talk delves into the realities of AI security, beyond the hype and promises. We will explore the vulnerabilities inherent in AI systems, including data privacy breaches, adversarial attacks, and model poisoning. The presentation will also discuss the ethical implications of AI, such as bias and discrimination, and how they intersect with security concerns. By the end of this talk, attendees will gain a comprehensive understanding of the challenges and best practices for securing AI systems, enabling them to build and deploy AI solutions responsibly. Between the main points of the talk we can highlight:
*Introduction to OWASP LLM Top 10
*Ethical implications of AI security
*Securing AI systems: Best practices and emerging technologies
*The future of AI security

Transcript

1. Beyond the hype: The reality of AI security José Manuel Ortega jose.manuelortega@owasp.org
2. AI CONNECT 2025 • Introduction to OWASP LLM Top 10 • Ethical implications of AI security • Securing AI systems: Best practices and emerging technologies • The future of AI security
3. AI CONNECT 2025 Introduction to OWASP LLM Top 10 https://genai.owasp.org
4. AI CONNECT 2025 Introduction to OWASP LLM Top 10
5. AI CONNECT 2025 Prompt Injection
6. AI CONNECT 2025 Indirect Prompt Injection
7. AI CONNECT 2025 Indirect Prompt Injection
8. AI CONNECT 2025 Indirect Prompt Injection
9. AI CONNECT 2025 ChiperChat https://arxiv.org/pdf/2308.0 6463
10. AI CONNECT 2025 Jailbreak prompts • https://jailbreak-llm s.xinyueshen.me/
11. AI CONNECT 2025 Jailbreak prompts • https://jailbreak-llm s.xinyueshen.me/
12. AI CONNECT 2025 Data Poisoning
13. AI CONNECT 2025 Adversarial Attacks
14. AI CONNECT 2025 Adversarial Attacks • Small Perturbations: Adversarial attacks typically involve adding small, carefully crafted perturbations to the input data that are often imperceptible to humans. These subtle changes can trick the AI system into making wrong predictions or classifications. • Model Vulnerabilities: These attacks exploit specific weaknesses in the machine learning model, such as its inability to generalize well to new, unseen data or the sensitivity of the model to certain types of input. • Impact on Critical Systems: Adversarial attacks can have severe consequences when applied to AI systems in critical domains such as autonomous vehicles, facial recognition systems, medical diagnostics, and security systems.
15. AI CONNECT 2025
16. AI CONNECT 2025 Adversarial Attacks
17. AI CONNECT 2025 Adversarial Attacks • 1. Prompt Injection • 2. Evasion Attacks • 3. Poisoning Attacks • 4. Model Inversion Attacks • 5. Model Stealing Attacks • 6. Membership Inference Attacks
18. AI CONNECT 2025 Adversarial Attacks
19. AI CONNECT 2025 Adversarial Attacks
20. AI CONNECT 2025 Adversarial Attacks
21. AI CONNECT 2025 Adversarial Attacks
22. AI CONNECT 2025 Adversarial Attacks
23. AI CONNECT 2025 Adversarial Attacks
24. AI CONNECT 2025 • Trust: Users must trust that AI systems are unbiased and transparent. • Fairness: Avoiding discrimination and ensuring equal treatment across all demographics. • Accountability: Ensuring that actions taken by AI systems can be explained and justified. The Importance of Ethics in AI-Driven Cybersecurity
25. AI CONNECT 2025 Understanding Algorithmic Bias in AI Systems • Data Bias: Incomplete or unrepresentative training data. • Algorithmic Bias: Flaws in the algorithm design. • Human Bias: Prejudices introduced by the developers.
26. AI CONNECT 2025 • False Positives/Negatives: Incorrectly identifying safe activities as threats or missing actual threats. • Discrimination: Unequal treatment of certain user groups, potentially exposing them to higher risks. • Erosion of Trust: Users may lose confidence in AI systems that are perceived as biased. The Impact of Bias on Cybersecurity Outcomes
27. AI CONNECT 2025 Strategies to Mitigate Algorithmic Bias • Diverse Training Data: Ensure the training data represents various demographics and scenarios. • Regular Audits: Conduct periodic reviews of AI systems to identify and correct biases. • Algorithm Transparency: Make the algorithm’s workings understandable and accessible to all stakeholders.
28. AI CONNECT 2025 • Explainable AI (XAI): Developing AI systems that can explain their decisions in human terms. • Open Source Models: Sharing AI models publicly to allow scrutiny and improvement by the broader community. • Documentation: Providing detailed documentation on the AI system’s functioning and decision criteria. Ensuring Transparency in AI Algorithms
29. AI CONNECT 2025 • Ethical Frameworks: Implement frameworks that guide the ethical use of AI in cybersecurity. • Stakeholder Involvement: Engage diverse stakeholders in developing and monitoring AI systems. • Continuous Improvement: Regularly update AI systems and policies to adapt to new ethical challenges and technological advancements. Balancing Security Needs with Ethical Concerns
30. AI CONNECT 2025 • Inclusive Design: Incorporate input from diverse groups during AI system development. • Bias Mitigation Techniques: Use techniques like reweighting or re-sampling to reduce bias in training data. • Ethics Committees: Establish committees to oversee the ethical aspects of AI deployment. Best Practices for Ethical AI Implementation in Cybersecurity
31. AI CONNECT 2025 • Decision Review: Humans should review critical decisions made by AI systems. • Exception Handling: Develop protocols for handling exceptions and anomalies that AI cannot address. • Ongoing Training: Train security personnel to understand and effectively oversee AI systems. The Role of Human Oversight in AI-Based Security Systems
32. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 1. Adversarial Defense
33. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 1. Adversarial Defense • Adversarial Training: This involves exposing AI models to adversarial examples during training to make them more robust. • Defensive Distillation: This technique involves training a model to be less sensitive to adversarial inputs by smoothing out the decision boundaries. • Robust Optimization: Utilizing optimization techniques that focus on creating models resilient to adversarial perturbations.
34. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 1. Adversarial Defense • Explainable AI (XAI): By making AI systems more transparent and interpretable, we can better understand why a system is vulnerable to adversarial attacks and improve its robustness. • Certified Defenses: Researchers are developing methods that provide provable guarantees that a model is resistant to adversarial examples, leading to more secure AI systems.
35. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 2. Secure Data Handling and Privacy Preservation • Data Encryption: Encrypt data both at rest and in transit to ensure that it cannot be accessed or tampered with by unauthorized actors. • Federated Learning: This technique allows models to be trained on decentralized data, ensuring that raw data is not shared across servers. This improves privacy by keeping sensitive data local. • Differential Privacy: A method that adds noise to datasets or model outputs to protect individual data points from being exposed.
36. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 2. Secure Data Handling and Privacy Preservation • Homomorphic Encryption: A technology that allows computations to be performed on encrypted data without decrypting it first, ensuring privacy during processing. • Secure Multi-Party Computation (SMPC): This allows different parties to collaboratively perform computations on data without revealing the underlying data itself.
37. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 3. AI Model Integrity and Trustworthiness
38. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 3. AI Model Integrity and Trustworthiness • Regular Model Audits: Continuously auditing AI models to ensure they are working as expected, and recalibrating them when necessary to maintain their integrity. • Bias Detection: Implement methods to identify and mitigate bias in AI models, ensuring fairness and preventing discriminatory outcomes. • Secure Model Versioning: Track and control the versions of models deployed, making sure that no unauthorized changes can be made without proper validation.
39. AI CONNECT 2025 Securing AI systems: Best practices and emerging technologies • 3. AI Model Integrity and Trustworthiness • Blockchain for Model Integrity: Blockchain can be used to create transparent, immutable records of AI model versions, ensuring their integrity and making it easier to track changes. • AI Testing Frameworks: New testing frameworks are emerging that can automatically assess AI models for vulnerabilities, biases, and overall performance.
40. AI CONNECT 2025 The future of AI security • Adversarial Machine Learning • AI as a Security Tool • AI-Powered Privacy Protection • Regulations and Standards • Human-AI Collaboration
41. AI CONNECT 2025
42. AI CONNECT 2025 • github.com/greshake/llm-security • github.com/corca-ai/awesome-llm-security • github.com/facebookresearch/PurpleLlama • github.com/protectai/llm-guard • github.com/cckuailong/awesome-gpt-security • github.com/jedi4ever/learning-llms-and-genai-for-dev -sec-ops • github.com/Hannibal046/Awesome-LLM
43. AI CONNECT 2025 • https://cloudsecurityalliance.org/artifacts/security-implicatio ns-of-chatgpt • https://www.nist.gov/itl/ai-risk-management-framework • https://blog.google/technology/safety-security/introducing-g oogles-secure-ai-framework • https://owasp.org/www-project-top-10-for-large-language- model-applications/