Upgrade to Pro — share decks privately, control downloads, hide ads and more …

A conversation about your code to build feature...

A conversation about your code to build features in a fraction of the time

CloudConf, Turin, May 29th, 2024

Generative artificial intelligence (AI) can accelerate all phases of your software development lifecycle (SDLC) and streamline tasks such as research, design, coding, testing, debugging, and troubleshooting. This approach can extend to modernization, including upgrades to the latest language version. Through practical demos, this session will show how to incorporate cutting-edge technologies into your daily workflow, unlocking new levels of productivity and efficiency.

Danilo Poccia

May 29, 2024
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. A conversation about your code to build features in a fraction of the time Danilo Poccia (he/him) Chief Evangelist (EMEA) @danilop 1
  2. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. A brief conversation with about generative AI 2 The capital of France is…
  3. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Generative AI – Base models 3 The capital of France is Paris. User AI
  4. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Generative AI – Instruction tuning 4 What is the capital of France? The capital of France is Paris. User Assistant
  5. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Generative AI – Hallucinations 5 What are the two capitals of France? The two capitals of France are Paris and Strasbourg. Paris is the capital of France and the seat of the French national government, while Strasbourg is the capital of the Grand Est region and the seat of the European Parliament and other European institutions. O lder m odels 🫠
  6. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Generative AI – Guardrails and checks 6 What are the two capitals of France? France has only one capital city, which is Paris. There are no other cities considered capital cities of France. Paris has been the capital and seat of government for centuries. N ew er m odels 👍
  7. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do large language models (LLMs) work? 7 Text Model Text The model tries to continue a sentence (base model) or follow instructions (instruction tuning) What is the capital of France? The capital of France is Paris.
  8. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do large language models (LLMs) work? 8 Text Model Text What is the capital of France? Oh, Paris, such a beautiful capital for France! Temperature (Randomness) Temperature controls how randomly next tokens (words) are chosen
  9. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How do large language models (LLMs) work? 9 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers
  10. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 10 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers Embeddings take something else and convert it into “numbers” (text, image, sound) → numbers + meaning
  11. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 11 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk
  12. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 12 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Similar concepts → Near
  13. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 13 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Something in between
  14. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 14 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors
  15. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 15 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors
  16. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 16 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x 2D Embeddings Vectors → Numbers
  17. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 17 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x (3, 3.5) (4, 3) (5, 2) 2D Embeddings
  18. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Embeddings 18 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) (5, 2, 1.6, 4, …) N-Dimensional Embeddings 2D are not enough
  19. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. (5, 2, 1.6, 4, …) (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) How do large language models (LLMs) work? 19 Text Model Input Context (Embeddings) Output (Embeddings) Text What is the capital of France? The capital of France is Paris.
  20. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input How do multimodal models work? 20 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Multimodal models are trained with a mix of different kind of inputs, such as text and images, including relations connecting different types, such as textual descriptions of images.
  21. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input Conversations? 21 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Models have no memory
  22. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input Conversations 22 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Models have no memory
  23. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Conversations – Input context 23 What is the capital of France? The capital of France is Paris. What about Sweden? The capital of Sweden is Stockholm. User Assistant User Assistant Input Output
  24. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Conversations – Lack of context 24 What about Sweden? Sweden is a nice country. User Assistant Input Output
  25. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input Conversations and agents 25 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat)
  26. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input 26 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Retrieval Augmented Generation Additional data Vector Index
  27. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input 27 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append + Plan (Chat / Agent) Retrieval Augmented Generation Additional data Vector Index Agent Plan / Chain of Thoughts 1. First, I need to do this 2. Then, I can do that 3. Finally, I can get the result
  28. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input 28 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Read / Write APIs Web Search Content Repositories (CRM, Emails, …) Indexed Content (Wikipedia, …) Access Databases (SQL, NoSQL)
  29. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Output Input 29 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Agent Plan + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result
  30. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Guardrails Output Input 30 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) M ental m odel Agent Plan + RAG + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result
  31. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Guardrails Output Input 31 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Context size
  32. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 32 Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in- the-haystack eval. …this sentence seems very out of place and unrelated to the rest of the content in the documents… I suspect this pizza topping “fact” may have been inserted as a joke or to test if I was paying attention. Context size
  33. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How can generative AI help software developers? 33
  34. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 35 Can you look after the customer feedback app? It’s the oldest app in production in our company! Don’t worry, we don’t expect many changes. It’ll mostly be maintenance…
  35. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. One week later… 36
  36. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 37 From: [email protected] To: [email protected] Subject: More important news
  37. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 38 The new Trouble Ticket platform is strategic for our company! Can you connect the old customer feedback app with the new Trouble Ticket SaaS? It’d be great to have the integration ready by the end of the quarter…
  38. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. And our adventure begins… 39
  39. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON)
  40. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 41
  41. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 43
  42. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON)
  43. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java 17 application External API TT API Feedback message (JSON) Docker “feedback”
  44. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 47
  45. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration
  46. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration
  47. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback”
  48. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 52
  49. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 55 Trouble Ticket SaaS REST API HTTP POST/PUT/GET/DELETE { "id": "123", "priority": "low, medium, high", "title": "Ticket Title", "message": "Long message or email" } { "ticket_id": "123", "priority": 1 to 5, "status": "open", "title": "Ticket Title", "body": "Max 200 characters" } Customer Feedback REST API HTTP POST/PUT/GET/DELETE
  50. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” CDK “Integration”
  51. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) Lambda function New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API Python Lambda function TT API Feedback message (JSON) Docker “feedback” CDK “Integration” Amazon Bedrock Summarization and categorization
  52. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization
  53. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 61
  54. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization
  55. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization DynamoDB Table Admin
  56. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 65
  57. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Done • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 66
  58. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Which technologies did we use? 67
  59. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 68 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI
  60. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 69 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat
  61. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 70 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev
  62. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 71 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform
  63. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 72 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform Code agent + repository Split a complex issue into smaller tasks Agent + RAG + Tools Amazon Q Developer in Amazon CodeCatalyst Issues + Pull Request (PR)
  64. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. C A N L A N G U A G E M O D E L S R E S O L V E R E A L - W O R L D G I T H U B I S S U E S ? 73 SWE-bench Leaderboard https://www.swebench.com/
  65. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 74 Start with Amazon Q Developer at no cost • Code faster with code suggestions in the IDE § JetBrains § Visual Studio Code § Visual Studio (preview) § CLI (macOS) • Review code licenses with reference tracking • Limited monthly access for advanced features
  66. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. 75 You can disable sharing content with AWS
  67. © 2024, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Thank you! Danilo Poccia @danilop Please give me your feedback on X/Twitter or LinkedIn 😊