Share
Embed
A conversation about your code to build features in a fraction of the time
CloudConf, Turin, May 29th, 2024
Generative artificial intelligence (AI) can accelerate all phases of your software development lifecycle (SDLC) and streamline tasks such as research, design, coding, testing, debugging, and troubleshooting. This approach can extend to modernization, including upgrades to the latest language version. Through practical demos, this session will show how to incorporate cutting-edge technologies into your daily workflow, unlocking new levels of productivity and efficiency.
Danilo Poccia
May 29, 2024
Transcript
76 slides
-
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. A conversation about your code to build features in a fraction of the time Danilo Poccia (he/him) Chief Evangelist (EMEA) @danilop 1 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. A brief conversation with about generative AI 2 The capital of France is… -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Generative AI – Base models 3 The capital of France is Paris. User AI -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Generative AI – Instruction tuning 4 What is the capital of France? The capital of France is Paris. User Assistant -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Generative AI – Hallucinations 5 What are the two capitals of France? The two capitals of France are Paris and Strasbourg. Paris is the capital of France and the seat of the French national government, while Strasbourg is the capital of the Grand Est region and the seat of the European Parliament and other European institutions. O lder m odels 🫠 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Generative AI – Guardrails and checks 6 What are the two capitals of France? France has only one capital city, which is Paris. There are no other cities considered capital cities of France. Paris has been the capital and seat of government for centuries. N ew er m odels 👍 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. How do large language models (LLMs) work? 7 Text Model Text The model tries to continue a sentence (base model) or follow instructions (instruction tuning) What is the capital of France? The capital of France is Paris. -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. How do large language models (LLMs) work? 8 Text Model Text What is the capital of France? Oh, Paris, such a beautiful capital for France! Temperature (Randomness) Temperature controls how randomly next tokens (words) are chosen -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. How do large language models (LLMs) work? 9 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 10 Text Model Input Context (Embeddings) Output (Embeddings) Text Machine learning models only understand numbers Embeddings take something else and convert it into “numbers” (text, image, sound) → numbers + meaning -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 11 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 12 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Similar concepts → Near -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 13 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Something in between -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 14 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 15 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino Vectors -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 16 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x 2D Embeddings Vectors → Numbers -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 17 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino 3 4 5 2 3 3.5 y x (3, 3.5) (4, 3) (5, 2) 2D Embeddings -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Embeddings 18 Text Model Input Context (Embeddings) Output (Embeddings) Text Coffee Milk Cappuccino (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) (5, 2, 1.6, 4, …) N-Dimensional Embeddings 2D are not enough -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. (5, 2, 1.6, 4, …) (3, 3.5, 5, 1.2, …) (4, 3, 2.7, 5, …) How do large language models (LLMs) work? 19 Text Model Input Context (Embeddings) Output (Embeddings) Text What is the capital of France? The capital of France is Paris. -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input How do multimodal models work? 20 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Multimodal models are trained with a mix of different kind of inputs, such as text and images, including relations connecting different types, such as textual descriptions of images. -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input Conversations? 21 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Models have no memory -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input Conversations 22 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Models have no memory -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Conversations – Input context 23 What is the capital of France? The capital of France is Paris. What about Sweden? The capital of Sweden is Stockholm. User Assistant User Assistant Input Output -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Conversations – Lack of context 24 What about Sweden? Sweden is a nice country. User Assistant Input Output -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input Conversations and agents 25 Text Images Sound Model Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input 26 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append (Chat) Retrieval Augmented Generation Additional data Vector Index -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input 27 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Append + Plan (Chat / Agent) Retrieval Augmented Generation Additional data Vector Index Agent Plan / Chain of Thoughts 1. First, I need to do this 2. Then, I can do that 3. Finally, I can get the result -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input 28 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Read / Write APIs Web Search Content Repositories (CRM, Emails, …) Indexed Content (Wikipedia, …) Access Databases (SQL, NoSQL) -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Output Input 29 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Agent Plan + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Guardrails Output Input 30 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) M ental m odel Agent Plan + RAG + Tool use 1. First, I need to do this 2. Then, I can use this tool 3. Finally, I provide the result -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Guardrails Output Input 31 Text Images Sound Model RAG Semantic Database Input Context (Embeddings) Output (Embeddings) Text Images Sound Tools Internet Intranet Append + Plan (Chat / Agent) Context size -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 32 Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in- the-haystack eval. …this sentence seems very out of place and unrelated to the rest of the content in the documents… I suspect this pizza topping “fact” may have been inserted as a joke or to test if I was paying attention. Context size -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. How can generative AI help software developers? 33 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 34 From: [email protected] To: [email protected] Subject: Important news -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 35 Can you look after the customer feedback app? It’s the oldest app in production in our company! Don’t worry, we don’t expect many changes. It’ll mostly be maintenance… -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. One week later… 36 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 37 From: [email protected] To: [email protected] Subject: More important news -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 38 The new Trouble Ticket platform is strategic for our company! Can you connect the old customer feedback app with the new Trouble Ticket SaaS? It’d be great to have the integration ready by the end of the quarter… -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. And our adventure begins… 39 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON) -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 41 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 42 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 43 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 44 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. EC2 instance EBS Volume (file system storage) Feedback App Trouble Ticket SaaS Java 8 application External API TT API Feedback message (JSON) -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java 17 application External API TT API Feedback message (JSON) Docker “feedback” -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 47 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 49 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) EFS Standard Feedback App Trouble Ticket SaaS Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” Integration -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 52 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 53 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 54 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 55 Trouble Ticket SaaS REST API HTTP POST/PUT/GET/DELETE { "id": "123", "priority": "low, medium, high", "title": "Ticket Title", "message": "Long message or email" } { "ticket_id": "123", "priority": 1 to 5, "status": "open", "title": "Ticket Title", "body": "Max 200 characters" } Customer Feedback REST API HTTP POST/PUT/GET/DELETE -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 56 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API TT API Feedback message (JSON) Docker “feedback” CDK “Integration” -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) Lambda function New feedback message Integration queue EFS Standard Feedback App Trouble Ticket SaaS Integration Containerized Java application External API Python Lambda function TT API Feedback message (JSON) Docker “feedback” CDK “Integration” Amazon Bedrock Summarization and categorization -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 60 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 61 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 62 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Containers Application Load Balancer (ALB) Lambda function + URL Lambda function New feedback message Integration queue EFS Standard DynamoDB Table Feedback App Trouble Ticket SaaS Integration Containerized Java application Simulated with a Python Lambda function Python Lambda function TT API AWS IAM Auth Feedback message (JSON) Docker “feedback” CDK “Integration” CDK “TroubleTicketService” Amazon Bedrock Summarization and categorization DynamoDB Table Admin -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. To Do • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 65 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Done • Connect the Customer Feedback app with the Trouble Ticket SaaS • Map data types between the two apps, including unstructured text • Upgrade Customer Feedback app from Java 8 to 17 • Migrate from single server to a scalable, highly available architecture • Understand, refactor, and fix old code • Manage infrastructure as code • Favor decoupled architectures • Make the integration observable 66 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Which technologies did we use? 67 -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 68 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 69 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 70 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 71 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 72 What How AI Tool Code completion Lowest latency Base model Amazon Q Developer in IDE Amazon Q CLI Code chat Low latency Limited context (files) Instruct model + RAG Amazon Q Developer in IDE, CLI, AWS Console Chat Code agent Full repo in context Agent + RAG Amazon Q Developer Agent for software development /dev Code agent + test environment Full repo in context Test changes Iterate on errors Agent + RAG + Tools Amazon Q Developer Agent for code transformation /transform Code agent + repository Split a complex issue into smaller tasks Agent + RAG + Tools Amazon Q Developer in Amazon CodeCatalyst Issues + Pull Request (PR) -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. C A N L A N G U A G E M O D E L S R E S O L V E R E A L - W O R L D G I T H U B I S S U E S ? 73 SWE-bench Leaderboard https://www.swebench.com/ -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 74 Start with Amazon Q Developer at no cost • Code faster with code suggestions in the IDE § JetBrains § Visual Studio Code § Visual Studio (preview) § CLI (macOS) • Review code licenses with reference tracking • Limited monthly access for advanced features -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. 75 You can disable sharing content with AWS -
© 2024, Amazon Web Services, Inc. or its affiliates. All
rights reserved. Thank you! Danilo Poccia @danilop Please give me your feedback on X/Twitter or LinkedIn 😊