Share
Embed
Artificial Intelligence Integration: Using Spring AI with OpenAI (ChatGPT) and AWS Bedrock to Deliver Efficient APIs
Introduction
Topic 1: Generative AI – Concepts and Advantages
Topic 2: Getting to Know OpenAI and AWS Bedrock
Topic 3: Integration: Spring AI with OpenAI (ChatGPT)
Topic 4: Integration: Spring AI with AWS (Bedrock)
Topic 5: Presentation of the integration architecture AWS Bedrock
Conclusion
weder96
September 20, 2024
Transcript
104 slides
-
Artificial Intelligence Integration: Using Spring AI with OpenAI (ChatGPT) and
AWS Bedrock to Deliver Efficient APIs #JoinCommunity2024 #en-US -
Integração de inteligência artificial: usando Spring AI com OpenAI (ChatGPT)
e AWS Bedrock para fornecer APIs eficientes #JoinCommunity2024 #pt-BR -
Weder Mariano de Sousa Post Graduate in Midias UFG https://www.linkedin.com/in/wedermarianodesousa/
AWS User Group Goiânia https://github.com/weder96 GOJava About the Speaker Specialist Senior Java - GFT Graduated Computer Science https://twitter.com/weder96 Post Graduate in Information Security https://dev.to/weder96 Technician System Development Serverless -
Agenda Introduction Topic 1: Generative AI – Concepts and Advantages
Topic 2: Getting to Know OpenAI and AWS Bedrock Topic 3: Integration: Spring AI with OpenAI (ChatGPT) Topic 4: Integration: Spring AI with AWS (Bedrock) Topic 5: Presentation of the integration architecture AWS Bedrock Conclusion -
What is Generative AI Image Text/Chat Generated with OpenAI Image
Generator G1 Generated with OpenAI Image Generator G1 > Question: What is generative AI? Generative AI refers to a type of artificial intelligence that is capable of creating new and original content. It uses algorithms and machine learning to generate text, images, music, and other forms of media based on patterns and rules learned from existing data. Create an image of a puppy eating a popsicle -
What is Generative AI Image Text/Chat Generated with Amazon Titan
Text G1 - Express Generated with Amazon Titan Image Generator G1 > Question: What is generative AI? Generative AI refers to a type of artificial intelligence that is capable of creating new and original content. It uses algorithms and machine learning to generate text, images, music, and other forms of media based on patterns and rules learned from existing data. A stylized picture of a cute old steampunk robot -
What is Generative AI Artificial Intelligence Machine Learning Deeping Learning
Supervised LLMS Generative AI -
Machine Learning Use Cases • Facial Recognition • Recognize Tumors
on x - ray scans • Abnormality on ultrasounds • Self-driving mode (recognize stop sign / pedestrian / etc) • Fraud detection • Product Recommendations (YouTube) • Spam Filtering -
1. Scientific Advances - Deep Learning 2. Availability of Big
Data (You need data to configure these neural networks) 3. Lots of Compute Power A r t i f i c i a l N e u r a l N e t w o r k -
L a r g e La n g u a
g e M o d e l s (LLM) -
Large Language Models So what are LLMs LLMs are a
type of artificial intelligence that can generate text, understand language, answer questions and more. They are large because they have a vast number of parameters, which are the parts of the model that are learned from data during training. ChatGPT 1. 175 Billion Parameters 2. Training Data - Hundreds of Billions of words -
Describe your qualities to me explaining to a developer ?
Sure! Here are some key qualities I can describe, specifically in the context of a developer: Access to Information: I have access to a wide range of information and knowledge... Troubleshooting: I am able to help identify and resolve software issues by providing diagnostics... Multi-Language Support: I can provide support for a variety of programming languages... Documentation and Reference: I have the ability to generate and explain documentation for APIs... Continuous Learning: I am always up to date with my latest knowledge... Project Management Assistance: In addition to helping with the actual coding... Personalization of Answers: I can tailor my answers to the developer’s level of knowledge... Tool and Libraries Suggestions: I can recommend tools, libraries... These qualities make me a valuable resource for developers looking for quick information, support, and solutions to their coding challenges. -
Bigger is Better Very Large Neural Networks Vast Amounts of
Training Data Huge Compute Power to Train Data General Purpose AI Attention is all you need https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf -
Specialized architecture for token prediction Key Innovation Attention mechanisms Not
Just a big neural network Attention is all you need Transformer Architecture -
As Artificial Intelligence (AI) models become more important and widespread
in almost every sector, it is increasingly important for businesses to understand how these artificial intelligence models work and the potential implications of using them. https://viso.ai/deep-learning/ml-ai-models/ AI models and their various applications. We will explore popular use cases of AI models and discuss how they can be used effectively in business and other settings. -
A new Spring Project Mark Pollack Current Version 1.0.0-M2 https://spring.io/projects/spring-ai
Inspired by Python projects LangChain LllamaIndex Spring AI AI for Spring Developers https://docs.spring.io/spring-ai/reference/index.html -
Aligns with Spring project design values Component Abstractions & Default
Implementations Portable Chat Completion and EmbeddingClient Multimodality Support Portable Vector Store API & Query Language Function Calling Key Components Spring AI AI for Spring Developers https://docs.spring.io/spring-ai/reference/index.html -
Spring AI API The Spring AI API covers a wide
range of functionalities Chat Model Give it some text, get some text back Embedding Portable API across Vector Store Providers Text to Image Give it some text, get an image back Transcription Give it some audio, get some text back Functions AI model invoke your POJO java.util.Function. Object -
Chat Model Give it some text, get some text back
Open AI Azure Open AI Amazon Bedrock Google Vertex AI Palm Google Gemeni HuggingFace - Access to thousands of models, including those from Meta such as Llama 2 Ollama - Run AI Models on your local machine MistralAI -
Text-to-Image Models Give it some text, get an image back
• OpenAI with DALL-E • StabilityAI -
Tr a n s c r i p t i
o n Give it some audio, get some text back • Open AI -
Embedding Models Portable API across Vector Store providers • Open
AI • Azure Open AI • Ollama • ONNX • PostgresML • Bedrock Cohere • Bedrock Titan • Google VertexAI • MistalAI -
Prompts serve as the foundation for the language-based inputs that
guide an AI model to produce specific outputs. For those familiar with ChatGPT, a prompt might seem like merely the text entered into a dialog box that is sent to the API. However, it encompasses much more than that. In many AI Models, the text for the prompt is not just a simple string. -
• Input directing an AI model to produce specific outputs
• Model outputs are greatly inference by prompt style and wording • Prompt Engineering • Prompt techniques and effective prompts are share in the community • OpenAI Guidelines • Mini Course: ChatGPT Prompt Engineering for developers • Prompts and Spring • Prompt management relies on Text Template Engines • Analogous to the view in Spring MVC Prompt Engineering Effective Communication -
public class Prompt implements ModelRequest<List<Message > > { private final
List<Message> messages; private ChatOptions modelOptions; public Prompt(String contents) { this((Message)(new UserMessage(contents))); } public Prompt(Message message) { this(Collections.singletonList(message)); } public Prompt(List<Message> messages) { this.messages = messages; } public Prompt(String contents, ChatOptions modelOptions) { this((Message)(new UserMessage(contents)), modelOptions); } public Prompt(Message message, ChatOptions modelOptions) { this(Collections.singletonList(message), modelOptions); } public Prompt(List<Message> messages, ChatOptions modelOptions) { this.messages = messages; this.modelOptions = modelOptions; } -
public class Prompt implements ModelRequest<List<Message > > { private final
List<Message> messages; private ChatOptions modelOptions; public Prompt(String contents) { this((Message)(new UserMessage(contents))); } public Prompt(Message message) { this(Collections.singletonList(message)); } public Prompt(List<Message> messages) { this.messages = messages; } public Prompt(String contents, ChatOptions modelOptions) { this((Message)(new UserMessage(contents)), modelOptions); } public Prompt(Message message, ChatOptions modelOptions) { this(Collections.singletonList(message), modelOptions); } public Prompt(List<Message> messages, ChatOptions modelOptions) { this.messages = messages; this.modelOptions = modelOptions; } -
Roles • System Role: Guides the AI’s behavior and response
style, setting parameters or rules for how the AI interprets and replies to the input. It’s a k i n to providing instructions to the AI before initiating a conversation. • U s e r R o l e : R e p r e s e n t s t h e u s e r ’ s i n p u t – t h e i r q u e s t i o n s , c o m m a n d s , o r s t a t e m e n t s t o t h e AI. T h i s r o l e i s f u n d a m e n t a l a s i t f o r m s t h e b a s i s o f t h e A I ’ s response. • Assistant Role: The AI’s response to the user’s input. More than just an answer or reaction, it’s c r u c i a l for m a i n t a i n i n g the f l o w of the conversation. B y t r a c k i n g the AI’s previous responses (its 'Assistant Role' messages), the s y s t e m ensures coherent and contextually relevant interactions. • Function Role: This role deals with specific tasks or operations during the conversation. While the System Role sets the AI’s overall behavior, the Function Role focuses on carrying out certain actions or commands the user asks for. It’s like a special feature in the AI, used when needed to perform specific functions such as calculations, fetching data, or other tasks beyond just talking. This role allows the AI to offer practical help in addition to conversational responses. -
O u t p u t Pa r s i
n g Challenges with handling the response The Challenge • Output of Generative LLM is a java.util.String • Even if you ask for JSON, you get a JSON String • ChatGPT wants to chat, not reply in JSON • OpenAI has introduced a new feature to help with this • Spring AI’s OutputParser uses refined prompts to desired results -
AI A p p l i c a t i
o n A r c h i t e c t u r e - RAG ? Argument Prompt Vector Store LLM Application -
Conceptual Flow of using RAG with LLMs Search Relevant Information
Prompt + Query Prompt + Query + Enhanced Context Query Large Language Model EndPoint Knowledge Sources Generate Text Response Relevant Information Enhanced context 4 2 3 5 https://aws.amazon.com/pt/what-is/retrieval-augmented-generation/ 1 -
Ve c t o r Stores Not just for text
search • Azure Vector Search • ChromaVectorStore • MilvusVectorStore • Neo4JVectorStore • PgVectorStore • QdrantVectorStore • RedisVectorStore • WeaviateVecvtorStore • SimpleVectorStore -
#! /bin/bash echo "Calling Open AI" MY_OPENAI_KEY="YOUR_API_KEY_HERE" PROMPT="When was the
first version of Java released?" curl https://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $MY_OPENAI_KEY" \ -d '{"model": "gpt-3.5-turbo-0125", "messages": [{"role":"user", "content": "'"${PROMPT}"'"}] }' -
Calling Open AI { "id": "chatcmpl-A5e48Ns4Tu9NscMH8oAIdClNF8dH6", "object": "chat.completion", "created": 1725909016,
"model": "gpt-3.5-turbo-0125", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "The first version of Java, known as Java 1.0, was released by Sun Microsystems on January 23, 1996.", "refusal": null }, "logprobs": null, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 16, "completion_tokens": 28, "total_tokens": 44 }, "system_fingerprint": null } Response -
public static void main(String[] args) throws IOException, InterruptedException { var
apiKey = "YOUR_API_KEY_HERE"; var body = """ { "model": "gpt-4", "messages": [ { "role": "assistant", "content": "What is Spring Boot?" } ] }"""; HttpRequest request = HttpRequest.newBuilder() .uri(URI.create("https://api.openai.com/v1/chat/completions")) .header("Content-Type", "application/json") .header("Authorization", "Bearer " + apiKey) .POST(HttpRequest.BodyPublishers.ofString(body)) .build(); var client = HttpClient.newHttpClient(); var response = client.send(request, HttpResponse.BodyHandlers.ofString()); System.out.println(response.body()); } -
INPUTS COMPLEX OUTPUTS SIMPLE INPUTS COMPLEX OUTPUTS COMPLEX INPUTS SIMPLE
OUTPUTS SIMPLE DEEP LEARNING FOUNDATION MODELS MACHINE LEARNING -
The easiest way to build and scale generative AI applications
with foundation models Amazon Summarization, complex reasoning, writing, coding Contextual answers, summarization, paraphrasing High-quality images and art Text generation, search, classification Q&A and reading comprehension Text summarization, generation, Q&A, search, image generation Amazon Titan Text Premier Amazon Titan Text Lite Amazon Titan Text Express Amazon Titan Text Embeddings Amazon Titan Text Embeddings V2 Amazon Titan Multimodal Embeddings Amazon Titan Image Generator Claude 3 Opus Claude 3 Sonnet Claude 3 Haiku Claude 2.1 Claude 2 Claude Instant Llama 3 8B Llama 3 70B Llama 2 13B Llama 2 70B Command Command Light Embed English Embed Multilingual Command R+ Command R Stable Diffusion XL1.0 Stable Diffusion XL 0.8 Jurassic-2 Ultra Jurassic-2 Mid Mistral Large Mistral 7B Mixtral 8x7B Text summarization, Q&A, text classification, text completion, code generation -
How does a foundation model work? Data Text Images Speech
Structured data 3D signals Pre-train *can take weeks or even months Fine-tune for specific tasks and domains *can take hours Gather data at scale Evaluate model Foundation model -
Amazon Bedrock Serveless Basic Architecture AWS Cloud AWS Lambda Amazon
API Gateway Amazon Bedrock statics Pages Amazon S3 Statics Pages Text Generation User s -
Advantages of the Serverless Approach AWS Cloud Client Authenticate User
Lambda function Extract Document Metadata Amazon S3 Statics Pages AWS Amplify Hosting React Application Amazon API Gateway Amazon Cognito Amazon SQS Queue Lambda function Embed Documents Amazon Bedrock Amazon DynamoDB Conversation Memory Lambda function Generate LLM Response Lambda function CRUD Operations Dump vector index Upload Document Get LLM Response Get vector Dump Get Conversation memory -
Advantages of the Serverless Approach Amplify Login Lambda function Amazon
API Gateway Amazon Cognito Amazon Bedrock Get Answer U s er s Amazon S3 Amazon CloudFront origins use A ma zon Pol l y text-to-speech React Voice By Microphone -
Amazon Q Developer The most capable generative AI–powered assistant for
software development -
The image shows a workspace that includes a laptop (an
ASUS model), a smartphone, a pen, a notebook or planner, and a coffee mug filled with coffee. Additionally, there are several baby toys and accessories scattered around, including a pacifier, a plush toy, colorful rings, and a small wooden toy with wheels. The setting suggests a combination of work and childcare activities in the same space. A imagem mostra um espaço de trabalho que inclui um laptop (um modelo ASUS), um smartphone, uma caneta, um caderno ou agenda e uma caneca de café cheia de café. Além disso, há vários brinquedos e acessórios de bebê espalhados, incluindo uma chupeta, um brinquedo de pelúcia, anéis coloridos e um pequeno brinquedo de madeira com rodas. O cenário sugere uma combinação de trabalho e atividades de cuidados infantis no mesmo espaço. 1 2 3 4 -
I don't have real-time access to current data, so I
can't provide the exact subscriber counts for YouTubers. However, I can certainly list some of the most popular tech YouTubers as of my last update: 1. Marques Brownlee (MKBHD) 2. Linus Tech Tips 3. Unbox Therapy 4. Dave Lee (Dave2D) 5. Austin Evans 6. Jonathan Morrison (TLD) 7. iJustine 8. The Verge 9. Android Authority 10. Mrwhosetheboss For the most up-to-date subscriber counts, I recommend visiting their YouTube channels directly. 1 2 -
Getting started Get started with Amazon Bedrock Discover features with
a step-by-step tutorial Dive deep with a hands-on workshop -
Resources #Links (BedRock) https://www.ranthebuilder.cloud/post/automating-api-calls-with-agents-for-amazon-bedrock-with-powertools https://www.slightinsight.com/tech/developing-a-spring-boot-application-with-amazon-bedrock-api/ https://community.aws/content/2dhKdwyY1kzhFTg9CTLbaJ9MmTN/build-generative-ai-applications-with-amazon-bedrock https://www.eficode.com/blog/building-ai-on-aws-bedrock-brings-brilliant-building- blocks?utm_campaign=AWS&utm_content=189323971&utm_medium=social&utm_source=twitter&hss_channel=tw- 142208607 https://cloudacademy.com/learning-paths/integrating-aws-services-with-llms-and-other-fms-14068/
https://www.youtube.com/watch?v=CE_-zrMvcuk&list=PLhr1KZpdzukfmv7jxvB0rL8SWoycA9TIM&index=5 https://programadriano.medium.com/conhecendo-o-amazon-bedrock-c687c7e9777f https://github.com/AWS-Cloud-Drops-Builders-Edition/show?tab=readme-ov-file https://levelup.gitconnected.com/ai-powered-video-summarizer-with-amazon-bedrock-and-anthropics-claude-9f1832f397dc https://catalog.workshops.aws/persona-based-access-genai-application/en-US/04-testing-application/01-updating-code https://dev.to/aws-builders/have-fun-with-aws-partyrock-3755 https://dev.to/aws-builders/build-serverless-generative-ai-api-service-with-aws-lambda-and-amazon-bedrock-3abc https://norahsakal.medium.com/how-to-use-aws-titans-ai-multimodal-embeddings-for-better-e-commerce- recommendations-b9f4adb60c02 #Importante https://www.linkedin.com/pulse/aplica%C3%A7%C3%B5es-serverless-llm-com-amazon-bedrock-diogo-santos-yjo3f/ https://aws.amazon.com/pt/blogs/aws-brasil/crie-aplicativos-de-ia-generativa-usando-o-aws-step-functions-e-o-amazon- bedrock/ https://community.aws/generative-ai https://aws.amazon.com/pt/what-is/retrieval-augmented-generation/ -
Resources #Links Spring AI OpenAI https://github.com/danvega/hello-gpt/blob/main/src/main/java/dev/danvega/Application.java https://www.youtube.com/watch?v=yyvjT0v3lpY&list=PLZV0a2jwt22uoDm3LNDFvN6i2cAVU_HTH https://tecnoblog.net/guias/4-sites-gratis-para-transcrever-video-do-youtube/ #Spring AI
Vector https://www.youtube.com/watch?v=azKntWC6d3w #Top https://github.com/kousen/openaidemo https://www.youtube.com/watch?v=ZeH3bBKdqRU -
Weder Mariano de Sousa Post Graduate in Midias UFG https://www.linkedin.com/in/wedermarianodesousa/
https://github.com/weder96 About the Speaker Specialist Senior Java - GFT Graduated Computer Science https://twitter.com/weder96 Post Graduate in Information Security https://dev.to/weder96 Q & A AWS User Group Goiânia GOJava Technician System Development Serverless