Building Production-Ready Apps with AI

Have you ever interacted with a multi-turn conversational model trained
through extensive transformer architectures for dynamic user engagement?

Have you ever used ChatGPT?

Foundation Models Prompts RAG Vector Database Fine-Tuning Few-Shot Learning Context
Hallucinations Zero-Shot Learning

Building production- ready apps with LLMs on AWS, without the
confusing slang

@slobodan_ How do the Large Language Models (LLMs) work?

Source: https://platform.openai.com/tokenizer

@slobodan_ Prompts

@slobodan_ Prompts •Prompts are just instructions. •You tell an LLM
what you want, and the LLM tries to reply based on its training. •More detailed and better explanation = better answer. •LLM will always answer, but answers are not always based on truth.

@slobodan_ Slobodan Stojanović CTO and co-founder of Vacation Tracker co-author
of Serverless Apps with Node.js book AWS Serverless Hero JS Belgrade meetup organizer

@slobodan_ Models

@slobodan_ LLM Models

@slobodan_ OpenAI - GPT-4 Turbo •OpenAI has multiple models, but
GPT-4 Turbo is the best one. •Price: •Input: US$ 10.00 / 1M tokens •Output: US$ 30.00 / 1M tokens •Quality: ChatGPT level*

@slobodan_ Generative Pre-trained Transformer (GPT)

@slobodan_ Multimodal Models

@slobodan_

@slobodan_ Anthropic Claude •Claude 3 oﬀers 3 models: Opus, Sonnet
and Haiku •Claude 3 Opus is at the "GPT-4 level." •Price: •Input: US$ 15 / 1M tokens (Opus), US$ 0.25 / 1M (Haiku) •Output: US$ 75 / 1M tokens (Opus), US$ 1.25 / 1M (Haiku)

@slobodan_ Google Gemini •People had high hopes for Google LLM.
•Good quality and an impressive 1M context. •Price*: •Input: US$ 7 / 1M tokens •Output: US$ 21 / 1M tokens

@slobodan_ Open-source models

@slobodan_ Mistral & Mixtral •First "WOW!" open-source model. •Both open-source
(Mixtral 8x7B and 8x22B) and commercial models (Mistral Large). •Price*: •Input: US$ 0.45 / 1M tokens (Mixtral 8x7B), US$ 8 / 1M (Mistral Large). •Output: US$ 0.7 / 1M tokens (Mixtral 8x7B), US$ 24 / M (Mistral Large)

@slobodan_ Mixture of Experts

@slobodan_

@slobodan_ Meta Llama3 •Llama 3 is Faaaast! And open-source. •Not
multi-modal yet, but very good. •Price*: •Input: US$ 0.4 / 1M tokens (8B), US$ 2.65 / 1M (70B) •Output: US$ 0.6 / 1M tokens (8B), US$ 3.5 / 1M (70B)

@slobodan_ Parameters (8 billion vs 70 billion)

@slobodan_ Small bits of memory that help an LLM decide
what to say next.

@slobodan_ How can you use these models today?

@slobodan_ OpenAI models are available via OpenAI API and Microsoft
Azure AI

@slobodan_ Azure OpenAI •OpenAI models integrated in Azure. •Same pricing
as OpenAI API, but paid with your Azure subscription. •You can control the region where the model is deployed. •Microso"'s ToC and SLA: •"Azure OpenAI doesn't use customer data to retrain models."

@slobodan_ Amazon Bedrock •Amazon's Generative AI platform •Amazon hosts foundation
models and oﬀers APIs for them. Select your region. •Amazon's ToC and SLA: •"Your training data isn't used to train the base Titan models or distributed to third parties."

@slobodan_ Amazon Bedrock Models •Claude (all models) •Mistral AI (most
models) •LLama (all models) •Stable Diﬀusion •Cohere (I just tried it initially and forgot about it) •Jurassic (I have no idea what's this) •Amazon's Titan (mostly useless at the moment)

@slobodan_

@slobodan_ Some models are available in speciﬁc regions only!

@slobodan_ US East (N. Virginia) and US West (Oregon) regions
has most of the models.

@slobodan_

@slobodan_ Amazon Bedrock Features •API •Playgrounds •Agents •Fine tuning •Guardrails
•Knowledge base (managed RAG*)

@slobodan_ What are AI Agents?

@slobodan_ LLMs are good at responding to wel-deﬁned and focused
tasks

@slobodan_ It's better to split a complex task into many
smaller and focused sub-tasks

@slobodan_

@slobodan_ AI Agent frameworks and tools •Langchain •AutoGPT •AutoGen, BabyAGI,
and many others •OpenAI Assistants •Bedrock Agents •Or build your own simple agent!

@slobodan_ Can I host/run a dedicated model on Bedrock?

@slobodan_ Yes, but…

@slobodan_

@slobodan_ Don't like Azure and AWS? There are also many
other places with AI models!

@slobodan_ Other platforms •Cloudflare AI: mostly open-source models. •Groq: open-source
models, fast! •LPU™ (Language Processing Unit) •Google Vertex AI Studio: Gemini models •Most of the models also oﬀer platforms. •Many others…

@slobodan_ Self-hosting of open-source models

@slobodan_ How to run an LLM locally •LM Studio (many
open-source LLMs, easy download, oﬀers playgrounds) •Ollama •Hugging Face

@slobodan_ LLMs require strong machines and a lot of memory!

@slobodan_ What can you build using LLMs today?

@slobodan_ What are LLMs good at? •Text summarization, labeling, and
structuring •Text generation •Personalization and translations •And many more things…

@slobodan_ LLMs can improve UX for many products

@slobodan_

@slobodan_ But we can go a step further: Send a
simple email!

@slobodan_ Show me the prompts!

@slobodan_ Simplified version

@slobodan_ "Prompt engineering"

@slobodan_ Zero-shot vs few-shot prompting

@slobodan_ Zero-shot prompting

@slobodan_ Few-shot prompting

@slobodan_ Few-shot is more expensive, but it gives more accurate
answers

@slobodan_ Context is a problem, you can't ﬁt everything in
a single prompt

@slobodan_ Retrieval-augmented generation (RAG)

@slobodan_

@slobodan_ Source: https://cthiriet.com/blog/infinite-memory-llm

@slobodan_ Vector Databases

@slobodan_ Do you actually need a vector database?

@slobodan_ Popular Vector Databases •Pinecone, etc. •ElasticSearch, OpenSearch •Pgvector PostgreSQL
extension •Or you can use S3 and similar!

@slobodan_ Security

@slobodan_ Should I host my own model?

The correct answer in 99.999% of cases. NO!

@slobodan_ All LLM platforms “We don't train our models with
your data!” Every single one of them!

@slobodan_ Prompt injections

@slobodan_

@slobodan_ OWASP Top 10 for LLM •Prompt Injection •Insecure Output
Handling •Training Data Poisoning •Model Denial of Service •Supply Chain Vulnerabilities Source: https://owasp.org/www-project-top-10-for-large-language-model-applications/

@slobodan_ OWASP Top 10 for LLM •Sensitive Information Disclosure •Insecure
Plugin Design •Excessive Agency •Overreliance •Model The" Source: https://owasp.org/www-project-top-10-for-large-language-model-applications/

@slobodan_ What about AGI?

@slobodan_ Artiﬁcial General Intelligence (AGI)

@slobodan_ ChatGPT “AGI is like a Swiss Army knife for
the brain, brilliantly juggling any task you throw at it—from cracking jokes to solving quantum physics puzzles!” Not an AGI yet :)

@slobodan_

@slobodan_ LLMs are not magic, but you can build amazing
things using them.

@slobodan_ Questions? Twitter: @slobodan_ Linkedin: sstojanovic I guess you can
simply use ChatGPT, too.

Building Production-Ready Apps with AI

Building Production-Ready Apps with AI

More Decks by Slobodan Stojanović

Other Decks in Programming

Featured

Transcript