• Emp Profile, 1on1, MBO, Performance, HR FAQ, etc… • Big Issue 1. Company with tens of thousands of employees 500 inquiries / 2 personnel / month →90% are listed in FAQ → consulted about closing the inquiry function
Augmented Generation) App • HR FAQ & Chat history with HR → Embedding • User question → retrieve and answer using ChatGPT(API) • 6 Days to make Demo for HR Conference • 2 Months to rebuild for PRODUCTION
framework for application development using LLM. ◦ There are two implementations: Python and JavaScript/TypeScript. ◦ Python version is more active development. • LangChain is available as OSS and is updated daily. • My recommendation is to use TypeScript for creating a demo, but if you want to build a production version or take full advantage of LangChain’s functionality in the long term, I recommend the Python version.
Easy to integrate ◦ Needs few lines of code. • Super Fast and Reliable ◦ Always respond within few msec. • Higher Security ◦ We need to care about many compliance because CYDAS is hosting many personal data in it.
with LLMs, but very hard to make something production-ready with them. - Chip Huyen • Security and Compliance: ◦ OpenAI → Azure OpenAI ◦ Pinecone → Azure Cognitive Search • Safety ◦ Accuracy, Hallucination, Fairness
is easy to implement -> Can traditional search UI (without LLMs) solve this problem? 2. Workflow to take advantage of LLM capabilities is important 1. Combine deterministic programming with non-deterministic LLMs 2. Chain multiple tasks together 🦜🔗. 3. 🦜🔗 is a treasure trove of ideas + implementations 1. ReAct → langchain.agents 2. HyDE → LLM fantasizes about the answer to a question and searches for knowledge similar to that answer from langchain.chains import HypotheticalDocumentEmbedder 4. Enterprise search is usually beer for everything than Vector Similarity Search only 5. LLMOps≠MLOps 1. Hard to notice changes in input / output 2. Limited ability to notice = replace API or model, adjust prompts (+ version control) 3. Response time should be captured e.g. LangSmith
Insecure Output Handling 3. Training Data Poisoning 4. Model Denial of Service 5. Supply Chain Vulnerabilities 6. Sensitive Information Disclosure 7. Insecure Plugin Design 8. Excessive Agency 9. Overreliance 10. Model Theft OWASP Top 10 for Large Language Model Applications https://owasp.org/www-project-top-10-for-large-language-model-applications/
Insecure Output Handling 3. Training Data Poisoning 4. Model Denial of Service 5. Supply Chain Vulnerabilities 6. Sensitive Information Disclosure 7. Insecure Plugin Design 8. Excessive Agency 9. Overreliance 10. Model Theft OWASP Top 10 for Large Language Model Applications https://owasp.org/www-project-top-10-for-large-language-model-applications/
including CVE (vulnerabilities) are now in a separate package (Experimental) ◦ Streamlining of the 🦜🔗 core ◦ Mentioned plans for a package called Community Chain • Means… ◦ Cannot be used in production → Can be used ◦ Unlimited expansion over the past year or so has meant that the Lambda Layer will one day no longer be ridden → Constant traffic control will be possible. ◦ Implementation of papers and ambitious ideas will be more PR-friendly • for AWS Lambda ◦ Current size: 130MB after expansion including dependent libraries ◦ Spin-up takes approximately 5 seconds → Multiple measures are needed, such as Lazy listeners and retry header checks when using from Slack
yoshidashingo
zsmb
77web
javiergs
ymtdzzz
gam0022
masuda220
maaretp
tutinoko
tsucchinoko
iriikeita
euxn23
palkan
iamctodd
shpigford
bryan
destraynor
brianwarren
notwaldorf
malarkey
bluesmoon
tammielis
lara
aki_iinuma
![Momento Confidential “PEOPLE Copilot Chat” = RAG[Grounding] App • RAG(Retrieval](https://files.speakerdeck.com/presentations/278bd262794143eb93043d549f6229b1/slide_4.jpg){kind=link}
