Lock in $30 Savings on PRO—Offer Ends Soon! ⏳

Reality is not an End-to-End Prediction Problem...

Reality is not an End-to-End Prediction Problem: Applied NLP in the Age of Generative AI

Video: https://www.youtube.com/watch?v=K_Y9wvGjNKw

Large Language Models (LLMs) and in-context learning have introduced a new paradigm for developing natural language understanding systems: prompts are all you need! Prototyping has never been easier, but not all prototypes give a smooth path to production. In this talk, I'll share the most important lessons we've learned from solving real-world information extraction problems in industry, and show you a new approach and mindset for designing robust and modular NLP pipelines in the age of Generative AI.

Breaking down larger business problems into actionable machine learning tasks is one of the central challenges of applied natural language processing. I will walk you through example applications and practical solutions, and show you how to use LLMs to their fullest potential, how and where to integrate your custom business logic and how to maximize efficiency, transparency and data privacy.

Ines Montani

October 17, 2024
Tweet

Video


Resources

A practical guide to human-in-the-loop distillation

https://explosion.ai/blog/human-in-the-loop-distillation

This blog post presents practical solutions for using the latest state-of-the-art models in real-world applications and distilling their knowledge into smaller and faster components that you can run and maintain in-house.

How S&P Global is making markets more transparent with NLP, spaCy and Prodigy

https://explosion.ai/blog/sp-global-commodities

A case study on S&P Global’s efficient information extraction pipelines for real-time commodities trading insights in a high-security environment using human-in-the-loop distillation.

How GitLab uses spaCy to analyze support tickets and empower their community

https://explosion.ai/blog/gitlab-support-insights

A case study on GitLab’s large-scale NLP pipelines for extracting actionable insights from support tickets and usage questions.

Applied NLP Thinking: How to Translate Problems into Solutions

https://explosion.ai/blog/applied-nlp-thinking

This blog post discusses some of the biggest challenges for applied NLP and translating business problems into machine learning solutions, including the distinction between utility and accuracy.

The Window-Knocking Machine Test

https://ines.io/blog/window-knocking-machine-test/

How will technology shape our world going forward? And what tools and products should we build? When imagining what the future could look like, it helps to look back in time and compare past visions to our reality today.

Using LLMs for human-in-the-loop distillation in Prodigy

https://prodi.gy/docs/large-language-models

Prodigy comes with preconfigured workflows for using LLMs to speed up and automate annotation and create datasets for distilling large generative models into more accurate, smaller, faster and fully private task-specific components.

More Decks by Ines Montani

Other Decks in Technology

Transcript

  1. de fi nition s E volution rules or instructions ✍

    programming & rules machine learning examples 📝 supervised learning
  2. de fi nition s E volution rules or instructions ✍

    programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering
  3. de fi nition s E volution rules or instructions ✍

    programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering instructions: human-shaped, easy for non-experts, risk of data drift ✍
  4. de fi nition s E volution rules or instructions ✍

    programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive
  5. de fi nition s E volution rules or instructions ✍

    programming & rules machine learning examples 📝 supervised learning in-context learning rules or instructions ✍ LLM prompt engineering ? ? LLM instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive
  6. Falcon MIXTRAL GPT-4 good contextual results ⚠ transparency ⚠ e

    iciency easy to use & configure fast prototyping LLM
  7. Falcon MIXTRAL GPT-4 good contextual results ⚠ data privacy ⚠

    transparency ⚠ e iciency easy to use & configure fast prototyping LLM
  8. P rototype task-specific output 💬 prompt 📖 text LLM prompt

    model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API
  9. 📖 text task-specific output P roduction P rototype task-specific output

    💬 prompt 📖 text LLM prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API
  10. 📖 text task-specific output P roduction P rototype task-specific output

    💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm GPT-4 API
  11. 📖 text task-specific output P roduction P rototype task-specific output

    💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm ✅ modular GPT-4 API
  12. 📖 text task-specific output P roduction P rototype task-specific output

    💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm ✅ small & fast ✅ modular GPT-4 API
  13. 📖 text task-specific output P roduction P rototype task-specific output

    💬 prompt 📖 text LLM distilled task-specific components prompt model & transform output to structured data github.com/explosion/spacy-llm ✅ data-private ✅ small & fast ✅ modular GPT-4 API
  14. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  15. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes • high-security environment 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  16. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  17. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  18. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  19. Case Stud y : S&P Global 99% 99% • real-time

    commodities trading insights by extracting structured attributes • high-security environment • used LLM during annotation • 10× data development speedup with humans and model in the loop • 8 market pipelines in production 6mb 6mb model size 16k+ 16k+ words/second F-score explosion.ai/blog/sp-global-commodities
  20. Software 1.0 Software 1.0 📄 code 💾 program compiler Software

    2.0 Software 2.0 📊 data 🔮 model algorithm
  21. Software 1.0 Software 1.0 📄 code 💾 program compiler Software

    2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation
  22. Software 1.0 Software 1.0 📄 code 💾 program compiler Software

    2.0 Software 2.0 📊 data 🔮 model algorithm ✅ tests 📈 evaluation refactoring refactoring iteration iteration
  23. I lo v e cats. SIMILAR OR NOT? I ha

    t e cats. Your application context always matters!
  24. Case Stud y : GitLab 1 year 1 year 6×

    • extract actionable insights from support tickets and usage questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights
  25. Case Stud y : GitLab 1 year 1 year 6×

    • extract actionable insights from support tickets and usage questions • high-security environment 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights
  26. Case Stud y : GitLab 1 year 1 year 6×

    • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights
  27. Case Stud y : GitLab 1 year 1 year 6×

    • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights
  28. Case Stud y : GitLab 1 year 1 year 6×

    • extract actionable insights from support tickets and usage questions • high-security environment • easy to adapt to new scenarios and business questions • separated general-purpose features from product-specific logic 6× speedup of support tickets explosion.ai/blog/gitlab-support-insights
  29. The Window K nocking Machine Tes t ines.io/blog/window-knocking-machine-test Are you

    designing a window-knocking machine or an alarm clock? “knocker-uppers”
  30. Hello, I ’ m Toni ’ s virtual assistant and

    I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. ines.io/blog/window-knocking-machine-test
  31. Hello, I ’ m Toni ’ s virtual assistant and

    I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. Calendly ines.io/blog/window-knocking-machine-test
  32. Hello, I ’ m Toni ’ s virtual assistant and

    I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. Calendly “window-knocking machine” “alarm clock” ines.io/blog/window-knocking-machine-test
  33. What ’ s the total services revenue from 2023? $2,923,531

    How many clients is that in total? 29 ⏺ ⏺ ⏺ ines.io/blog/window-knocking-machine-test
  34. What ’ s the total services revenue from 2023? $2,923,531

    How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test
  35. 2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD

    Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test
  36. 2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD

    Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue A I still needs produc t decisions! Kim Miller Analyst What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation ines.io/blog/window-knocking-machine-test
  37. Reason and refactor. The key to success lies in your

    data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI
  38. Reason and refactor. The key to success lies in your

    data and may surprise you! Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.
  39. Reason and refactor. The key to success lies in your

    data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Summar y APPLIED NLP & GEN AI APPLIED NLP & GEN AI Think beyond chat bots. You don’t want to build a “window-knocking machine”.