Applied NLP in the Age of Generative AI: Future-Proof Strategies for Banking and Finance

Large Language Models (LLMs) and in-context learning have introduced a new paradigm for developing natural language understanding systems: prompts are all you need! Prototyping has never been easier, but not all prototypes give a smooth path to production. Many new ideas that are emerging also challenge existing workflows in industry that require modularity, transparency and data privacy. In this talk, I'll share the most important lessons we've learned from solving real-world information extraction problems in industry, and show you a new approach and mindset for building modular and future-proof NLP pipelines in-house.

Transcript

1. Ines Montani Explosion Future-Proof Strategies for Banking & Finance
2. Open-source library for industrial- strength natural language processing spacy.io 390m+ downloads
3. Generative AI can write spaCy code! Open-source library for industrial- strength natural language processing spacy.io 390m+ downloads
4. Modern scriptable annotation tool for machine learning developers prodigy.ai 900+ companies 10k+ users
5. Modern scriptable annotation tool for machine learning developers prodigy.ai Alex Smith Developer Kim Miller Analyst GPT-4 API 900+ companies 10k+ users
6. MIXTRAL GPT-4 LLM
7. MIXTRAL GPT-4 good contextual results LLM
8. MIXTRAL GPT-4 good contextual results easy to use & configure LLM
9. MIXTRAL GPT-4 good contextual results easy to use & configure fast prototyping LLM
10. MIXTRAL GPT-4 good contextual results ⚠ transparency easy to use & configure fast prototyping LLM
11. MIXTRAL GPT-4 good contextual results ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM
12. MIXTRAL GPT-4 good contextual results ⚠ data privacy ⚠ transparency ⚠ e iciency easy to use & configure fast prototyping LLM
13. Language is just another interface.
15. “knocker-uppers”
16. “knocker-uppers” ines.io/blog/window-knocking-machine-test
17. Are you designing a window-knocking machine or an alarm clock? “knocker-uppers” ines.io/blog/window-knocking-machine-test
18. Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. ines.io/blog/window-knocking-machine-test
19. Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. Calendly ines.io/blog/window-knocking-machine-test
20. Hello, I ’ m Toni ’ s virtual assistant and I help schedule appointments. Do you have time at 1pm on Monday? No, but Tuesday would work for me. Okay, please confirm: Tuesday at 1pm? 1pm is unideal but 3pm would work. Toni doesn ’ t have availability at 3pm but I could offer a slot at 4pm or 5 : 30pm. Which time zone is this by the way? I ’ m in CET. Calendly “window-knocking machine” “alarm clock” ines.io/blog/window-knocking-machine-test
21. ines.io/blog/window-knocking-machine-test What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺
22. ines.io/blog/window-knocking-machine-test What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation
23. ines.io/blog/window-knocking-machine-test 2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation
24. ines.io/blog/window-knocking-machine-test 2023 Year Services Type ACME Inc. FooBar GmbH NLPCorp XKCD Ltd. Python AG 432,032 82,000 1,500 193,000 91,320 $ 2,625,032 Clients (28) Revenue AI still needs product decisions! What ’ s the total services revenue from 2023? $2,923,531 How many clients is that in total? 29 ⏺ ⏺ ⏺ 🔮 LLM 📚 database 🤖 agents ⚙ query Retrieval-Augmented Generation
26. explosion.ai/blog/history-web-future-ai static pages WEB
27. dynamic pages explosion.ai/blog/history-web-future-ai static pages WEB
28. dynamic pages static pages explosion.ai/blog/history-web-future-ai static pages WEB
29. dynamic pages static pages compile static data at build time explosion.ai/blog/history-web-future-ai static pages WEB
30. dynamic pages static pages compile static data at build time custom models AI explosion.ai/blog/history-web-future-ai static pages WEB
31. dynamic pages static pages pretrained models compile static data at build time custom models AI explosion.ai/blog/history-web-future-ai static pages WEB
32. dynamic pages static pages custom models pretrained models compile static data at build time custom models AI explosion.ai/blog/history-web-future-ai static pages WEB
33. dynamic pages static pages custom models distill models into smaller, faster and private components pretrained models compile static data at build time custom models AI explosion.ai/blog/history-web-future-ai static pages WEB
34. explosion.ai/blog/sp-global-commodities 99% F-score 6mb model size 16k+ words/second • S&P Global: real-time commodities trading insights by extracting structured attributes
35. explosion.ai/blog/sp-global-commodities 99% F-score 6mb model size 16k+ words/second • S&P Global: real-time commodities trading insights by extracting structured attributes • high-security environment
36. explosion.ai/blog/sp-global-commodities 99% F-score 6mb model size 16k+ words/second • S&P Global: real-time commodities trading insights by extracting structured attributes • high-security environment • 10× data development speedup with humans and LLM in the loop
37. explosion.ai/blog/sp-global-commodities 99% F-score 6mb model size 16k+ words/second • S&P Global: real-time commodities trading insights by extracting structured attributes • high-security environment • 10× data development speedup with humans and LLM in the loop • 8+ market pipelines in production
38. explosion.ai/blog/sp-global-commodities 99% F-score 6mb model size 16k+ words/second • S&P Global: real-time commodities trading insights by extracting structured attributes • high-security environment • 10× data development speedup with humans and LLM in the loop • 8+ market pipelines in production
39. explosion.ai/blog/sp-global-commodities “heards”: trading activities
40. explosion.ai/blog/sp-global-commodities “heards”: trading activities
41. explosion.ai/blog/sp-global-commodities ⚠ data-private “heards”: trading activities
42. explosion.ai/blog/sp-global-commodities ⚠ data-private ⚠ real-time “heards”: trading activities
43. 📖 text 🏷 entities explosion.ai/blog/sp-global-commodities ⚠ data-private ⚠ real-time “heards”: trading activities
44. 📖 text 🏷 entities explosion.ai/blog/sp-global-commodities ⚠ data-private ⚠ real-time rules “heards”: trading activities
45. 📖 text 🏷 entities explosion.ai/blog/sp-global-commodities ⚠ data-private ⚠ real-time rules model + “heards”: trading activities
46. explosion.ai/blog/sp-global-commodities
47. explosion.ai/blog/sp-global-commodities 🧑💻 human experts in the loop
48. explosion.ai/blog/sp-global-commodities 🧑💻 human experts in the loop 📚 task-specific data ➕
49. explosion.ai/blog/sp-global-commodities 🧑💻 human experts in the loop 📦 model package 📚 task-specific data ➕
50. explosion.ai/blog/sp-global-commodities 🧑💻 human experts in the loop 🚀 structured data 📦 model package 📚 task-specific data ➕
51. explosion.ai/blog/sp-global-commodities 🧑💻 human experts in the loop 🚀 structured data 📦 model package 📚 task-specific data ➕ 🔮 suggestions from LLM
52. explosion.ai/blog/human-in-the-loop-distillation LLM
53. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM
54. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting
55. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting
56. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning COM PONENT
57. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning COM PONENT distilled model
58. explosion.ai/blog/human-in-the-loop-distillation continuous evaluation baseline LLM prompting transfer learning COM PONENT distilled model deploy 🚀
59. explosion.ai/blog/sp-global-commodities
60. explosion.ai/blog/sp-global-commodities
61. explosion.ai/blog/sp-global-commodities
62. explosion.ai/blog/sp-global-commodities 30min per attribute GPT-4 API
63. explosion.ai/blog/sp-global-commodities 10× faster reduce cognitive load 30min per attribute GPT-4 API
65. 🧑💻 developer tooling
66. 🧑💻 developer tooling 🧡 open source
67. 🧑💻 developer tooling 🧡 open source 🤖 development support
68. 🧑💻 developer tooling 🧡 open source 👩🔬 subject matter experts 🤖 development support
69. 🧑💻 developer tooling 🧡 open source 👩🔬 subject matter experts 🔄 iteration 🤖 development support
70. 🧑💻 developer tooling 🧡 open source 👩🔬 subject matter experts 🔄 iteration 🤖 development support 🛠 refactoring
71. 🧑💻 developer tooling 🧡 open source 👩🔬 subject matter experts 🔄 iteration 🤖 development support 🛠 refactoring 🏢 in-house development
72. 🧑💻 developer tooling 🧡 open source 👩🔬 subject matter experts 🔄 iteration 🤖 development support 🛠 refactoring 🧠 mindset 🏢 in-house development
73. explosion.ai/blog/pdfs-nlp-structured-data
74. explosion.ai/blog/pdfs-nlp-structured-data Businesses want electronic copies that map 1:1 to paper documents.
75. explosion.ai/blog/pdfs-nlp-structured-data Businesses want electronic copies that map 1:1 to paper documents.
76. ⚠ PDFs are a bad “source of truth” explosion.ai/blog/pdfs-nlp-structured-data Businesses want electronic copies that map 1:1 to paper documents.
77. github.com/explosion/spacy-layout
78. github.com/explosion/spacy-layout spaCy + Docling
79. github.com/explosion/spacy-layout spaCy + Docling
80. github.com/explosion/spacy-layout process and create a spaCy Doc spaCy + Docling
81. github.com/explosion/spacy-layout text-based contents process and create a spaCy Doc spaCy + Docling
82. github.com/explosion/spacy-layout text-based contents document layout process and create a spaCy Doc spaCy + Docling
83. github.com/explosion/spacy-layout text-based contents document layout process and create a spaCy Doc spaCy + Docling
84. github.com/explosion/spacy-layout text-based contents layout sections document layout process and create a spaCy Doc spaCy + Docling
85. github.com/explosion/spacy-layout text-based contents layout sections document layout content, tokens, o sets process and create a spaCy Doc spaCy + Docling
86. github.com/explosion/spacy-layout text-based contents layout sections section type document layout content, tokens, o sets process and create a spaCy Doc spaCy + Docling
87. github.com/explosion/spacy-layout text-based contents layout sections section type document layout bounding box content, tokens, o sets process and create a spaCy Doc spaCy + Docling
88. github.com/explosion/spacy-layout text-based contents layout sections section type document layout bounding box content, tokens, o sets process and create a spaCy Doc spaCy + Docling annotation in context
89. At their core, many NLP systems consist of flat classifications. You can shove them into a single prompt, or you can decompose them into smaller pieces. Many classification tasks are straightforward to solve nowadays – but they become vastly more complicated if one model needs to do them all at once. explosion.ai/blog/human-in-the-loop-distillation
91. Reason and refactor. The key to success lies in your data and may surprise you!
92. Reason and refactor. The key to success lies in your data and may surprise you! Think beyond chat bots. You don’t want to build a “window-knocking machine”.
93. Reason and refactor. The key to success lies in your data and may surprise you! LLM Stay ambitious. Don’t compromise on best practices, e iciency and privacy. Think beyond chat bots. You don’t want to build a “window-knocking machine”.
94. Explosion spaCy Prodigy Bluesky Mastodon explosion.ai spacy.io prodigy.ai @inesmontani.bsky.social @ines@sigmoid.social LinkedIn