The AI Revolution Will Not Be Monopolized: How open-source beats economies of scale, even for LLMs

With the latest advancements in Natural Language Processing and Large Language Models (LLMs), and big companies like OpenAI dominating the space, many people wonder: Are we heading further into a black box era with larger and larger models, obscured behind APIs controlled by big tech monopolies?

I don’t think so, and in this talk, I’ll show you why. I’ll dive deeper into the open-source model ecosystem, some common misconceptions about use cases for LLMs in industry, practical real-world examples and how basic principles of software development such as modularity, testability and flexibility still apply. LLMs are a great new tool in our toolkits, but the end goal remains to create a system that does what you want it to do. Explicit is still better than implicit, and composable building blocks still beat huge black boxes.

As ideas develop, we’re seeing more and more ways to use compute efficiently, producing AI systems that are cheaper to run and easier to control. In this talk, I'll share some practical approaches that you can apply today. If you’re trying to build a system that does a particular thing, you don’t need to transform your request into arbitrary language and call into the largest model that understands arbitrary language the best. The people developing those models are telling that story, but the rest of us aren’t obliged to believe them.

Transcript

1. Ines Montani Explosion
2. Open-source library for industrial-strength natural language processing spacy.io SPACY 200m+ downloads
3. Open-source library for industrial-strength natural language processing spacy.io SPACY ChatGPT can write spaCy code! 200m+ downloads
4. Modern scriptable annotation tool for machine learning developers prodigy.ai PRODIGY 9k+ 800+ users companies
5. Modern scriptable annotation tool for machine learning developers prodigy.ai PRODIGY 9k+ 800+ users companies Alex Smith Developer Kim Miller Analyst
6. Collaborative data development platform prodigy.ai/teams PRODIGY TEAMS BETA
7. Collaborative data development platform Alex Smith Developer Kim Miller Analyst GPT-4 API prodigy.ai/teams PRODIGY TEAMS BETA
8. WHY OPEN SOURCE?
9. WHY OPEN SOURCE? transparent
10. WHY OPEN SOURCE? transparent no lock-in
11. WHY OPEN SOURCE? transparent no lock-in extensible
12. WHY OPEN SOURCE? transparent no lock-in extensible runs in-house
13. WHY OPEN SOURCE? transparent no lock-in extensible runs in-house easy to get started
14. WHY OPEN SOURCE? transparent no lock-in extensible community-vetted runs in-house easy to get started
15. WHY OPEN SOURCE? transparent no lock-in programmable extensible community-vetted runs in-house easy to get started
16. WHY OPEN SOURCE? transparent no lock-in up to date programmable extensible community-vetted runs in-house easy to get started
17. WHY OPEN SOURCE? transparent no lock-in up to date programmable extensible community-vetted runs in-house easy to get started also free!
18. OPEN-SOURCE MODELS
19. task-specific models OPEN-SOURCE MODELS
20. task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune OPEN-SOURCE MODELS
21. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune OPEN-SOURCE MODELS
22. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune OPEN-SOURCE MODELS
23. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune relatively small and fast, a ordable to run, generalize & adapt well, need data to fine-tune OPEN-SOURCE MODELS
24. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune relatively small and fast, a ordable to run, generalize & adapt well, need data to fine-tune OPEN-SOURCE MODELS large generative models Falcon MIXTRAL
25. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune relatively small and fast, a ordable to run, generalize & adapt well, need data to fine-tune OPEN-SOURCE MODELS large generative models Falcon MIXTRAL very large, often slower, expensive to run, generalize & adapt well, need little to no data
26. encoder models large generative models ENCODING & DECODING TASKS
27. encoder models large generative models ENCODING & DECODING TASKS network trained for specific tasks using model to encode input 🔮 model 📖 text vectors 🔮 task model task output 🧬 task network labels
28. encoder models large generative models ENCODING & DECODING TASKS model generates text that can be parsed into task-specific output 📖 text 🔮 model raw output ⚙ parser task output 💬 template prompt network trained for specific tasks using model to encode input 🔮 model 📖 text vectors 🔮 task model task output 🧬 task network labels
29. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune relatively small and fast, a ordable to run, generalize & adapt well, need data to fine-tune OPEN-SOURCE MODELS large generative models Falcon MIXTRAL very large, often slower, expensive to run, generalize & adapt well, need little to no data
30. encoder models ELECTRA T5 task-specific models small, often fast, cheap to run, don’t always generalize well, need data to fine-tune relatively small and fast, a ordable to run, generalize & adapt well, need data to fine-tune OPEN-SOURCE MODELS large generative models Falcon MIXTRAL very large, often slower, expensive to run, generalize & adapt well, need little to no data
31. output costs ECONOMIES OF SCALE
32. output costs OpenAI Google ECONOMIES OF SCALE
33. output costs OpenAI Google ECONOMIES OF SCALE access to talent, compute etc.
34. output costs OpenAI Google ECONOMIES OF SCALE access to talent, compute etc. API request batching
35. output costs OpenAI Google ECONOMIES OF SCALE access to talent, compute etc. API request batching high tra ff ic 💧 💧 💧 💧 💧 💧 💧 💧 low tra ff ic batch 💧 💧 💧 💧 💧 💧 💧 💧 …
36. output costs OpenAI Google you 🤠 ECONOMIES OF SCALE access to talent, compute etc. API request batching high tra ff ic 💧 💧 💧 💧 💧 💧 💧 💧 low tra ff ic batch 💧 💧 💧 💧 💧 💧 💧 💧 …
37. human-facing systems machine-facing models ChatGPT GPT-4 AI PRODUCTS ARE MORE THAN JUST A MODEL
38. human-facing systems machine-facing models ChatGPT GPT-4 most important di erentiation is product, not just technology AI PRODUCTS ARE MORE THAN JUST A MODEL
39. human-facing systems machine-facing models ChatGPT GPT-4 most important di erentiation is product, not just technology UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL
40. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL
41. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology cost speed accuracy latency UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL
42. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology cost speed accuracy latency UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL But what about the data?
43. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology cost speed accuracy latency UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL But what about the data? User data is an advantage for product, not the foundation for machine-facing tasks.
44. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology cost speed accuracy latency UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL But what about the data? User data is an advantage for product, not the foundation for machine-facing tasks. You don’t need specific data to gain general knowledge.
45. human-facing systems machine-facing models ChatGPT GPT-4 swappable components based on research, impacts are quantifiable most important di erentiation is product, not just technology cost speed accuracy latency UI / UX marketing customization AI PRODUCTS ARE MORE THAN JUST A MODEL But what about the data? User data is an advantage for product, not the foundation for machine-facing tasks. You don’t need specific data to gain general knowledge.
46. USE CASES IN INDUSTRY predictive tasks 🔖 entity recognition 🔗 relation extraction 👫 coreference resolution 🧬 grammar & morphology 🎯 semantic parsing 💬 discourse structure 📚 text classification generative tasks 📖 single/multi-doc summarization 🧮 reasoning ✅ problem solving ✍ paraphrasing 🖼 style transfer ⁉ question answering
47. USE CASES IN INDUSTRY predictive tasks 🔖 entity recognition 🔗 relation extraction 👫 coreference resolution 🧬 grammar & morphology 🎯 semantic parsing 💬 discourse structure 📚 text classification generative tasks 📖 single/multi-doc summarization 🧮 reasoning ✅ problem solving ✍ paraphrasing 🖼 style transfer ⁉ question answering many industry problems have remained the same, they just changed in scale structured data
48. EVOLUTION OF PROBLEM DEFINITIONS
49. rules or instructions ✍ EVOLUTION OF PROBLEM DEFINITIONS
50. programming & rules rules or instructions ✍ EVOLUTION OF PROBLEM DEFINITIONS
51. programming & rules rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS
52. supervised learning programming & rules rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS
53. supervised learning programming & rules rules or instructions ✍ in-context learning rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS
54. supervised learning prompt engineering programming & rules rules or instructions ✍ in-context learning rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS
55. supervised learning prompt engineering programming & rules rules or instructions ✍ in-context learning rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS instructions: human-shaped, easy for non-experts, risk of data drift ✍
56. supervised learning prompt engineering programming & rules rules or instructions ✍ in-context learning rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive
57. supervised learning prompt engineering programming & rules rules or instructions ✍ in-context learning rules or instructions ✍ machine learning examples 📝 EVOLUTION OF PROBLEM DEFINITIONS instructions: human-shaped, easy for non-experts, risk of data drift ✍ 📝 examples: nuanced and intuitive behaviors, specific to use case, labor-intensive
58. large general- purpose model domain- specific data WORKFLOW EXAMPLE
59. prompting large general- purpose model domain- specific data WORKFLOW EXAMPLE
60. prompting large general- purpose model continuous evaluation baseline domain- specific data WORKFLOW EXAMPLE
61. prompting large general- purpose model continuous evaluation baseline domain- specific data WORKFLOW EXAMPLE iterative model-assisted data annotation prodigy.ai
62. prompting large general- purpose model continuous evaluation baseline domain- specific data WORKFLOW EXAMPLE iterative model-assisted data annotation prodigy.ai
63. prompting large general- purpose model distilled task- specific model transfer learning continuous evaluation baseline domain- specific data WORKFLOW EXAMPLE iterative model-assisted data annotation prodigy.ai
64. prompting large general- purpose model distilled task- specific model transfer learning continuous evaluation baseline distilled model domain- specific data WORKFLOW EXAMPLE iterative model-assisted data annotation prodigy.ai
65. processing pipeline prototype PROTOTYPE TO PRODUCTION
66. github.com/explosion/spacy-llm prompt model & transform output to structured data processing pipeline prototype PROTOTYPE TO PRODUCTION
67. processing pipeline in production swap, replace and mix components github.com/explosion/spacy-llm prompt model & transform output to structured data processing pipeline prototype PROTOTYPE TO PRODUCTION
68. processing pipeline in production swap, replace and mix components github.com/explosion/spacy-llm prompt model & transform output to structured data processing pipeline prototype PROTOTYPE TO PRODUCTION
69. processing pipeline in production swap, replace and mix components github.com/explosion/spacy-llm prompt model & transform output to structured data structured machine-facing Doc object processing pipeline prototype PROTOTYPE TO PRODUCTION
70. DISTILLED TASK-SPECIFIC COMPONENTS
71. modular DISTILLED TASK-SPECIFIC COMPONENTS
72. modular no lock-in DISTILLED TASK-SPECIFIC COMPONENTS
73. modular testable no lock-in DISTILLED TASK-SPECIFIC COMPONENTS
74. modular testable no lock-in extensible DISTILLED TASK-SPECIFIC COMPONENTS
75. modular testable flexible no lock-in extensible DISTILLED TASK-SPECIFIC COMPONENTS
76. modular testable flexible no lock-in extensible cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
77. modular testable flexible no lock-in extensible run in-house cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
78. modular testable flexible no lock-in programmable extensible run in-house cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
79. modular testable flexible predictable no lock-in programmable extensible run in-house cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
80. modular testable flexible predictable transparent no lock-in programmable extensible run in-house cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
81. modular testable flexible predictable transparent no lock-in programmable extensible run in-house cheap to run DISTILLED TASK-SPECIFIC COMPONENTS
82. The Zen of Python >>> import this Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess.
83. The Zen of Python >>> import this don’t abandon what’s made software successful Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren't special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess.
84. control resource regulation compounding economies of scale network e ects MONOPOLY STRATEGIES
85. control resource regulation compounding economies of scale network e ects MONOPOLY STRATEGIES
86. control resource regulation compounding economies of scale network e ects MONOPOLY STRATEGIES
87. control resource regulation compounding economies of scale network e ects MONOPOLY STRATEGIES human-facing products vs. machine-facing models
88. control resource regulation compounding economies of scale network e ects MONOPOLY STRATEGIES human-facing products vs. machine-facing models
89. THE AI REVOLUTION WON’T BE MONOPOLIZED
90. THE AI REVOLUTION WON’T BE MONOPOLIZED The software industry does not run on secret sauce. Knowledge gets shared and published. Secrets won’t give anyone a monopoly.
91. THE AI REVOLUTION WON’T BE MONOPOLIZED The software industry does not run on secret sauce. Knowledge gets shared and published. Secrets won’t give anyone a monopoly. Usage data is great for improving a product, but it doesn’t generalize. Data won’t give anyone a monopoly.
92. THE AI REVOLUTION WON’T BE MONOPOLIZED The software industry does not run on secret sauce. Knowledge gets shared and published. Secrets won’t give anyone a monopoly. LLMs can be one part of a product or process, and swapped for di erent approaches. Interoperability is the opposite of monopoly. Usage data is great for improving a product, but it doesn’t generalize. Data won’t give anyone a monopoly.
93. THE AI REVOLUTION WON’T BE MONOPOLIZED The software industry does not run on secret sauce. Knowledge gets shared and published. Secrets won’t give anyone a monopoly. LLMs can be one part of a product or process, and swapped for di erent approaches. Interoperability is the opposite of monopoly. Usage data is great for improving a product, but it doesn’t generalize. Data won’t give anyone a monopoly. Regulation could give someone a monopoly, if we let it. It should focus on products and actions, not components.
94. Explosion spaCy Prodigy Twitter Mastodon Bluesky explosion.ai spacy.io prodigy.ai @_inesmontani @ines@sigmoid.social @inesmontani.bsky.social LinkedIn