spaCy v3: State-of-the-art NLP from Prototype to Production

Video: https://www.youtube.com/watch?v=9k_EfV7Cns0

spaCy is a popular open-source library for industrial-strength Natural Language Processing in Python. spaCy v3.0 features new transformer-based pipelines that get spaCy’s accuracy right up to the current state-of-the-art, and a new training config and workflow system to help you take projects from prototype to production.

Transcript

1. v3 FROM PROTOTYPE TO State-of-the-art NLP BRINGING Production @_inesmontani @honnibal
2. Open-source library for industrial-strength Natural Language Processing spacy.io
3. Open-source library for industrial-strength Natural Language Processing spacy.io 23m+ DOWNLOADS 17k+ GITHUB STARS
4. spacy.io cheap to run easy to use
5. spacy.io cheap to run easy to use big ecosystem
6. spacy.io cheap to run easy to use big ecosystem v3 now even more powerful
7. State-of-the-art TRANSFORMER-BASED PIPELINES
8. Benchmarks spacy.io/usage/facts- fi gures Parser Tagger NER Speed en_core_web_trf 95.8 98.1 90.6 4k wps en_core_web_lg 92.0 97.2 87.0 30k wps SOTA 96.2 98.3 89.7 - NEW
9. spacy.io/usage/embeddings-transformers Entity Recognizer Dependency Parser Transformer ... packages with state- of-the-art trained pipelines en _core _web _trf
10. spacy.io/usage/embeddings-transformers Entity Recognizer Dependency Parser Transformer ... BERT DistilBERT RoBERTa ... easily use any transformer in your pipeline packages with state- of-the-art trained pipelines en _core _web _trf
11. spacy.io/usage/embeddings-transformers Entity Recognizer Dependency Parser Transformer ... share one transformer across your whole pipeline BERT DistilBERT RoBERTa ... easily use any transformer in your pipeline packages with state- of-the-art trained pipelines en _core _web _trf
12. spacy.io/usage/embeddings-transformers Entity Recognizer Dependency Parser Transformer ... share one transformer across your whole pipeline BERT DistilBERT RoBERTa ... easily use any transformer in your pipeline update one transformer from multiple components packages with state- of-the-art trained pipelines en _core _web _trf
13. Configuration DECLARATIVE SYSTEM
14. CONFIGURE ALL THE THINGS!
15. CONFIGURE ALL THE THINGS! MODEL ARCHITECTURES!
16. CONFIGURE ALL THE THINGS! MODEL ARCHITECTURES! HYPERPARAMETERS!
17. CONFIGURE ALL THE THINGS! MODEL ARCHITECTURES! HYPERPARAMETERS! TRAINING SETTINGS!
18. CONFIGURE ALL THE THINGS! MODEL ARCHITECTURES! HYPERPARAMETERS! TRAINING SETTINGS! LOADING!
19. CONFIGURE ALL THE THINGS! MODEL ARCHITECTURES! HYPERPARAMETERS! TRAINING SETTINGS! LOADING! METRICS!
20. problems.py
21. problem #1: nested defaults become hidden defaults problems.py
22. problem #1: nested defaults become hidden defaults problem #2: defaults will conflict problems.py
23. problem #1: nested defaults become hidden defaults problem #3: difficult to iterate, swap, mix and match problem #2: defaults will conflict problems.py
24. con fi g.cfg
25. con fi g.cfg structured sections
26. con fi g.cfg registered functions structured sections
27. con fi g.cfg registered functions resolved bottom-up structured sections
28. con fi g.cfg registered functions resolved bottom-up variable interpolation structured sections
29. code.py con fi g.cfg
30. code.py con fi g.cfg custom functions
31. code.py con fi g.cfg type-based validation custom functions
32. spacy.io/usage/training
33. spacy.io/usage/training training config
34. custom registered functions and code spacy.io/usage/training training config
35. custom registered functions and code config overrides (data paths) spacy.io/usage/training training config
36. spacy.io/usage/training
37. spacy.io/usage/training generate starter config
38. components to train spacy.io/usage/training generate starter config
39. optimize model settings for accuracy or efficiency components to train spacy.io/usage/training generate starter config
40. step #1: train and package pipeline
41. step #1: train and package pipeline step #2: get pip-installable Python package
42. step #1: train and package pipeline step #2: get pip-installable Python package en_your_pipeline
43. step #1: train and package pipeline step #2: get pip-installable Python package en_your_pipeline step #3: ship and use pipeline
44. Workflows FOR END-TO-END PROJECTS
45. spacy.io/usage/projects
46. spacy.io/usage/projects project CLI
47. spacy.io/usage/projects project CLI
48. project.yml
49. project.yml data assets
50. project.yml data assets named workflows
51. project.yml data assets named workflows commands
52. project.yml data assets named workflows commands file dependencies
53. github.com/explosion/projects
54. con fi g.cfg built-in logger for tracking experiments
55. con fi g.cfg built-in logger for tracking experiments
56. con fi g.cfg built-in logger for tracking experiments log all config settings & discover correlations
57. parallel & distributed training
58. parallel & distributed training asynchronous stochastic gradient descent
59. build interactive spaCy apps
60. build interactive spaCy apps visualizer.py
61. build interactive spaCy apps visualizer.py
62. Components TRAINABLE & RULE-BASED
63. Doc Text
64. Doc Dependency Parser Text
65. Doc Part-of-speech Tagger Dependency Parser Text
66. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Text
67. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Text Categorizer Text
68. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Text Categorizer Entity Linker Text
69. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Lemmatizer Text Categorizer Entity Linker Text
70. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Lemmatizer Text Categorizer Entity Linker Entity Ruler Text
71. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Lemmatizer Text Categorizer Entity Linker Sentence Recognizer Entity Ruler Text
72. Doc Part-of-speech Tagger Dependency Parser Entity Recognizer Lemmatizer Text Categorizer Entity Linker Sentence Recognizer Entity Ruler ... Text
73. con fi g.cfg customize settings, hyperparameters and architectures
74. con fi g.cfg
75. con fi g.cfg source components from trained pipelines
76. con fi g.cfg choose which components to update during training source components from trained pipelines
77. spacy.io/usage/processing-pipelines custom pipeline component
78. spacy.io/usage/processing-pipelines custom pipeline component
79. spacy.io/usage/processing-pipelines custom pipeline component con fi g.cfg
80. custom pipeline component factory
81. con fi g.cfg custom pipeline component factory
82. con fi g.cfg factory function custom pipeline component factory
83. con fi g.cfg factory function factory arguments custom pipeline component factory
84. Any framework CUSTOM MODELS IN
85. thinc.ai type-checked functional programming API for composing models
86. thinc.ai type-checked functional programming API for composing models wrap layers defined in any framework
87. spacy.io/usage/layers-architectures wrap custom PyTorch model con fi g.cfg set parameters in config
88. Type hints AND TYPE-BASED VALIDATION
89. Model
90. Model Model[InputT, OutputT] generic types
91. Model custom array types Floats2d Model[InputT, OutputT] generic types
92. Model custom array types Floats2d Ints1d ... Padded Ragged Model[InputT, OutputT] generic types
94. expected return types
95. Y: Floats3d Incompatible return value type (got "Tuple[Floats3d, Callable[[Any], Any]]", expected return types
96. Y: Floats3d Incompatible return value type (got "Tuple[Floats3d, Callable[[Any], Any]]", expected return types static analysis: catch errors as you type
98. mypy.ini optional mypy plugin for more checks
99. Relu: Relu Layer outputs type (thinc.types.Floats2d) but the next layer expects (thinc.types.Ragged) as an input mypy.ini optional mypy plugin for more checks
100. Relu: Relu Layer outputs type (thinc.types.Floats2d) but the next layer expects (thinc.types.Ragged) as an input static analysis: catch errors as you type mypy.ini optional mypy plugin for more checks
101. spacy.io
102. Base support & trained pipelines for many languages spacy.io
103. Base support & trained pipelines for many languages Multi-task learning with transformers like BERT spacy.io
104. Base support & trained pipelines for many languages Multi-task learning with transformers like BERT State-of-the- art speed spacy.io
105. Base support & trained pipelines for many languages Multi-task learning with transformers like BERT State-of-the- art speed Components for NER, tagging, parsing, text classification, entity linking & more spacy.io
106. Base support & trained pipelines for many languages Multi-task learning with transformers like BERT State-of-the- art speed Custom trainable & rule-based components Components for NER, tagging, parsing, text classification, entity linking & more spacy.io
107. Base support & trained pipelines for many languages Multi-task learning with transformers like BERT State-of-the- art speed Custom trainable & rule-based components Components for NER, tagging, parsing, text classification, entity linking & more Production-ready training system, model packaging & workflow management spacy.io
108. Base support & trained pipelines for many languages Awesome ecosystem Multi-task learning with transformers like BERT State-of-the- art speed Custom trainable & rule-based components Components for NER, tagging, parsing, text classification, entity linking & more Production-ready training system, model packaging & workflow management spacy.io
109. spacy.io/usage/v3 @spacy_io @_inesmontani @honnibal
110. spacy.io/usage/v3 @spacy_io @_inesmontani @honnibal install spaCy v3 from pip or conda
111. spacy.io/usage/v3 @spacy_io @_inesmontani @honnibal documentation and quickstart install spaCy v3 from pip or conda
112. spacy.io/usage/v3 @spacy_io @_inesmontani @honnibal documentation and quickstart install spaCy v3 from pip or conda thank you! —