2021-02-01-rel_video

Creating a custom trainable component with spaCy: an example use-case for Relation Extraction

Slides to support this tutorial video: https://www.youtube.com/watch?v=8HL-Ap5_Axo

1. v3 So fi e Van Landeghem FOR NAMED ENTITY Trainable Component Relation Extraction CUSTOM
2. spacy.io
3. Production-ready training system, model packaging & workflow management spacy.io
4. Models written in any framework Production-ready training system, model packaging & workflow management spacy.io
5. Models written in any framework Multi-task learning with transformers like BERT Production-ready training system, model packaging & workflow management spacy.io
6. Models written in any framework Multi-task learning with transformers like BERT Production-ready training system, model packaging & workflow management Fully custom trainable pipeline components spacy.io
8. named entity
9. named entity semantic relationship
11. gene or gene product
12. gene or gene product positive regulation
13. Document Machine Learning model Predictions matrix Doc Text ner rel
14. Document Machine Learning model Predictions matrix Doc Text ner rel step #1: implement model
15. Document Machine Learning model Predictions matrix Doc Text ner rel step #1: implement model step #2: implement pipeline component
16. Document Machine Learning model Predictions matrix Doc Text ner rel step #1: implement model step #3:enhance accuracy transformer step #2: implement pipeline component
17. thinc.ai type-checked functional programming API for composing models
18. thinc.ai type-checked functional programming API for composing models wrap layers defined in any framework
19. Document GATA3 inhibits FOXP3 expression
20. Document GATA3 inhibits FOXP3 expression Tokens + NER [GATA3, inhibits, FOXP3, expression]
21. Document GATA3 inhibits FOXP3 expression Tokens + NER [GATA3, inhibits, FOXP3, expression] Token vectors [[-0.42, 1.93, -1.08, 0.28, -0.71] [ 3.84, 2.59, -0.14, -3.77, -0.66] [ 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.77, -2.17, -0.48, -1.73, 1.10]]
22. Document GATA3 inhibits FOXP3 expression Tokens + NER [GATA3, inhibits, FOXP3, expression] Token vectors [[-0.42, 1.93, -1.08, 0.28, -0.71] [ 3.84, 2.59, -0.14, -3.77, -0.66] [ 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.77, -2.17, -0.48, -1.73, 1.10]] Instance 1 Instance 2 GATA3 -> FOXP3 [-0.42, 1.93, -1.08, 0.28, -0.71, 3.35, -1.51, 1.23, -0.88, -2.19] FOXP3 -> GATA3 [ 3.35, -1.51, 1.23, -0.88, -2.19, -0.42, 1.93, -1.08, 0.28, -0.71]
23. [GATA3, inhibits, FOXP3, expression] Instance data [[-0.42, 1.93, -1.08, 0.28, -0.71, 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.35, -1.51, 1.23, -0.88, -2.19, -0.42, 1.93, -1.08, 0.28, -0.71]]
24. [GATA3, inhibits, FOXP3, expression] Instance data [[-0.42, 1.93, -1.08, 0.28, -0.71, 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.35, -1.51, 1.23, -0.88, -2.19, -0.42, 1.93, -1.08, 0.28, -0.71]] Classi fi cation layer Relation types: BINDING ACTIVATION INHIBITION
25. [GATA3, inhibits, FOXP3, expression] Instance data [[-0.42, 1.93, -1.08, 0.28, -0.71, 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.35, -1.51, 1.23, -0.88, -2.19, -0.42, 1.93, -1.08, 0.28, -0.71]] Classi fi cation layer Relation types: BINDING ACTIVATION INHIBITION Predictions [[ 0.09, 0.14, 0.93 ] [ 0.11, 0.15, 0.31 ]] BINDING ACTIVATION INHIBITION
26. [GATA3, inhibits, FOXP3, expression] Instance data [[-0.42, 1.93, -1.08, 0.28, -0.71, 3.35, -1.51, 1.23, -0.88, -2.19] [ 3.35, -1.51, 1.23, -0.88, -2.19, -0.42, 1.93, -1.08, 0.28, -0.71]] Classi fi cation layer Relation types: BINDING ACTIVATION INHIBITION GATA3 -> FOXP3 BINDING: False, ACTIVATION: False, INHIBITION: True Instance 1 Instance 2 FOXP3 -> GATA3 BINDING: False, ACTIVATION: False, INHIBITION: False Predictions [[ 0.09, 0.14, 0.93 ] [ 0.11, 0.15, 0.31 ]] BINDING ACTIVATION INHIBITION
27. Documents List[Doc]
28. Documents List[Doc] Token vectors List[Floats2d] tok2vec
29. Documents List[Doc] Token vectors List[Floats2d] tok2vec Entity vectors List[Floats2d] pooling
30. Documents List[Doc] Token vectors List[Floats2d] tok2vec Entity vectors List[Floats2d] pooling Candidate instances List[Tuple[Span, Span]] get_instances
31. create_instance_tensor Instance tensor Floats2d Documents List[Doc] Token vectors List[Floats2d] tok2vec Entity vectors List[Floats2d] pooling Candidate instances List[Tuple[Span, Span]] get_instances
32. create_instance_tensor Instance tensor Floats2d Documents List[Doc] Predictions matrix Floats2d classification layer Token vectors List[Floats2d] tok2vec Entity vectors List[Floats2d] pooling Candidate instances List[Tuple[Span, Span]] get_instances
33. Document TGF-beta signalling induces Id2
34. Document TGF-beta signalling induces Id2 Tokens + NER [TGF, -, beta, signalling, induces, Id2]
35. Document TGF-beta signalling induces Id2 Tokens + NER [TGF, -, beta, signalling, induces, Id2] Token vectors [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-1.27, 2.21, -0.75, 1.07, -0.48] [-1.03, 0.94, 1.64, -0.05, -0.98] [-0.81, 0.72, -0.52, 0.67, -0.16]]
36. Document TGF-beta signalling induces Id2 Tokens + NER [TGF, -, beta, signalling, induces, Id2] Token vectors [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-1.27, 2.21, -0.75, 1.07, -0.48] [-1.03, 0.94, 1.64, -0.05, -0.98] [-0.81, 0.72, -0.52, 0.67, -0.16]] Entities Ragged [3, 1, 1, 3] Lengths Data [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74]]
37. Document TGF-beta signalling induces Id2 Tokens + NER [TGF, -, beta, signalling, induces, Id2] Token vectors [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-1.27, 2.21, -0.75, 1.07, -0.48] [-1.03, 0.94, 1.64, -0.05, -0.98] [-0.81, 0.72, -0.52, 0.67, -0.16]] Entities Ragged [3, 1, 1, 3] Lengths Data [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74]] Instance 1 Instance 2
38. [TGF, -, beta, signalling, induces, Id2] Entities Ragged [3, 1, 1, 3] Lengths Data [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74]]
39. [TGF, -, beta, signalling, induces, Id2] Entities Ragged [3, 1, 1, 3] Lengths Data [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74]] Pooled entities Floats2d [[-0.60, 0.11, -0.37, 0.17, 0.17] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.60, 0.11, -0.37, 0.17, 0.17]] Instance 1 Instance 2
40. [TGF, -, beta, signalling, induces, Id2] Entities Ragged [3, 1, 1, 3] Lengths Data [[ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [ 1.22, -3.12, -0.19, 0.51, -0.46] [-1.71, 0.92, -0.67, 0.86, 2.70] [-1.32, 2.52, -0.26, -0.86, -1.74]] Instance tensor Floats2d [[-0.60, 0.11, -0.37, 0.17, 0.17, -0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16, -0.60, 0.11, -0.37, 0.17, 0.17]] Pooled entities Floats2d [[-0.60, 0.11, -0.37, 0.17, 0.17] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.81, 0.72, -0.52, 0.67, -0.16] [-0.60, 0.11, -0.37, 0.17, 0.17]] Instance 1 Instance 2
41. optimize model settings for accuracy or efficiency components to train spacy.io/usage/training generate starter config
42. con fi g.cfg structured section describing nlp object
43. con fi g.cfg structured section describing nlp object pipeline component names
44. con fi g.cfg structured section defining components
45. con fi g.cfg structured section defining components factory function used to create component
46. con fi g.cfg structured section defining components factory function used to create component registered function to create model architecture
47. con fi g.cfg structured section defining components factory function used to create component registered function to create model architecture function arguments
48. con fi g.cfg custom factory
49. con fi g.cfg custom factory model architecture
50. con fi g.cfg custom factory sublayers model architecture
51. con fi g.cfg custom factory sublayers model architecture listener layer to connect to tok2vec component
52. spacy.io/usage/layers-architectures
53. predictions reference spacy.io/usage/layers-architectures
55. Entities
56. Entities Entity Relations custom attribute
57. github.com/explosion/projects manage and share end-to-end workflows
58. github.com/explosion/projects manage and share end-to-end workflows clone the project template for this tutorial
59. transformer component spacy.io/usage/embeddings-transformers con fi g.cfg
60. use any pretrained transformer models transformer component spacy.io/usage/embeddings-transformers con fi g.cfg
61. con fi g.cfg spacy.io/usage/embeddings-transformers
62. listener layer to connect to transformer component con fi g.cfg spacy.io/usage/embeddings-transformers
63. spacy.io/usage/v3 @spacy_io install spaCy v3 @OxyKodit
64. spacy.io/usage/v3 @spacy_io documentation and quickstart install spaCy v3 @OxyKodit
65. spacy.io/usage/v3 @spacy_io documentation and quickstart install spaCy v3 clone the project template @OxyKodit
66. spacy.io/usage/v3 @spacy_io documentation and quickstart install spaCy v3 thank you! — clone the project template @OxyKodit