Unstructured Data is the New Gold: ML + NLP is the New Shiz!

Machine Learning (ML) is a key to develop intelligent systems. It has had some successes not only in data science and engineering but also in information security domain. While data gathered help in identifying threats, it only accounts for a small part of the whole picture. Using Natural Language Processing (NLP) to make sense of unstructured resources such as social media posts, online news articles and blog posts is what gives us the edge over the machines.

Transcript

1. Unstructured data
 is the new gold! Machine Learning and Natural Language Processing
 is the new $h*z! Jim Geovedi
2. Artificial intelligence is already a part of our everyday lives.
3. VIP
9. Research areas • Confidence in the robustness of the decisions taken by machines and in the security of the systems within which they operate. • Ability to understand how machine learning system works and improving transparency and interpretability. • Better interaction between human and computers. • Better handling real world biases and messiness in data.
12. Machines can process huge data and extract complex patterns, but without a human to guide them, they only produce garbage.
13. Software that creates machine- processable structure can utilize the linguistic, auditory, and visual structure that exist in all forms of human communication.
14. Natural Language Processing
15. NLP application areas • Finding relationship accross text data and structured metadata. • Identifying underlying intents behind customer comments. • Classifying, labeling, and routing messages. • Surfacing and tracking trends in unstructured data. • Emotion analysis.
16. Natural Language Processing Semantics Lexical semantics Machine translation Named entity recognition Natural language generation Natural language understanding Optical character recognition Question answering Recognizing textual entailment Relationship extraction Sentiment analysis Topic modelling Word sense disambiguation Syntax Morphological segmentation Part of speech tagging Parsing Stemming Lemmatization Sentence breaking Word segmentation Terminology extraction Discourse Automatic summarization Coreference resolution Discourse Analysis Speech Speech recognition Speech segmentation Text to speech
17. Natural Language Processing Semantics Lexical semantics Machine translation Named entity recognition Natural language generation Natural language understanding Optical character recognition Question answering Recognizing textual entailment Relationship extraction Sentiment analysis Topic modelling Word sense disambiguation Syntax Morphological segmentation Part of speech tagging Parsing Stemming Lemmatization Sentence breaking Word segmentation Terminology extraction Discourse Automatic summarization Coreference resolution Discourse Analysis Speech Speech recognition Speech segmentation Text to speech
18. Natural Language Processing Semantics Lexical semantics Machine translation Named entity recognition Natural language generation Natural language understanding Optical character recognition Question answering Recognizing textual entailment Relationship extraction Sentiment analysis Topic modelling Word sense disambiguation Syntax Morphological segmentation Part of speech tagging Parsing Stemming Lemmatization Sentence breaking Word segmentation Terminology extraction Discourse Automatic summarization Coreference resolution Discourse Analysis Speech Speech recognition Speech segmentation Text to speech
19. Natural Language Processing Semantics Lexical semantics Machine translation Named entity recognition Natural language generation Natural language understanding Optical character recognition Question answering Recognizing textual entailment Relationship extraction Sentiment analysis Topic modelling Word sense disambiguation Syntax Morphological segmentation Part of speech tagging Parsing Stemming Lemmatization Sentence breaking Word segmentation Terminology extraction Discourse Automatic summarization Coreference resolution Discourse Analysis Speech Speech recognition Speech segmentation Text to speech
20. Natural Language Processing Semantics Lexical semantics Machine translation Named entity recognition Natural language generation Natural language understanding Optical character recognition Question answering Recognizing textual entailment Relationship extraction Sentiment analysis Topic modelling Word sense disambiguation Syntax Morphological segmentation Part of speech tagging Parsing Stemming Lemmatization Sentence breaking Word segmentation Terminology extraction Discourse Automatic summarization Coreference resolution Discourse Analysis Speech Speech recognition Speech segmentation Text to speech
26. Natural Language Processing 
 x
 Deep Learning
37. Theresa May PERSON was accused of a climbdown over the future sovereignty of British GPE courts after a newly published government paper appeared to leave open the possibility that the European GPE court of justice would influence UK GPE law after Brexit. The latest of a flurry of Brexit policy papers, due to be published on Wednesday DATE , will repeat the government’s insistence that the “direct jurisdiction” of the Luxembourg GPE -based ECJ must end when Britain GPE leaves the EU GPE in March 2019 DATE . But it will set out a range of options for resolving future disputes between Britain GPE and the EU GPE – over the rules of any new trade deal, for example – some of which are likely to involve European GPE judges, or the application of ECJ case law.
40. Tools of trade • General library: NLTK, spaCy, CoreNLP, OpenNLP, ClearNLP, gensim • Dependency parser: Maltparser, spaCy, SyntaxNet • Machine learning: Scikit learn, Liblinear, LibSVM, CRF++, Weka • Deep learning: Theano, Tensorflow, Torch/Pytorch, Dynet • Language modeling: IRSTLM, SRILM, KenLM • Machine translation: • Alignment: Berkeley aligner, GIZA++ • StatMT: Moses, Joshua, cdec, travatar • NMT: seq2seq, OpenNMT/OpenNMT-py, fairseq, sockeye, Lamtram, Marian • Services: Amazon Web Service, Google Cloud, Microsoft Azure, IBM Bluemix
41. Bradford Cross, Five AI Startup Predictions for 2017 “Machine Learning as a Service is an idea we’ve been seeing for nearly 10 years and it’s been failing the whole time. The bottom line on why it doesn’t work: the people that know what they’re doing just use open source, and the people that don’t will not get anything to work, ever, even with APIs.”
48. Q?