$30 off During Our Annual Pro Sale. View Details »

Teaching AI about Human Knowledge

Teaching AI about Human Knowledge

Most AI systems today rely on supervised learning: you provide labelled input and output pairs, and get a program that can perform analogous computation for new data. This allows an approach to software engineering Andrej Karpathy has termed "Software 2.0": programming by example data. This is the machine learning revolution that's already here, which we need to be careful to distinguish from more futuristic visions such as Artificial General Intelligence. If "Software 2.0" is driven by example data, how is that example data created – and how can we make that process better?

Ines Montani

May 05, 2018
Tweet

More Decks by Ines Montani

Other Decks in Programming

Transcript

  1. Teaching AI about human knowledge Supervised learning is great —

    it’s 
 data collection that’s broken Ines Montani Explosion AI
  2. Explosion AI is a digital studio specialising in Artificial Intelligence

    and Natural Language Processing. Open-source library for industrial-strength Natural Language Processing spaCy’s next-generation Machine Learning library for deep learning with text Coming soon: pre-trained, customisable models
 for a variety of languages and domains A radically efficient data collection and annotation tool, powered by active learning
  3. Machine Learning is “programming by example” annotations let us specify

    the output we’re looking for draw examples from the same distribution as runtime inputs goal: system’s prediction given some input matches label a human would have assigned
  4. def train_tagger(examples): W = defaultdict(lambda: zeros(n_tags)) for (word, prev, next),

    human_tag in examples: scores = W[word] + W[prev] + W[next] guess = scores.argmax() if guess != human_tag: for feat in (word, prev, next): W[feat][guess] -= 1 W[feat][human_tag] += 1 examples = words, tags, contexts Example: Training a simple part-of-speech tagger with the perceptron algorithm
  5. the weights we’ll train Example: Training a simple part-of-speech tagger

    with the perceptron algorithm def train_tagger(examples): W = defaultdict(lambda: zeros(n_tags)) for (word, prev, next), human_tag in examples: scores = W[word] + W[prev] + W[next] guess = scores.argmax() if guess != human_tag: for feat in (word, prev, next): W[feat][guess] -= 1 W[feat][human_tag] += 1
  6. score tag given weight & context Example: Training a simple

    part-of-speech tagger with the perceptron algorithm def train_tagger(examples): W = defaultdict(lambda: zeros(n_tags)) for (word, prev, next), human_tag in examples: scores = W[word] + W[prev] + W[next] guess = scores.argmax() if guess != human_tag: for feat in (word, prev, next): W[feat][guess] -= 1 W[feat][human_tag] += 1
  7. get the best-scoring tag Example: Training a simple part-of-speech tagger

    with the perceptron algorithm def train_tagger(examples): W = defaultdict(lambda: zeros(n_tags)) for (word, prev, next), human_tag in examples: scores = W[word] + W[prev] + W[next] guess = scores.argmax() if guess != human_tag: for feat in (word, prev, next): W[feat][guess] -= 1 W[feat][human_tag] += 1
  8. decrease score for bad tag in this context increase score

    for good tag in this context Example: Training a simple part-of-speech tagger with the perceptron algorithm def train_tagger(examples): W = defaultdict(lambda: zeros(n_tags)) for (word, prev, next), human_tag in examples: scores = W[word] + W[prev] + W[next] guess = scores.argmax() if guess != human_tag: for feat in (word, prev, next): W[feat][guess] -= 1 W[feat][human_tag] += 1
  9. the part you 
 work on source code compiler runtime

    
 program “Regular” programming
  10. the part you 
 should work on source code compiler

    runtime 
 program training data training algorithm runtime model “Regular” programming Machine Learning
  11. Where human knowledge in AI really comes from Images: Amazon

    Mechanical Turk, depressing.org Mechanical Turk human annotators ~$5 per hour boring tasks low incentives
  12. Ask simple questions, 
 even for complex tasks better annotation

    speed better, easier-to-measure reliability in theory: any task can be broken down into a sequence of simpler or even binary decisions Solution #1
  13. assist human with good UX and task structure the things

    that are hard for the computer are usually easy for the human, and vice versa don’t waste time on what the model already knows, ask human about what the model is 
 most interested in Solution #2 UX-driven data collection with active learning
  14. Batch learning vs. active learning approach to annotation and training

    model human tasks human annotates all tasks annotated tasks are used as training data for model BATCH BATCH
  15. Batch learning vs. active learning approach to annotation and training

    model human tasks human annotates all tasks annotated tasks are used as training data for model model chooses one task human annotates chosen task annotated single task influences model’s decision on what to ask next BATCH BATCH ACTIVE
  16. Import knowledge with pre-trained models start off with general information

    about the language, the world etc. fine-tune and improve to fit custom needs big models can work with little training data backpropagate error signals to correct model Solution #3
  17. Backpropagation user input word meanings entity labels phrase meanings intent

    your examples “whats the best way to catalinas” fit meaning representations to your data
  18. If you can master annotation... ... you can try out

    more ideas quickly. Most ideas don’t work – but some succeed wildly. ... fewer projects will fail. Figure out what works before trying to scale it up. ... you can build entirely custom solutions and nobody can lock you in.