Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Machine Learning on Source Code

Machine Learning on Source Code

source{d} is building the open-source components to enable large-scale code analysis and machine learning on source code.

Their powerful tools can ingest all of the world’s public git repositories turning code into ASTs ready for machine learning and other analyses, all exposed through a flexible and friendly API.

Francesc Campoy, VP of Developer Relations at source{d}, will show you how to run machine learning on source code with a series of live demos.

Francesc Campoy Flores

April 25, 2018
Tweet

More Decks by Francesc Campoy Flores

Other Decks in Programming

Transcript

  1. Deep Learning Revolutions - Computer Vision: ImageNet - Natural Language

    Processing: Siri, Alexa, Google Assistant, - Go: AlphaGo
  2. Machine Learning on Source Code - Automated code review -

    Source code similarity: clustering + search - Translation: - source code to natural language - source code to source code - natural language to source code?
  3. '112', '97', '99', '107', '97', '103', '101', '32', '109', '97',

    '105', '110', '10', '10', '105', '109', '112', '111', '114', '116', '32', '40', '10', '9', '34', '102', '109', '116', '34', '10', '41', '10', '10', '102', '117', '110', '99', '32', '109', '97', '105', '110', '40', '41', '32', '123', '10', '9', '102', '109', '116', '46', '80', '114', '105', '110', '116', '108', '110', '40', '34', '72', '101', '108', '108', '111', '44', '32', '112', '108', '97', '121', '103', '114', '111', '117', '110', '100', '34', '41', '10', '125', '10' package main import “fmt” func main() { fmt.Println(“Hello, Chicago”) } What is Source Code
  4. package package IDENT main ; import import STRING "fmt" ;

    func func IDENT main ( ) package main import “fmt” func main() { fmt.Println(“Hello, Chicago”) } What is Source Code { IDENT fmt . IDENT Println ( STRING "Hello, Chicago" ) ; } ;
  5. github.com/src-d/borges Borges • Store git repositories • Rooted repositories Rovers

    • Crawls for git repos • Supports: ◦ GitHub ◦ BitBucket ◦ Cgit github.com/src-d/rovers Siva • Seekable Indexed Block Archiver • Small, concatenable, constant access. Data retrieval at source{d} github.com/src-d/siva
  6. Learning from Code Code is: - a sequence of bytes

    - a sequence of tokens - a tree
  7. Learning sequences Input Output Speech recognition: audio clip text Sentiment

    analysis: text rating -1 to 1 Machine Translation: Hello, everyone! Hola a tothom! Video recognition Sequence of frames text
  8. Is this useful? - Predicting implies understanding - We can

    predict aspects of code: - Help with assisted editing - Detect anomalies
  9. We could use the same mechanism! package main ; import

    “fmt” ; main ; import “fmt” ; func
  10. Categorical (non continuous) variables require one-hot encoding: Predict characters zero

    to 9: 0 -> [1, 0, 0, 0, 0, 0, 0, 0, 0, 0] 1 -> [0, 1, 0, 0, 0, 0, 0, 0, 0, 0] 2 -> [0, 0, 1, 0, 0, 0, 0, 0, 0, 0] 3 -> [0, 0, 0, 1, 0, 0, 0, 0, 0, 0] … 9 -> [0, 0, 0, 0, 0, 0, 0, 0, 0, 1] But … one-hot encoding
  11. Possible characters in human text: hundreds Possible characters in computer

    programs: hundreds Possible words in English: 171,476 (Oxford dictionary*) * urban dictionary not included But … one-hot encoding
  12. Abstract Syntax Trees • Each programming language has its own

    grammar • Each grammar generates slightly different ASTs • We want to learn from *all* languages!
  13. Other projects - Vecino: finding similar repositories - Apollo: finding

    source code duplication at scale - TMSC: topic modeling on source code repositories - Snippet-Ranger: topic modeling on source code snippets