Machine Learning On Go Code

We've all wondered how to use Machine Learning with Go, but what about turning the tables for once? What can Machine Learning do *for* Go? During this presentation, we will discover how different Machine Learning models can help us write better go by predicting from our next character to our next bug!

Francesc’s talk will cover the basics of what Machine Learning techniques can be applied to source code, specifically:

- [embeddings over identifiers] (https://bit.ly/2HEcQhg)
- structural embeddings over source code, answering the question of how similar two fragments of code are,
- recurrent neural networks for code completion,
- future direction of the research.

While the topic is advanced, the level of mathematics required for this talk will be kept to a minimum. Rather than getting stuck in the details, we'll discuss the advantages and limitations of these techniques, and their possible implications to our developer lives.

Transcript

1. “Software is eating the world”
2. 128k LoC
3. 4-5M LoC
4. 9M LoC
5. 18M LoC
6. 45M LoC
7. 150M LoC
13. Machine Learning on Go Code Francesc Campoy
14. Machine Learning in Go Code Francesc Campoy
15. VP of Developer Relations Previously: • Developer Advocate at Google (Go team and Google Cloud Platform) twitter.com/francesc | github.com/campoy Francesc Campoy
16. just for func
17. Agenda • Machine Learning on Source Code • Research • Use Cases • The Future
18. Machine Learning on Source Code
19. Machine Learning on Source Code Field of Machine Learning where the input data is source code. MLonCode
20. Machine Learning on Source Code Requires: • Lots of data • Really, lots and lots of data • Fancy ML Algorithms • A little bit of luck Related Fields: • Data Mining • Natural Language Processing • Graph Based Machine Learning
21. Challenge #1 Data Retrieval
22. The datasets of ML on Code • GH Archive: https://www.gharchive.org • Public Git Archive https://pga.sourced.tech
23. Tasks • Language Classification • File Parsing • Token Extraction • Reference Resolution • History Analysis Retrieving data for ML on Code Tools • enry, linguist, etc • Babelfish, ad-hoc parsers • XPath / CSS selectors • Kythe • go-git
24. srcd sql # total lines of code per language in the Go repo SELECT lang, SUM(lines) as total_lines FROM ( SELECT LANGUAGE(t.tree_entry_name, b.blob_content) AS lang, ARRAY_LENGTH(SPLIT(b.blob_content, '\n')) as lines FROM refs r NATURAL JOIN commits c NATURAL JOIN commit_trees ct NATURAL JOIN tree_entries t NATURAL JOIN blobs b WHERE r.ref_name = 'HEAD' and r.repository_id = 'go' ) AS lines WHERE lang is not null GROUP BY lang ORDER BY total_lines DESC;
25. SELECT files.repository_id, files.file_path, ARRAY_LENGTH(UAST( files.blob_content, LANGUAGE(files.file_path, files.blob_content), '//*[@roleFunction and @roleDeclaration]') ) as functions FROM files NATURAL JOIN refs WHERE LANGUAGE(files.file_path,files.blob_content) = 'Go' AND refs.ref_name = 'HEAD' srcd sql
27. source{d} engine github.com/src-d/engine
28. Challenge #2 Data Analysis
29. '112', '97', '99', '107', '97', '103', '101', '32', '109', '97', '105', '110', '10', '10', '105', '109', '112', '111', '114', '116', '32', '40', '10', '9', '34', '102', '109', '116', '34', '10', '41', '10', '10', '102', '117', '110', '99', '32', '109', '97', '105', '110', '40', '41', '32', '123', '10', '9', '102', '109', '116', '46', '80', '114', '105', '110', '116', '108', '110', '40', '34', '72', '101', '108', '108', '111', '44', '32', '112', '108', '97', '121', '103', '114', '111', '117', '110', '100', '34', '41', '10', '125', '10' package main import “fmt” func main() { fmt.Println(“Hello, Denver”) } What is Source Code
30. package package IDENT main ; import import STRING "fmt" ; func func IDENT main ( ) What is Source Code { IDENT fmt . IDENT Println ( STRING "Hello, Denver" ) ; } ; package main import “fmt” func main() { fmt.Println(“Hello, Denver”) }
31. What is Source Code package main import “fmt” func main() { fmt.Println(“Hello, Denver”) }
32. What is Source Code package main import “fmt” func main() { fmt.Println(“Hello, Denver”) }
33. What is Source Code • A sequence of bytes • A sequence of tokens • An abstract syntax tree • A Graph (e.g. Control Flow Graph)
34. Challenge #3 Learning from Source Code
35. Neural Networks Basically fancy linear regression machines Given an input of a constant length, they predict an output of constant length. Example: MNIST: Input: images with 28x28 px Output: a digit from zero to 9
36. MNIST ~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0 ~1 ~0
37. MLonCode: Predict the next token for i := 0 ; i < 10 ; i ++
38. Recurrent Neural Networks Can process sequences of variable length. Uses its own output as a new input. Example: Natural Language Translation: Input: “bonjour, les gauffres” Output: “hi, waffles”
39. MLonCode: Code Generation charRNN: Given n characters, predict the next one Trained over the Go standard library Achieved 61% accuracy on predictions.
40. Before training r t, kp0t@pp kpktp 0p000 xS%%%?ttk?^@p0rk^@%ppp@ta#p^@ #pp}}%p^@?P%^@@k#%@P}}ta S?@}^@t%@% %%aNt i ^@SSt@@ pyikkp?%y ?t k L P0L t% ^@i%yy ^@p i? %L%LL tyLP?a ?L@Ly?tkk^@ @^@ykk^@i#P^@iL@??@%1tt%^@tPTta L ^@LL%% %i1::yyy^@^@t tP @?@a#Patt 1^@@ k^@k ? yt%L1^@tP%k1?k? % ^@i ^@ta1?1taktt1P?a^@^@Pkt?#^@t^@##1?## #^@t11#:^@%??t%1^@a 1?a at1P ^@^@Pt #%^@^@ ^@aaak^@#a#?P1Pa^@tt%?^@kt?#akP ?#^@i%%aa ^@1%t tt?a?% t^@k^@^@k^@ a : ^@1 P# % ^@^@#t% :% kkP ^@#?P: t^@a ?%##?kkPaP^@ #a k?t?? ai?i%PPk taP% P^@ k^@iiai#?^@# #t ?# P?P^@ i^@ttPt # 1%11 ti a^@k P^@k ^@kt %^@%y?#a a#% @? kt ^@t%k? ^@PtttkL tkLa1 ^@iaay?p1% Pta tt ik?ty k^@kpt%^@tktpkryyp^@?pP# %kt?ki? i @t^@k^@%#P} ?at}akP##Pa11%^@i% ^@?ia ia%##%tki % }i%%%}} a ay^@%yt }%t ^@tU%a% t}yi^@ ^@ @t yt%? aP @% ^@??^@%? ^@??k#% kk#%t?a: P}^@t :#^@#1t^@#: w^@P#%w:Pt t # t%aa%i@ak@@^@ka@^@a # y}^@# ^@? % tP i? ?tk ktPPt a tpprrpt? a^@ pP pt %p ? k? ^@^@ kP^@%%?tk a Pt^@# tP? P kkP1L1tP a%? t1P%PPti^@?%ytk %#%%t?@?^@ty^@iyk%1#^@@^@1#t a t@P^@^@ P@^@1P^@%%#@P:^@%^@ t 1:#P#@LtL#@L L1 %%dt??^@L ^@iBt yTk%p ^@i
41. After one epoch (dataset seen once) if testingValuesIntering() { t.SetCaterCleen(time.SewsallSetrive(true) if weq := nil { t.Errorf("eshould: wont %v", touts anverals, prc.Strnared, error } t, err := ntr.Soare(cueper(err, err) if err != nil { t.Errorf("preveth dime il resetests:%d; want %#',", tr.test.Into } if err != nil { return } if err == nel { t.Errorf("LoconserrSathe foot %q::%q: %s;%want %d", d, err) }, defarenContateFule(temt.Canses) } if err != nil { return err } // Treters and restives of the sesconse stmpeletatareservet // This no to the result digares wheckader. Constate bytes alleal
42. After two epochs if !ok { t.Errorf("%d: %v not %v", i, err) } if !ot.Close() if enr != nil { t.Fatal(err) } if !ers != nil { t.Fatal(err) } if err != nil { t.Fatal(err) } if err != nil { t.Errorf("error %q: %s not %v", i, err) } return nil }
43. if got := t.struct(); !ok { t.Fatalf("Got %q: %q, %v, want %q", test, true } if !strings.Connig(t) { t.Fatalf("Got %q: %q", want %q", t, err) } if !ot { t.Errorf("%s < %v", x, y) } if !ok { t.Errorf("%d <= %d", err) } if !stricgs(); !ot { t.Errorf("!(%d <= %v", x, e) } } if !ot != nil { return "" } After many epochs
44. Learning to Represent Programs with Graphs from, err := os.Open("a.txt") if err != nil { log.Fatal(err) } defer from.Close() to, err := os.Open("b.txt") if err != nil { log.Fatal(err) } defer ???.Close() io.Copy(to, from) Miltiadis Allamanis, Marc Brockschmidt, Mahmoud Khademi https://arxiv.org/abs/1711.00740 The VARMISUSE Task: Given a program and a gap in it, predict what variable is missing.
45. code2vec: Learning Distributed Representations of Code Uri Alon, Meital Zilberstein, Omer Levy, Eran Yahav https://arxiv.org/abs/1803.09473 | https://code2vec.org/
46. Much more research github.com/src-d/awesome-machine-learning-on-source-code
47. Challenge #4 What can we build?
48. Predictable vs Predicted ~0 ~0 ~0 ~0 ~0 ~0 ~0 ~0 ~1 ~0
49. A G o PR An attention model for code reviews.
52. Can you see the mistake? VARMISUSE from, err := os.Open("a.txt") if err != nil { log.Fatal(err) } defer from.Close() to, err := os.Open("b.txt") if err != nil { log.Fatal(err) } defer from.Close() io.Copy(to, from)
53. Can you see the mistake? VARMISUSE from, err := os.Open("a.txt") if err != nil { log.Fatal(err) } defer from.Close() to, err := os.Open("b.txt") if err != nil { log.Fatal(err) } defer from.Close() ← s/from/to/ io.Copy(to, from)
54. Is this a good name? func XXX(list []string, text string) bool { for _, s := range list { if s == text { return true } } return false } Suggestions: • Contains • Has func XXX(list []string, text string) int { for i, s := range list { if s == text { return i } } return -1 } Suggestions: • Find • Index code2vec: Learning Distributed Representations of Code
55. source: WOCinTech Assisted code review. src-d/lookout
56. And so much more Coming soon: • Automated Style Guide Enforcing • Bug Prediction • Automated Code Review • Education Coming … later: • Code Generation: from unit tests, specification, natural language description. • Natural Analysis: code description and conversational analysis.
57. Will developers be replaced?
58. Developers will be empowered.
59. Want to know more? • sourced.tech (pssh, we’re hiring) • github.com/src-d/awesome-machine-learning-on-source-code • francesc@sourced.tech • come say hi, I have stickers
60. Thanks francesc