Upgrade to Pro — share decks privately, control downloads, hide ads and more …

YAJS.vim and Vim Syntax Highlight

othree
June 25, 2016

YAJS.vim and Vim Syntax Highlight

JavaScript is very important for the world wide web universe. But JavaScript support on Vim is not update very often. So I keep contribute to the Vim community. One of my contribution is yajs.vim(Yet Another JavaScript Syntax). One of the most powerful JavaScript syntax highlight plug-in for Vim now. It supports almost every new syntax from ECMAScript 2015, JSDoc, lots of new Web API and fixed several old issues. This talk will first explain Vim syntax system. The design, concept and limitation of it. Then will share the problems encountered during the development in this environment. And finally, how I solved these problems.

othree

June 25, 2016
Tweet

More Decks by othree

Other Decks in Programming

Transcript

  1. me • othree @ flickr, github, twitter… • MozTW •

    F2E at Qihoo • Speeches in COSCUP, OSDC.tw, JSDC… • https://blog.othree.net/
  2. YAJS.vim • Yet Another JavaScript Syntax for Vim • Supports

    ECMAScript 6 syntax • Remove legacy code support • and JSDoc, Web APIs, console, …. • javascript instead of javaScript
  3. Why YAJS • ECMAScript 6(a.k.a ES6, ES2015) starts since 2011

    • People can’t wait to use new features • Transpilers: Traceur, 6to5(Babel)…
  4. • People start to use ES6(and lots of ES.next feature)

    • JavaScript syntax for Vim doesn’t support ES6 • So I start to do it
  5. Dev Target • Support all valid ES6 syntax • If

    possible, add some error hint • Performance is prioritize to second place
  6. Why no Push to Upstream • Lots of incompatible code

    change during dev • Can’t wait for a PR review
  7. Status Now • Support major ES6 syntax • Still in

    development • Trying to build auto test
  8. Keyword syntax keyword lsKeyword function syntax keyword lsKeyword return highlight

    link lsKeyword Statement highlight Statement ctermfg=251 ctermbg=None
  9. syntax keyword lsKeyword function syntax keyword lsKeyword return highlight link

    lsKeyword Statement highlight Statement ctermfg=251 ctermbg=None group name item type
  10. syntax keyword lsKeyword function syntax keyword lsKeyword return highlight link

    lsKeyword Statement highlight Statement ctermfg=251 ctermbg=None
  11. syntax keyword lsKeyword function syntax keyword lsKeyword return highlight link

    lsKeyword Statement highlight Statement ctermfg=251 ctermbg=None target group name source group name link groups
  12. syntax keyword lsKeyword function syntax keyword lsKeyword return highlight link

    lsKeyword Statement highlight Statement ctermfg=251 ctermbg=None color definition group name
  13. Match syntax keyword lsKeyword function syntax keyword lsKeyword return syntax

    match lsParens /(\k*)/ syntax match lsContains /contains/
  14. contained syntax keyword lsKeyword function syntax keyword lsKeyword return syntax

    match lsParens contained /(\k*)/ syntax match lsContains /contains/
  15. nextgroup syntax keyword lsKeyword function nextgroup=lsParens syntax keyword lsKeyword return

    syntax match lsParens contained /(\k*)/ syntax match lsContains /contains/
  16. nextgroup • Vim will lookup groups listed in nextgroup first

    • Before normal keyword, match, region • Not limited by contained
  17. var foo = function(arg) { return 1; }; return 1;

    (arg); Ending of keyword function
  18. var foo = function(arg) { return 1; }; return 1;

    (arg); Looking for groups in nextgroup
  19. skipwhite syntax keyword lsKeyword function nextgroup=lsParens skipwhite syntax keyword lsKeyword

    return syntax match lsParens contained /(\k*)/ syntax match lsContains /contains/
  20. Vim Keywords syntax keyword lsKeyword function syntax keyword lsKeyword return

    syntax match lsParens /(\k*)/ syntax keyword lsContains contains
  21. syntax keyword lsKeyword function syntax keyword lsKeyword return syntax match

    lsParens /(\k*)/ syntax match lsContains /contains/
  22. syntax keyword lsKeyword function syntax keyword lsKeyword return syntax match

    lsParens /(\k*)/ syntax match lsContains /contains/ syntax keyword lsKeyword con
  23. back to the issue var foo = function () {

    return 1; }; return 1; match keyword: return
  24. Region syntax keyword lsKeyword function syntax keyword lsReturn return syntax

    match lsParens /(\k*)/ syntax region lsBlock start=/{/ end=/}/
  25. syntax keyword lsKeyword function syntax keyword lsReturn contained return syntax

    match lsParens /(\k*)/ syntax region lsBlock start=/{/ end=/}/ contains=lsReturn
  26. Nesting Region syntax keyword lsKeyword function syntax keyword lsReturn contained

    return syntax match lsParens /(\k*)/ syntax region lsBlock matchgroup=lsBrack start=/{/ end=/}/ contains=lsBlock,lsKeyword,lsReturn
  27. syntax keyword lsKeyword function syntax keyword lsReturn contained return syntax

    match lsParens /(\k*)/ syntax region lsBlock matchgroup=lsBrack start=/{/ end=/}/ contains=lsBlock,lsKeyword,lsReturn syntax keyword lsDeclare var let
  28. TOP region syntax keyword lsKeyword function syntax keyword lsReturn contained

    return syntax match lsParens /(\k*)/ syntax region lsBlock matchgroup=lsBrack start=/{/ end=/}/ contains=TOP syntax keyword lsDeclare var let
  29. • Not real lexical analysis, have limitation • For performance

    • Best situation: • every token are independent • no nesting
  30. Quest? • Vim syntax highlight system has limitation • To

    support all ES2015 syntax is like solve quests
  31. • Have everything outside of the block • TOP region

    groups • Function block region groups
  32. • Have almost everything in TOP region • TOP region

    groups • And `return` • But no import/export related stuff
  33. • Have all property name syntax and value(RHS) • Property

    name syntaxes • identifier, number, string • computed • And then RHS expression
  34. Issues • These blocks have very different contents • I

    have to know what this region is at position `{`
  35. Normal don’t need to deal Function after `function` keyword, method

    syntax Class always after `class` keyword Object in RHS expression
  36. function block • Have almost everything in TOP region •

    TOP region groups • And `return` • But no import/export related stuff
  37. TOP • A special group cluster • Contains every group

    without `contained` • I called these stuff belongs to TOP region
  38. but

  39. in HTML Syntax syn include @htmlJavaScript syntax/javascript.vim unlet b:current_syntax syn

    region javaScript start=+<script\_[^>]*>+ keepend end=+</script\_[^>]*>+me=s-1 contains=@htmlJavaScript
  40. in Jinja Syntax " Pull in the HTML syntax. if

    g:jinja_syntax_html if version < 600 so <sfile>:p:h/html.vim else runtime! syntax/html.vim unlet b:current_syntax endif endif
  41. Detect Filetype if &filetype =~ 'javascript' syntax region javascriptBlock matchgroup=javascriptBraces

    start=/{/ end=/}/ contains=TOP else syntax region javascriptBlock matchgroup=javascriptBraces start=/{/ end=/}/ contains=@htmlJavaScript endif
  42. Template String `string text` `string text line 1 string text

    line 2` `string text ${expression} string text`
  43. `string text` `string text line 1 string text line 2`

    `string text ${expression} string text`
  44. • The string ends when match an accent ` •

    Try to recognize inner content
  45. var foo = function (arg1) {}; var fooArrow = (arg1)

    => {}; var fooParen = (arg1); Can’t lookup following token When cursor is here
  46. => var foo = function (arg1) {}; var fooArrow =

    (arg1) => {}; var fooParen = (arg1);
  47. Problem • Don’t know what it is when meet `(`

    • Function parameters or • Expression in parentheses
  48. Before Default Parameter • Only identifier and comma inside function

    argument parens • Identifier name possible characters: [a-zA-Z_0-9$] • Not contain right parentheses
  49. Workaround Solution • Use match to match an arrow function

    • Put this match rule after parentheses region
 (higher priority)
  50. syntax match javascriptArrowFunc /=>/ syntax region javascriptArrowFuncArg contained matchgroup=javascriptParens start=/(/

    end=/)/ contains=@javascriptFuncArgElements nextgroup=javascriptArrowFunc skipwhite skipempty
  51. function multiply(a, b) { var b = typeof b !==

    'undefined' ? b : 1; return a*b; }
  52. syntax keyword javascriptFuncKeyword function nextgroup=javascriptFuncName skipwhite syntax match javascriptFuncName contained

    /[a-zA-Z_$]\k*/ nextgroup=javascriptFuncArg skipwhite syntax match javascriptFuncArg contained /(\_[^)]*)/ nextgroup=javascriptBlock skipwhite
  53. function multiply(a, b) { var b = typeof b !==

    'undefined' ? b : 1; return a*b; }
  54. function multiply(a, b) { var b = typeof b !==

    'undefined' ? b : 1; return a*b; } FuncKeyword FuncName FuncArg
  55. • Not able to really know which paren is ending

    one • Have to well know the content of parens • Use region
  56. syntax region javascriptFuncArg contained matchgroup=javascriptParens start=/(/ end=/)/ contains=@javascriptFuncArgElements nextgroup=javascriptBlock skipwhite

    syntax cluster javascriptFuncArgElements contains=... javascriptComma, javascriptDefaultAssign, @javascriptComments... syntax match javascriptDefaultAssign contained /=/ nextgroup=@javascriptExpression skipwhite
  57. syntax region javascriptFuncArg contained matchgroup=javascriptParens start=/(/ end=/)/ contains=@javascriptFuncArgElements nextgroup=javascriptBlock skipwhite

    syntax cluster javascriptFuncArgElements contains=... javascriptComma, javascriptDefaultAssign, @javascriptComments... syntax match javascriptDefaultAssign contained /=/ nextgroup=@javascriptExpression skipwhite
  58. syntax region javascriptFuncArg contained matchgroup=javascriptParens start=/(/ end=/)/ contains=@javascriptFuncArgElements nextgroup=javascriptBlock skipwhite

    syntax cluster javascriptFuncArgElements contains=... javascriptComma, javascriptDefaultAssign, @javascriptComments... syntax match javascriptDefaultAssign contained /=/ nextgroup=@javascriptExpression skipwhite
  59. var foo = function (arg1) {}; var fooArrow = (arg1

    = 1) => {}; var fooArrow = (arg1 = 1, arg2 = `string`) => {};
  60. var foo = function (arg1) {}; var fooArrow = (arg1

    = 1) => {}; var fooArrow = (arg1 = function () {}) => {};
  61. Matching fails here var foo = function (arg1) {}; var

    fooArrow = (arg1 = 1) => {}; var fooArrow = (arg1 = function () {}) => {};
  62. var fooArrow1 = function (arg1 = () => {}) {};

    var fooArrow2 = function (arg1 = () => { return function () {}; }) {};
  63. var fooArrow1 = function (arg1 = () => {}) {};

    var fooArrow2 = function (arg1 = () => { return function () {}; }) {}; var fooArrow3 = function (arg1 = (arg2 = () => {}) => { return function () {}; }) {};