Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a go tool to modify struct tags

Building a go tool to modify struct tags

Struct field tags are an important part of encode/decode types, especially when using packages such as encoding/json. However, modifying tags is repetitive, cumbersome and open to human errors. We can make it easy to modify tags with an automated tool that is written for this sole purpose.

Fatih Arslan

July 13, 2017
Tweet

More Decks by Fatih Arslan

Other Decks in Programming

Transcript

  1. type Example struct { StatusID int64 Foo string Bar bool

    Server struct { Address string TLS bool } DiskSize int64 Volumes []string }
  2. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  3. type Example struct { Foo string "json:\"foo\"" } quotes instead

    of backticks (works, but not fun to deal with)
  4. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  5. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  6. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"tls" xml:"tls"`
  7. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  8. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"server"`
  9. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  10. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"disk_size"`
  11. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` }
  12. type Example struct { StatusID int64 `json:"status_id"` Foo string `json:"foo"

    xml:"foo"` Bar bool `json:"bar" xml:"bar"` Server struct { Address string `json:"address"` TLS bool `json:"tls",xml:"tls"` } `json="server"` DiskSize int64 `json:disk_size` Volumes []string `"json":"volumes"` } `json:"volumes"`
  13. Old implementation • From the cursor search backwards for the

    first 'struct {' literal • Once found, do a forward search for the right hand brace } • Get the line numbers between the two braces • For each line, get the first identifier, convert it to camel_case word and then append to the same line • A combination of Vim's search() function and regex is being used
  14. No Formatting type Example struct { Bar string `json:"bar"` Foo

    bool `json:"foo"` } type Example struct { Bar string `json:"bar"` Foo bool `json:"foo"` } want have
  15. In-line comments type Example struct { Bar string `json:"bar"` //

    comment for bar Foo bool `json:"foo"` } type Example struct { Bar string // comment for bar `json:"bar"` Foo bool `json:"foo"` } want have
  16. Nested structs type Example struct { Bar bool `json:"bar"` Server

    struct { `json:"server"` Address string `json:"address"` } } type Example struct { Bar bool `json:"bar"` Server struct { Address string `json:"address"` } `json:"server"` } have want
  17. Duplicate tags type Example struct { Bar string `json:"bar"` `xml:"bar"`

    Foo bool `xml:"foo"` } have type Example struct { Bar string `json:"bar" xml:"bar"` Foo bool `xml:"foo"` } want
  18. more quirks • not able to remove tags • not

    able to add or remove options • field with interface{} types don't work • comment with braces ({ and }) don't work • ...
  19. type Example struct { Foo string `json:"foo"` } *ast.TypeSpec *ast.StructType

    *ast.FieldList *ast.Field *ast.BasicLit `json:"foo"` (a.k.a: struct tag)
  20. type Example struct { Foo string `json:"foo"` } *ast.TypeSpec *ast.StructType

    *ast.FieldList *ast.Field []*ast.Ident *ast.Ident *ast.BasicLit
  21. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  22. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  23. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  24. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  25. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false })
  26. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { fmt.Printf("Field: %s\n", field.Names[0].Name) fmt.Printf("Tag: %s\n", field.Tag.Value) } return false }) Field: Foo Tag: `json:"foo"`
  27. package main import ( "fmt" "reflect" ) func main() {

    tag := reflect.StructTag(`json:"foo"`) value := tag.Get("json") fmt.Printf("value: %q\n", value) }
  28. package main import ( "fmt" "reflect" ) func main() {

    tag := reflect.StructTag(`json:"foo"`) value := tag.Get("json") fmt.Printf("value: %q\n", value) } $ go run main.go value: "foo"
  29. Issues with reflect.StructTag • can't detect if the tag is

    malformed (only go vet knows that) • doesn't know the semantics of options (i.e: omitempty) • doesn't return all existing tags • modifying existing tags is not possible
  30. Parse and list all tags tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`)

    if err != nil { panic(err) } // iterate over all key-value pairs for _, t := range tags.Tags() { fmt.Printf("tag: %+v\n", t) } $ go run main.go tag: json:"foo,omitempty" tag: xml:"foo"
  31. Get a single Tag tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`) if

    err != nil { panic(err) } jsonTag, err := tags.Get("json") if err != nil { panic(err) } fmt.Println(jsonTag) // Output: json:"foo,omitempty" fmt.Println(jsonTag.Key) // Output: json fmt.Println(jsonTag.Name) // Output: foo fmt.Println(jsonTag.Options) // Output: [omitempty]
  32. Change existing tag jsonTag, err := tags.Get(`json:"foo,omitempty"`) if err !=

    nil { panic(err) } jsonTag.Name = "bar" jsonTag.Options = nil tags.Set(jsonTag) fmt.Println(tags) json:"bar"
  33. Add new tag tags, err := structtag.Parse(`json:"foo,omitempty" xml:"foo"`) hclTag :=

    &structtag.Tag{ Key: "hcl", Name: "gopher", Options: []string{"squash"}, } // add new tag tags.Set(hclTag) fmt.Println(tags) json:"foo,omitempty" xml:"foo" hcl:"gopher,squash"
  34. Add/remove options tags, err := structtag.Parse(`json:"foo" xml:"bar,comment"`) if err !=

    nil { panic(err) } tags.AddOptions("json", "omitempty") tags.DeleteOptions("xml", "comment") fmt.Println(tags) // json:"foo,omitempty" xml:"bar"
  35. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  36. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  37. cfg := &config{ file: *flagFile, line: *flagLine, structName: *flagStruct, offset:

    *flagOffset, output: *flagOutput, write: *flagWrite, clear: *flagClearTags, clearOption: *flagClearOptions, transform: *flagTransform, sort: *flagSort, override: *flagOverride, } $ gomodifytags --file example.go ... Use flags to set configuration
  38. Things we need: 1. What content to process 2. Where

    and what to output 3. Which struct to modify 4. Which tags to modify
  39. Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  40. Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  41. Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  42. Things we need: type config struct { file string modified

    io.Reader output string write bool offset int structName string line string start, end int remove []string add []string override bool transform string sort bool clear bool addOpts []string removeOpts []string clearOpt bool } 1. What content to process 2. Where and what to output 3. Which struct to modify 4. Which tags to modify
  43. package main type Example struct { Foo string } $

    gomodifytags -file example.go -struct Example -add-tags json
  44. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  45. gomodifytags parse parse go/parser package main type Example struct {

    Foo string } *ast.TypeSpec *ast.StructType *ast.FieldList *ast.Field []*ast.Ident *ast.Ident tag
  46. func (c *config) parse() (ast.Node, error) { c.fset = token.NewFileSet()

    var contents interface{} if c.modified != nil { archive, err := buildutil.ParseOverlayArchive(c.modified) if err != nil { return nil, fmt.Errorf("failed to parse -modified archive: %v", err) } fc, ok := archive[c.file] if !ok { return nil, fmt.Errorf("couldn't find %s in archive", c.file) } contents = fc } return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments) } Parse content
  47. func (c *config) parse() (ast.Node, error) { c.fset = token.NewFileSet()

    var contents interface{} if c.modified != nil { archive, err := buildutil.ParseOverlayArchive(c.modified) if err != nil { return nil, fmt.Errorf("failed to parse -modified archive: %v", err) } fc, ok := archive[c.file] if !ok { return nil, fmt.Errorf("couldn't find %s in archive", c.file) } contents = fc } return parser.ParseFile(c.fset, c.file, contents, parser.ParseComments) } Parse content (cont.)
  48. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  49. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  50. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
  51. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 struct name: Server
  52. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 struct name: Server start line: 8 end line: 10
  53. package main type Example struct { Foo string } type

    Server struct { Name string Port| int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 offset byte: 96 (of 163)
  54. package main type Example struct { Foo string } type

    Server struct { Name string Port| int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 offset byte: 96 (of 163) start line: 8 end line: 10
  55. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 start line: 8 end line: 10 .. or specify explicit lines
  56. package main type Example struct { Foo string } type

    Server struct { Name string Port int EnableLogs bool } type Person struct { Name string } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 start line: 9 end line: 9 .. or specify explicit lines
  57. func (c *config) findSelection(node ast.Node) (int, int, error) { if

    c.line != "" { return c.lineSelection(node) } else if c.offset != 0 { return c.offsetSelection(node) } else if c.structName != "" { return c.structSelection(node) } return 0, 0, errors.New("-line, -offset or -struct is not passed") } Find selection
  58. func (c *config) findSelection(node ast.Node) (int, int, error) { if

    c.line != "" { return c.lineSelection(node) } else if c.offset != 0 { return c.offsetSelection(node) } else if c.structName != "" { return c.structSelection(node) } return 0, 0, errors.New("-line, -offset or -struct is not passed") } Find selection
  59. // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType, 0) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  60. // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs } type structType struct { name string node *ast.StructType }
  61. // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  62. // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs } type structType struct { name string node *ast.StructType }
  63. // collectStructs collects and maps structType nodes to their positions

    func collectStructs(node ast.Node) map[token.Pos]*structType { structs := make(map[token.Pos]*structType) collectStructs := func(n ast.Node) bool { t, ok := n.(*ast.TypeSpec) if !ok { return true } structName := t.Name.Name x, ok := t.Type.(*ast.StructType) if !ok { return true } structs[x.Pos()] = &structType{ name: structName, node: x, } return true } ast.Inspect(node, collectStructs) return structs }
  64. var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  65. var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  66. var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  67. var encStruct *ast.StructType for _, st := range collectStructs() {

    if st.name == c.structName { encStruct = st.node } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line } Struct selection
  68. var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  69. var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  70. var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  71. var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  72. var encStruct *ast.StructType for _, st := range collectStructs() {

    structBegin := c.fset.Position(st.node.Pos()).Offset structEnd := c.fset.Position(st.node.End()).Offset if structBegin <= c.offset && c.offset <= structEnd { encStruct = st.node break } } start = c.fset.Position(encStruct.Pos()).Line end = c.fset.Position(encStruct.End()).Line Offset selection
  73. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  74. gomodifytags *ast.StructType 6 7 8 9 10 11 12 *ast.Field

    *ast.Field *ast.Field *ast.Field *ast.Field
  75. type Example struct { Foo string `json:"foo"` } gomodifytags *ast.StructType

    *ast.Field []*ast.Ident *ast.Ident *ast.BasicLit
  76. type Example struct { Foo string `json:"bar" } gomodifytags *ast.StructType

    *ast.Field Foo string `json:"foo" `json:"bar"` type Example struct { Foo string `json:"foo"` }
  77. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = `json:"bar"` } return false }) After finding a field, replace the value
  78. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = `json:"bar"` } return false }) Tag: `json:"foo"` Tag: `json:"bar"` After finding a field, replace the value
  79. gomodifytags *ast.StructType *ast.Field DiskSize string json:"disk_size" process() type Example struct

    { DiskSize string } type Example struct { DiskSize string `json:"disk_size"` }
  80. src := `package main type Example struct { Foo string`

    + " `json:\"foo\"` }" fset := token.NewFileSet() file, err := parser.ParseFile(fset, "demo", src, parser.ParseComments) if err != nil { panic(err) } ast.Inspect(file, func(x ast.Node) bool { s, ok := x.(*ast.StructType) if !ok { return true } for _, field := range s.Fields.List { // found field! field.Tag.Value = process(field) } return false }) tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  81. gomodifytags *ast.StructType 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field

    select field between start and end lines ... Modify overview
  82. gomodifytags *ast.StructType 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field

    ... and rewrite selected fields tags 8 9 10 *ast.Field *ast.Field *ast.Field *ast.Field *ast.Field rewrite Modify overview
  83. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  84. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  85. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  86. for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  87. for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  88. for _, f := range x.Fields.List { line := c.fset.Position(f.Pos()).Line

    if !(c.start <= line && line <= c.end) { continue } if f.Tag == nil { f.Tag = &ast.BasicLit{} } // ... }
  89. for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  90. for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  91. for _, f := range x.Fields.List { fieldName := ""

    if len(f.Names) != 0 { fieldName = f.Names[0].Name } if f.Names == nil { ident, ok := f.Type.(*ast.Ident) if !ok { continue // anonymous field } fieldName = ident.Name } res, err := c.process(fieldName, f.Tag.Value) if err != nil { rewriteErr = err return true } f.Tag.Value = res }
  92. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  93. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  94. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  95. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil } tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  96. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil } tags = c.removeTags(tags) tags, err = c.removeTagOptions(tags) if err != nil { return "", err } tags = c.clearTags(tags) tags = c.clearOptions(tags) tags, err = c.addTags(fieldName, tags) if err != nil { return "", err } tags, err = c.addTagOptions(tags) if err != nil { return "", err } if c.sort { sort.Sort(tags) }
  97. func (c *config) process(fieldName, tagVal string) (string, error) { var

    tag string if tagVal != "" { var err error tag, err = strconv.Unquote(tagVal) if err != nil { return "", err } } tags, err := structtag.Parse(tag) if err != nil { return "", err } // process tags ... res := tags.String() if res != "" { res = quote(tags.String()) } return res, nil }
  98. func (c *config) rewriteFields(node ast.Node) (ast.Node, error) { var rewriteErr

    error rewriteFunc := func(n ast.Node) bool { x, ok := n.(*ast.StructType) if !ok { return true } for _, f := range x.Fields.List { // process each field // ... } return true } ast.Inspect(node, rewriteFunc) return node, rewriteErr }
  99. func main() { var cfg config node = cfg.parse() start,

    end = cfg.findSelection(node) rewritten = cfg.rewrite(node, start, end) out = cfg.format(rewritten) fmt.Println(out) } 1. Fetch configuration settings 2. Parse content 3. Find selection 4. Modify the struct tag 5. Output the result
  100. gomodifytags ast.Node (modified) go/format ... and output the result to

    stdout package main type Example struct { Foo string `json:"foo"` }
  101. package main type Server struct { Name string Port int

    EnableLogs bool BaseDomain string Credentials struct { Username string Password string } } package main type Server struct { Name string `json:"name"` Port int `json:"port"` EnableLogs bool `json:"enable_logs"` BaseDomain string `json:"base_domain"` Credentials struct { Username string `json:"username"` Password string `json:"password"` } `json:"credentials"` } Input Output $ gomodifytags -file example.go -struct Server -add-tags json
  102. gomodifytags ast.Node (modified) go/format Also has support for custom JSON

    output { "start": 3, "end": 5, "lines": [ "type Example struct {", " Foo string `json:\"foo\"`", "}" ] }
  103. package main type Server struct { Name string Port int

    EnableLogs bool BaseDomain string Credentials struct { Username string Password string } } { "start": 3, "end": 12, "lines": [ "type Server struct {", " Name string `xml:\"name\"`", " Port int `xml:\"port\"`", " EnableLogs bool `xml:\"enable_logs\"`", " BaseDomain string `xml:\"base_domain\"`", " Credentials struct {", " Username string `xml:\"username\"`", " Password string `xml:\"password\"`", " } `xml:\"credentials\"`", "}" ] } Input JSON Output $ gomodifytags -file example.go -struct Server -add-tags json -format json
  104. Why stdout? • by default, it means "dry-run" • immediate

    feedback • tools can redirect the output • composable (gomodifytags myfile.go > newfile.go)
  105. func (c *config) format(file ast.Node) (string, error) { switch c.output

    { case "source": // return formatted source case "json": // return only changes in json default: return "", fmt.Errorf("unknown output mode: %s", c.output) } } Output the result
  106. func (c *config) format(file ast.Node) (string, error) { switch c.output

    { case "source": // return formatted source case "json": // return only changes in json default: return "", fmt.Errorf("unknown output mode: %s", c.output) } } Output the result
  107. var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  108. var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  109. var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } if c.write { err = ioutil.WriteFile(c.file, buf.Bytes(), 0) if err != nil { return "", err } } return buf.String(), nil Source file
  110. var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } var lines []string scanner := bufio.NewScanner(bytes.NewBufferString(buf.String())) for scanner.Scan() { lines = append(lines, scanner.Text()) } if c.start > len(lines) { return "", errors.New("line selection is invalid") } ... JSON output
  111. var buf bytes.Buffer err := format.Node(&buf, c.fset, file) if err

    != nil { return "", err } var lines []string scanner := bufio.NewScanner(bytes.NewBufferString(buf.String())) for scanner.Scan() { lines = append(lines, scanner.Text()) } if c.start > len(lines) { return "", errors.New("line selection is invalid") } ... JSON output
  112. out := &output{ Start: c.start, End: c.end, Lines: lines[c.start-1 :

    c.end], } o, err := json.MarshalIndent(out, "", " ") if err != nil { return "", err } return string(o), nil JSON output (cont.)
  113. out := &output{ Start: c.start, End: c.end, Lines: lines[c.start-1 :

    c.end], } o, err := json.MarshalIndent(out, "", " ") if err != nil { return "", err } return string(o), nil JSON output (cont.)
  114. package main type Example struct { Foo string } parse

    find modify gomodifytags format format
  115. package main type Example struct { Foo string } parse

    find modify gomodifytags format Overview
  116. package main type Example struct { Foo string } -file

    example.go -struct Example -add-tags json gomodifytags
  117. package main type Example struct { Foo string } package

    main type Example struct { Foo string `json:"foo"` } parse find modify format gomodifytags
  118. Built from the ground up for editors • selecting structs

    based on offset or range of lines • write result to stdout or file • json output for easy parsing • support for unsaved buffers • individual cli flags for all features
  119. Supported editors • vim with vim-go • atom with go-plus

    (thanks Zac Bergquist) • vscode with vscode-go (thanks Ramya Rao) • emacs - WIP (https://github.com/syohex/emacs-go-add-tags/issues/ 6)
  120. Recap • A single tool to rule all editors •

    Easy maintenance • Free of bugs due stdlib tooling (go/parser family) • Happy and productive users