Upgrade to Pro — share decks privately, control downloads, hide ads and more …

impguard - protect your project structure

Oleg Kovalov
November 26, 2020

impguard - protect your project structure

Oleg Kovalov

November 26, 2020
Tweet

More Decks by Oleg Kovalov

Other Decks in Programming

Transcript

  1. • Gopher for ~5 years • Open source contributor •

    Engineer at GoGoApps Hi, i’m Oleg olegk.dev @oleg_kovalov @cristaloleg
  2. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic
  3. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be
  4. At GoGoApps we’re... • (re)building a product for a big

    company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
  5. At GoGoApps we’re... But there is one thing is running

    in my mind for a long time… • (re)building a product for a big company™ • with a lot of microservices • and a complicated business logic • with a care on our code and architecture • CI, linters, code reviews, everything as it should be • ...and everything is fine!
  6. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast
  7. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above
  8. Define me a problem first • linters give you suggestions

    on code level • mighty golangci-lint is The Beast • but let’s look level above • how your packages are organized?
  9. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something
  10. Setup, goals, solution • you’ve a monorepo • or just

    a bunch of repos • and you want to keep them similar • same folder structure • same filenames • same imports • same something • custom tooling solves this!
  11. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said
  12. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed
  13. Let’s talk about imports • core idea of (re)using others

    code • staying on the shoulders of giants as Newton said • reducing repeating work • it’s simpler to integrate something than build from scratch • ^^^ citation needed • but nothing is free • others code needs to be verified and used carefully • you can rely on any code and use it everywhere • but that might be a ticking bomb (LeftPad-like problem)
  14. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization
  15. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb
  16. When in doubts - see golang/go • Go doesn’t allow

    import cycles - that’s cool • simpler compilation routine and code organization • But that’s not enough • compiling that you don’t really need is time and cpu wasteful • See: compiler bomb • Stdlib has a test for imports https://github.com/golang/go/blob/release-branch.go1.14/src/go/build/deps_test.go https://github.com/golang/go/blob/release-branch.go1.15/src/go/build/deps_test.go
  17. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change
  18. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages
  19. What does it mean? • in other words you’ve a

    rigid code structure • you cannot import something without an explicit change • this can be a solution for some rare changed code • but not for feature rich and fast moving packages • what we can do? • check Github!
  20. What do we’ve right now? • `$ go mod why`

    • out of the box solution
  21. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint)
  22. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file)
  23. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies
  24. What do we’ve right now? • `$ go mod why`

    • out of the box solution • https://github.com/OpenPeeDeeP/depguard • allow/block list for deps (presented in golangci-lint) • https://github.com/tailscale/depaware • know what is imported by whom (via golden file) • https://github.com/divan/depscheck • helps to find (and copy-paste) small dependencies • … (sorry, I cannot copy-paste all the Github ):
  25. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole
  26. go mod why > go mod why github.com/go-ole/go-ole # github.com/go-ole/go-ole

    tailscale.com/wgengine/router github.com/go-ole/go-ole > go mod why github.com/RoaringBitmap/roaring # github.com/RoaringBitmap/roaring code.gitea.io/gitea/modules/indexer/code github.com/blevesearch/bleve github.com/blevesearch/bleve/index/scorch github.com/RoaringBitmap/roaring
  27. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem
  28. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list
  29. What is not ok? (only for me?) • solving author’s

    problem • let’s look at a broader problem • not extensible • golden file or nothing • allow/block list • already feature complete • boooooooo, we need a room for the improvements!
  30. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed
  31. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me!
  32. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found
  33. Welcome impguard • read as „import guard” • helps to

    verify • what is imported • what files are presented • what rules are passed • community driven! • have an idea? - make an issue or ping me! • also no relations to „imperial guard” found yet?…
  34. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things
  35. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic
  36. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file
  37. How does it work? • go list -json all |

    impguard project.yaml • go list -json all • returns a loooooot of things • | • simple Unix magic • impguard project.yaml • checks results based on config file • simple, isn’t ?
  38. go list gitea ❯ time go list -json all >

    1.txt go list -json all > 1.txt 3.07s user 2.63s system 376% cpu 1.516 total ❯ du -sh 1.txt 4.0M 1.txt
  39. Show me examples! "*": - name: must have doc.go checks:

    type: has_file pattern: doc.go - name: explicit runtime is forbidden checks: type: has_import pattern: runtime exclude: github.com/company/runtimeutil
  40. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid
  41. Show me more examples! github.com/company: - name: don't use bad

    UUID libs checks: type: no_import patterns: - github.com/oh-please-no/uuid - github.com/this-too-please/go-uuid - name: no vendors, just modules checks: type: no_dir pattern: vendor
  42. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md
  43. Show me some more examples! github.com/company/project/cmd/*: - name: no subpackages

    for cmd checks: - type: no_folders pattern: "*" - type: has_file patterns: - Dockerfile - main.go - README.md github.com/company/project/services/*: - name: services should be independent checks: - type: no_import pattern: github.com/company/project/services/*
  44. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard
  45. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas!
  46. Where is the code Lebowski? • good question • will

    be available • https://github.com/cristaloleg/impguard • feel free to share your ideas! • no, really
  47. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi
  48. Thanks • Yegor Myskin • Iskander (Alex) Sharipov • Bohdan

    Storozhuk • Alexey Palazhchenko • Sylwia Barbachowska • Roman Bystrytskyi • you and Golang Poland <3
  49. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`)
  50. ruleguard // +build ignore package gorules import "github.com/quasilyte/go-ruleguard/dsl/fluent" func _(m

    fluent.Matcher) { m.Match(`$x || $x`, `$x && $x`). Where(m["x"].Pure). Report(`suspicious identical LHS and RHS`) m.Match(`!($x != $y)`).Suggest(`$x == $y`) m.Match(`!($x == $y)`).Suggest(`$x != $y`) }