Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Building a full-stack app with Golang and Googl...

Felix Raab
October 18, 2018

Building a full-stack app with Golang and Google Cloud Platform in one week 

The talk will cover how to effectively build a production-ready, full-stack app with Golang and GCP under time constraints. I'll discuss how to approach making quick and sound technical decisions and how to apply modern software engineering practices for end-to-end apps. The presentation shows, in an opinionated and "meme-ful" way, various lessons learned, tools, and key takeaways for cloud environments.

Felix Raab

October 18, 2018
Tweet

More Decks by Felix Raab

Other Decks in Technology

Transcript

  1. Memes in the Cloud – Building a full-stack app with

    Golang and Google Cloud Platform in one week Dr. Felix Raab | Head of Engineering @ KI labs | ki-labs.com [email protected] | medium.com/@fe9lix
  2. SCOPE OF TALK Case Study Project idea and journey for

    an MVP finished within one week Software Engineering for Cloud Apps The “full” stack: Frontend, Backend, Build, Deployment, Logging… Towards the end of the talk: Software Engineering practices in Go
  3. SCOPE OF TALK Case Study Project idea and journey for

    an MVP finished within one week Software Engineering for Cloud Apps The “full” stack: Frontend, Backend, Build, Deployment, Logging… Towards the end of the talk: Software Engineering practices in Go Memes At the heart of the app itself – consequently, Memes to illustrate some opinionated views
  4. BACK AT MY PREVIOUS EMPLOYER… Hours claiming – trying not

    get “shitlisted”… because the boss would be notified!
  5. FIRST: HOW TO CHOOSE A CLOUD? …when you can start

    from scratch – no corporate constraints, no restrictions, no Frankfurt.
  6. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON

    WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise
  7. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON

    WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops
  8. LOOK AT THE HISTORY OF CLOUD COMPANIES MICROSOFT AZURE AMAZON

    WEB SERVICES GOOGLE CLOUD PLATFORM Enterprise Ops S(R)E
  9. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… API access, permissions, config…
  10. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… CI/CD API access, permissions, config…
  11. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… CI/CD Database (NoSQL document store) API access, permissions, config…
  12. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) API access, permissions, config…
  13. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) Task Queue (with scheduling!) API access, permissions, config…
  14. WORKS LIKE YOU WOULD EXPECT IT… Service Offerings used to

    build MemeMail More on that later… CI/CD Database (NoSQL document store) Object Storage (Blobs…) Task Queue (with scheduling!) Logging… API access, permissions, config…
  15. RESIST THE TEMPTATION TO… Also, do not use Github Stars

    as your only criterion! Popularity is one thing…
  16. HOW TO CHOOSE 1) Does it solve your problem in

    a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base.
  17. HOW TO CHOOSE 1) Does it solve your problem in

    a simple way? Rendering the frontend and adding interaction without sprinkling jQuery calls throughout your code base. 2) Look at the quality of the documentation! Because you need to read it and be able to understand it – quickly!
  18. STATE MUTATION Take notice when somebody tells you that you

    should or can entirely avoid mutating state.
  19. MUTATING STATE. HERE’S HOW… 1) Put your entire app state

    into a single model Avoid primitive obsession – use proper objects! This is your single source of truth.
  20. MUTATING STATE. HERE’S HOW… 1) Put your entire app state

    into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads.
  21. MUTATING STATE. HERE’S HOW… 1) Put your entire app state

    into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads. 3) Accept or reject the state mutation in your model At the end of your (async.) action, “commit” / “propose” a new value to the model.
  22. MUTATING STATE. HERE’S HOW… 1) Put your entire app state

    into a single model Avoid primitive obsession – use proper objects! This is your single source of truth. 2) Dispatch Actions from the UI to trigger a state mutation Do not yet mutate the state in the action itself! Actions are intents with payloads. 3) Accept or reject the state mutation in your model At the end of your (async.) action, “commit” / “propose” a new value to the model. 4) Update the UI when the state changes Derive the state to render from your model, e.g. by UI bindings.
  23. –Ryan Dahl “[...] if you’re building a server, I can’t

    imagine using anything other than ” THE BACKEND.
  24. HOW TO BUILD? Go 1.11 Because Go Modules! Docker To

    be flexible with Go versions
 Multi-stage build!
  25. HOW TO BUILD? Go 1.11 Because Go Modules! Docker To

    be flexible with Go versions
 Multi-stage build! entr + Makefile No Rake, Grunt, Gulp, NPM, Yarn – Makefile + Bash Script fine! Check entrproject.org
  26. APP ENGINE – START SIMPLE App Engine Flex You can

    migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster.
  27. APP ENGINE – START SIMPLE App Engine Flex You can

    migrate to Kubernetes (GKE) later if you wish to – but for your MVP, don’t waste time managing a cluster. Use defaults Start with App Engine default options and auto-scaling. Nobody knows how much traffic you will get. Watch and see what happens.
  28. YES AND NO! Expose your services as API to the

    frontend Frontend is rendered via a Go template. On that rendered page, a Single Page App can run in memory and call your backend API. Use Go templates for other static pages. Makes routing easier.
  29. YES AND NO! Expose your services as API to the

    frontend Frontend is rendered via a Go template. On that rendered page, a Single Page App can run in memory and call your backend API. Use Go templates for other static pages. Makes routing easier. But no need to create two deployment units No need for premature optimisation (scaling). One Go binary is simpler than two Go binaries! 
 Static assets are loaded from the file system in your container.
  30. CI/CD Google Cloud Build Will trigger and deploy when you

    push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think.
  31. CI/CD Google Cloud Build Will trigger and deploy when you

    push to your Git repo. Maintaining your own Jenkins server and setting up a build pipeline can be more time-consuming than you might think. App Engine Deployment Straightforward versioning and service concept via DNS – version.service.appname Promoting app to production, traffic splitting etc. – 
 supported out of the box
  32. PLAYGROUNDS SAVE TIME Create a GCP “Playground” project Got stuck

    with Cloud Datastore (vs. Cloud Firestore) and 
 Cloud Tasks (only certain EU region). Led me to test service offerings first via playground.
  33. SO MANY LAYERS AND CONCEPTS Also, ask yourself : Is

    there a rich domain? Or are you really building a CRUD app?
  34. Take notice when nobody tells you to separate functionality into

    lots and lots of really small methods and delegating objects.
  35. DEEP VS SHALLOW MODULES Shallow Module “The best modules are

    deep: they allow a lot of functionality to be accessed through a simple interface. A shallow module is one with a relatively complex interface, but not much functionality: it doesn’t hide much complexity.” 
 – From A Philosophy of Software Design, John Ousterhout Deep module Interface Functionality
  36. EXAMPLE OF A DEEP MODULE Meme Image Generation Package “meme”,

    image.go: Simple interface, a lot of functionality.
  37. Separate the outside from the inside Kind of onion, just

    fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers
  38. Separate the outside from the inside Kind of onion, just

    fewer layers Outside (package http) Protocol, routing (gorilla/mux), handlers Inside (package meme) Core, your actual (business) functionality
  39. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here… To be

    able to inject a different implementation, you would need to either create an interface that mirrors the datastore api or create a generic interface -> shallow wrapper
  40. IMPLICIT INTERFACE CONFORMANCE Awesome Golang feature, but here… To be

    able to inject a different implementation, you would need to either create an interface that mirrors the datastore api or create a generic interface -> shallow wrapper Have a look at google/go-cloud for unstructured binary storage (supports GCP and AWS)
  41. STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But

    Go’s default logging has limitations. Even better: Structured Logging Implicit conformance to uber-go/zap for structured logging
  42. STRONGER ARGUMENT FOR AN INTERFACE? Logging Needed almost everywhere! But

    Go’s default logging has limitations. Even better: Structured Logging Implicit conformance to uber-go/zap for structured logging
  43. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of

    passing the logger through method parameters, pass a Go context object through middleware.
  44. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of

    passing the logger through method parameters, pass a Go context object through middleware.
  45. CONTEXT OBJECT PATTERN Reduce coupling of method parameters Instead of

    passing the logger through method parameters, pass a Go context object through middleware. Still not ideal, but good enough? Also see: https://medium.com/@gosamv/using-gos-context-library-for-logging-4a8feea26690
  46. FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should

    see the most recent list of meme reminders 
 (strong consistency supported by Cloud Datastore)
  47. FINALLY, BEING CONSISTENT… Strong Consistency After creating, the user should

    see the most recent list of meme reminders 
 (strong consistency supported by Cloud Datastore) Eventual Consistency Meme detail view might not have been generated and stored yet – 
 but you can always apologise to the user. Eventual consistency is often fine.
  48. CONCLUSION Keep it simple (but not stupid!) Reduce the number

    of “things” you need to deal with – both from an architecture and a product perspective.
  49. CONCLUSION Keep it simple (but not stupid!) Reduce the number

    of “things” you need to deal with – both from an architecture and a product perspective. Challenge existing “best practices” Have you been brainwashed by cargo cults in the past? The “best” practices always depend on the context and scope.
  50. CONCLUSION Keep it simple (but not stupid!) Reduce the number

    of “things” you need to deal with – both from an architecture and a product perspective. Challenge existing “best practices” Have you been brainwashed by cargo cults in the past? The “best” practices always depend on the context and scope. Develop end-to-end Allows you to make better decisions for the entire system – and to iterate quickly for your MVP!