Frontend Archaeology

Given at Codelicious on 7th July 2016

Transcript

1. Frontend Archaeology Digging through TweetDeck for lessons at scale @tgvashworth 1
2. Tom Ashworth @tgvashworth Tech Lead, TweetDeck @tgvashworth 2
3. @tgvashworth 3
4. @tgvashworth 4
5. TweetDeck A few million users. Six billion API requests every day. Maybe eighty thousand lines of JavaScript. Six years of git log. @tgvashworth 5
6. Topics Architecture & refactoring Complexity & communication Operations @tgvashworth 6
7. As frontend technology matures, so must our practice. @tgvashworth 7
8. Architecture & refactoring @tgvashworth 8
9. Flight.js Event-driven JavaScript framework Used on TweetDeck & Twitter.com As a company, we're moving to React @tgvashworth 9
10. const Example = component(example, withExtras); function example() { this.after("initialize", function () { this.on("click", this.onClick); }); this.onClick = function (event) { this.trigger("exampleWasClicked", { some: "data" }); }; } @tgvashworth 10
11. "Attaching" a component to the DOM: Example.attachTo("#example", { config: "data" }); @tgvashworth 11
12. Using "advice": this.onClick = function () { ... }; this.before("onClick", this.doAThing); this.after("onClick", this.doSomethingElse); @tgvashworth 12
13. Flight had some problems: » Couldn't nest components » No standard state management » Events for data flow @tgvashworth 13
14. Frameworks we like: » React (user interface library) » Elm (language for functional & reactive UI) » Cycle (functional & reactive UI in JS) @tgvashworth 14
15. Ideas we like: » Component nesting & composition » Easy, predictable state management » Normal functions for data manipulation @tgvashworth 15
16. In Flight, nesting components was icky: » Tightly coupled » Not reusable » Hard to debug (because events) @tgvashworth 16
17. // Parent this.trigger("parentTearingDown"); // Child this.on("parentTearingDown", this.teardown); @tgvashworth 17
18. @tgvashworth 18
19. @tgvashworth 19
20. @tgvashworth 20
21. It's this easy now: this.attachChild( Child, this.select('childNode') ); @tgvashworth 21
22. State management was ad-hoc: this.active = true; this.$node.attr("disabled", false); this.attr.tooltip = "..."; this.update(); @tgvashworth 22
23. We created a mixin called withState: this.mergeState({ active: true, tooltip: "Hello!" }); this.after("stateChanged", this.render); @tgvashworth 23
24. this.on("usersResponse", (e, response) => { if (response.requestId === this.id) { response.users.map(user => { this.trigger("tweetsRequest", { requestId: this.id, userId: user.id }); }); } }); this.on("tweetsResponse", (e, response) => { /* ... */ }); this.trigger("usersRequest", { requestId: this.id }); @tgvashworth 24
25. Users.getAll() .then(users => Promise.all( users.map(user => Tweets.getByUser(user.id)) )); @tgvashworth 25
26. Ideas we like: » Functional programming » Promises » Observables @tgvashworth 26
27. Observables with RxJS: » Observable: represents a collection of future values » Observer: a collection of callbacks » Operators: pure functions like map, filter, concat, flatMap... @tgvashworth 27
28. // Poll the force-refresh endpoint this.observe(this.getTimer()) .flatMap(this.getVersion) .map(this.processVersion) .do(this.sendMetrics) .filter(this.shouldRefresh) .subscribe(this.refresh); @tgvashworth 28
29. Distill good tech into great ideas. You can apply ideas anywhere. @tgvashworth 29
30. Refactoring @tgvashworth 30
31. Way back in 2015... » require.js & AMD modules » Monoglobal (TD) » Bower dependency management @tgvashworth 31
32. We had... define(["lodash"], function (_) { // . . . }); and we had... define(function (require) { var _ = require("lodash"); // . . . }); @tgvashworth 32
33. But we wanted... var _ = require("lodash"); and eventually... import _ from "lodash"; @tgvashworth 33
34. codemod noun A large-scale codebase refactor, often mechanical or repetitive. js-codeshift noun Toolkit for running automated codemods. @tgvashworth 34
35. module.exports = function (fileInfo, api) { return api .jscodeshift(fileInfo.source) .findVariableDeclarators('foo') .renameTo('bar') .toSource(); }; @tgvashworth 35
36. @@ -3,10 +3,9 @@ * * Manages & removing tweets from collections */ -define([ - 'flight/lib/component', - 'data/with_client' -], function (defineComponent, withClient) { +define(function (require) { + var defineComponent = require('flight').component; + var withClient = require('data/with_client'); return defineComponent(customTimelines, withClient); @tgvashworth 36
37. commit 3f617af133cb251ca10a0fadf223a49e71be2440 Author: Tom Ashworth <tashworth@twitter.com> Date: Mon Nov 16 15:47:18 2015 +0000 . . . 70 files changed, 3508 insertions(+), 3617 deletions(-) @tgvashworth 37
38. Three step refactor recipe: 1. Find all the patterns 2. Choose the two most similar 3. Unify with a codemod Repeat. @tgvashworth 38
39. Evolving Complex Systems Incrementally by Christoph Pojer facebook/jscodeshift reactjs/react-codemod cpojer/js-codemod benjamn/recast @tgvashworth 39
40. @tgvashworth 40
41. Complexity & communication @tgvashworth 41
42. Over time we had acquired... » Lots of complexity » Dependency on people, not systems » Word-of-mouth sharing @tgvashworth 42
43. Complexity @tgvashworth 43
44. Complexity: intrinsic or incidental? @tgvashworth 44
45. Intrinsic complexity » Fundamental to the problem » Evident in design » Bounded @tgvashworth 45
46. Incidental complexity » Unrelated to a specific problem » Emergent over time » Unbounded @tgvashworth 46
47. Code is entropic; it naturally tends to disorder. @tgvashworth 47
48. Some factors in increasing entropy and incidental complexity: » Changing requirements (it did X, now it must do Y too) » Bugs (ugh, it broke, fix it quick) » Intrinsic complexity (complexity replicates) @tgvashworth 48
49. Take an active stance against debt & decay. @tgvashworth 49
50. How we've addressed it... » Portfolio approach » Feature and Platform balance » Advocate for the codebase @tgvashworth 50
51. PM & Design: user advocates. You: codebase advocate. @tgvashworth 51
52. Sharing knowledge @tgvashworth 52
53. Avoid complexity & decay by sharing knowledge: » Pairing » Onboarding » Code review @tgvashworth 53
54. Take time to design your onboarding experience. @tgvashworth 54
55. A great onboarding has side-effects: » Documentation! » More outside contributions » Faster incident management @tgvashworth 55
56. @tgvashworth 56
57. Code review! We all do it... right? @tgvashworth 57
58. Code review is the single most effective practice you can introduce to improve your team's work. @tgvashworth 58
59. Every pull request gets a +1 from someone else. @tgvashworth 59
60. Over time, we learned some things about code review: » Don't review for more than hourDunsmore 2000 » Keep reviews smaller than ~400 linesCohen 2006 » Code review your own code firstCohen 2006 Yes, this slide features science. Cohen 2006 Cohen, Jason. 2006. Best Kept Secrets of Peer Code Review. Proceedings of the 22nd ICSE 2000: 467-476. Dunsmore 2000 Dunsmore et al. 2000. Object-Oriented Inspection in the Face of Delocalisation. Beverly, MA: SmartBear Software. @tgvashworth 60
61. Making Software What Really Works, and Why We Believe It @tgvashworth 61
62. @tgvashworth 62
63. Operations @tgvashworth 63
64. Way-er back in 2014: » Scary deploys (a nervous day in person-hours) » Built from a laptop » Manual testing checklist » Big commits (hard to code review) @tgvashworth 64
65. Scary deploys: » Manual » Infrequent » Lots of code » Slow feedback loop (what broke?) @tgvashworth 65
66. Optimise for confidence by tightening feedback loops. @tgvashworth 66
67. A tight feedback loop lends agility and confidence leading to momentum. @tgvashworth 67
68. Practical ways to optimise for confidence: » Work on master (branches mean big merges) » Use feature flags (no-sweat releases) » Deploy as often as you can » Use alerts to spot trouble @tgvashworth 68
69. Feature flags? function enabled(id, flag, threshold) { return hash(id + flag) > threshold; } const showSuperCoolFeature = enabled(user.id, "super_cool_feature", 50); @tgvashworth 69
70. Track everything with fire-and-forget metrics. @tgvashworth 70
71. Why can't it be this easy? count("login/forgot-password/click"); distribution("api/request/time", 100); @tgvashworth 71
72. In summary » As frontend technology matures, so must our practice. » Find and reapply good ideas. » Take an active stance against debt & decay. » Optimise for confidence by tightening feedback loops. @tgvashworth 72
73. Thanks ❤ Tom Ashworth @tgvashworth ta@twitter.com Special thanks to the team @TwitterUK, especially @passy, for feedback! @tgvashworth 73