Managing Your Node.js Monorepos With NX

Transcript

1. Tamar Twena-Stern Implementing Monorepo in your Node.js project with NX
2. Tamar Twena-Stern twitter @SternTwena linkedin /tamarstern tamar.twena@gmail.com
3. Traditional Microservices Architecture
4. Should I Write My Code In The Same Git Repository ? Or Should I Write Every Service In It’s Own Repository ?
5. Monorepo - All micro service in one git repository Multi-repo - Every micro service in a separate git repo
6. Choose Monorepo Or Multi Repo ? There is a real debate on that question
7. Pros And Cons
8. In Multi-repos - All Those Are Easy Maintain version for each component Each service to have it’s own deployment cycle and releases Define access control to different components
9. Disadvantages Of Multi-repos Manage dependencies Multiple copy-paste Refactoring across multiple repositories
10. Since We Will Focus On Mono-repos Let’s Dive To Them
11. Advantages Of Mono-repos Easier for new developers to get familiar with the code and setup environment Less copy-paste, more code reuse between components Easier to refactor code across services / components Separate development teams to share the same culture - libraries, programming methodologies, development tools
12. Disadvantages Of Mono-repos Maintain feature branches for a long period, conflict resolving becomes very complicated Version control slow down - Pulling, cloning, checking out becomes long, it slows down the team’s work Managing access control - Impossible to block read access to all the code
13. The Nightmare - One Build To Rule Them All
14. Long, Complicated build process and CI/CD No ability to test separately only specific service Inability to version and deploy specific service One Build For All Services
15. NX - A Tool To Manage Mono-Repos
16. NX In A Nutshell A tool to manage all your code components in one repo Gives the ability to build, deploy and test each component separately
17. Building An NX Project With Several NPM Packages
18. Creating a new NX project
20. Initial Project Structure After Creation- No Packages Added
21. Project Structure Under Packages - All the packages in the repo will be located. Each package will have each own package.json and it’s own node_modules folder A global package.json to control the main build Typescript is not installed - if you use it , you need to install it globally/ per package
22. Creating A First Package For Generating Random Number
23. Creating A Folder And Module Files Manually
24. Index.ts const getRandomInt = (max) : number => { return Math.floor(Math.random() * max); } export default getRandomInt;
25. Package.json { "name": "rand-number", "version": "0.0.0", "main": "dist/index.js", "devDependencies": {}, "scripts": { "build": "tsc index.ts --outDir dist" } }
26. Building rand-number
27. Creating Manually A Linked Module Called Print Number
28. Creating Manually A Linked Module Called Print Number
29. Print Number - Index.ts import getRandomInt from 'rand-number'; const printRandomInt = (max) : void => { console.log('Printing number' + getRandomInt(max)) } export default printRandomInt;
30. Print Number - Package.json { "name": "print-number", "version": "0.0.0", "main": "dist/index.js", "devDependencies": {}, "scripts": { "build": "tsc index.ts --outDir dist", "test": "jest --coverage", "lint": "eslint src --ext .ts" }, "dependencies": { "rand-number": "*" } }
31. NPM Install On Root Level Identified That Projects Are Linked
32. Building ‘print-number’ - ‘rand-number’ Will Be Built As Well
33. Building All The Project
34. Generate NX Visualization Of Package Dependencies
35. NX Core Features
36. NX Tasks - All Scripts Defined In Package.json "scripts": { "build": "tsc index.ts --outDir dist", "test": "jest --coverage", "lint": "eslint src --ext .ts" } A project can have thousands of tasks !
37. NX Tasks • Each script in package json defined as a task • Possibility to run single task, multiple tasks or all of them • If a specific task is running - nx will run all the task dependencies • Possibility to control the order of tasks
38. NX Caching System
39. Setup The Cache- nx.json { "extends": "nx/presets/npm.json", "$schema": "./node_modules/nx/schemas/nx-schema.json", "tasksRunnerOptions": { "default": { "runner": "@nrwl/nx-cloud", "options": { "cacheableOperations": [ "build", "lint", "test", "E2e" ], "accessToken": "<GENERATED_TOKEN>" } } } }
40. Setup The Cache- nx.json { "extends": "nx/presets/npm.json", "$schema": "./node_modules/nx/schemas/nx-schema.json", "tasksRunnerOptions": { "default": { "runner": "@nrwl/nx-cloud", "options": { "cacheableOperations": [ "build", "lint", "test", "E2e" ], "accessToken": "<GENERATED_TOKEN>" } } } }
41. How The Caching Hash Is Generated
42. Distributed Caching
43. What The Cache Is Storing ? • Build Operations - Output of typescript compiler / babel • Test - test results of Jest/Mocha • Lint - ESlint results
44. What Nx Cache Adds Above TypeScript Compiler Build Optimizations ?
45. Caching Build Operations - TypeScript And NX
46. The Project Graph
47. The Elements Used To Build The Graph import getRandomInt from 'rand-number'; "dependencies": { "rand-number": "*" } Typescript Imports: Package.json dependencies
48. What You Can Do With The Project Graph ? See How Projects Are dependent Technical Description Of Each Module
49. The Project Graph Makes Detecting Circular Dependencies To A Much Easier Task
50. We talked a lot about NX packaged project . Does NX has other project types ?
51. Node.js NX Project
52. Generating Node.js Project Choose ‘node-standalone’ from the options presented
53. General Information - Node.js NX Project • A skeleton project is generated with native support of Typescript and Jest. • A single API file is generated under the root directory containing express endpoints • Utilities can be generated as a NX modules inside the project.
54. The Project Structure - Main Entry Point For Express Routes
55. Main Server File Under src/ Folder - Node.js Project import express from 'express'; import { doAuth } from '@products-api/auth'; const host = process.env.HOST ?? 'localhost'; const port = process.env.PORT ? Number(process.env.PORT) : 3000; const app = express(); app.get('/', (req, res) => { res.send({ message: 'Hello API' }); }); app.post('/auth', (req, res) => { res.send(doAuth()); }); app.listen(port, host, () => { console.log(`[ ready ] http://${host}:${port}`); });
56. Project Structure For An Internal Module Module is generated with a test file
57. Which NX Project Type Fit To Which Component In Our Architecture ? Packaged Based NX Project
58. Which NX Project Type Fit To Which Component In Our Architecture ? Node Standalone NX Project
59. How NX Helps Us To Solve Problems In The Mono-repo Approach ?
60. Separate Build For every Package Use That Command In Your CI/CD Tool
61. Build The Service Separately Gives Us The Ability To deploy It separately
62. Testing Each Component Separately • A CI / CD which contains all the tests of all the micro services of the component can be very complex. • By adding a test script for each microservice package.json we can build a CI/CD that will enable you to test each component by itself.
63. NX Cache - Solving Long Builds Problem NX cache optimizes the build process saves a lot of time - especially when you have hundreds of different services and libraries Critical aspect that will make you consider add it to you projects
64. Strategy To Integrate Existing Code To A Packaged Based Project
65. Start With Making Sure Your Code Dependencies Are Aligned
66. Suggested Code Structure
67. And If They Are Not Aligned - Start Breaking Them
68. Best Project Type To Integrate Existing Code With - Packaged Based NX Project
69. Why ‘Packaged Based’ Is The Best Project Type To Integrate Existing Code With ? • You generally leave the module/express/nest code’s existing build/test/lint tooling untouched. • In other NX project types - you need to change your build/test/lint tools to fit to NX built in tools
70. Steps To Integrate Existing Code With Packaged Based NX Project • Move an existing express/nest/utility into a package-based including package.json • NX will detect the your code as a package, cache the relevant scripts and build the package dependency graph.
71. Tamar Twena-Stern twitter @SternTwena linkedin /tamarstern tamar.twena@gmail.com
72. Questions ?