Speeding up the build process of a monolithic application

Working on an application used by thousands of users might be hard. The app usually consists of multiple files with the source code and even thousands or millions of lines of code - it takes ages to compile. Splitting a monolith into modules and caching the binaries can speed up the build process!

The talk was presented on the iOS Conf SG 2021 on January 21st.

Transcript

1. Speeding up the build 
 process of a monolithic app Maciej Piotrowski
2. 20 million customers each month one of top ten most visited e-commerce sites worldwide mobile apps big chunk of traffic The most popular product search engine in Poland
3. 370 804 LOC 3.2 GHz 6-Core CPU 32 GB RAM 5+ minutes on CI servers 2.2 GHz 6-Core CPU 32 GB RAM 10+ minutes on workstations compiled
4. Swift + Objective-C (LoC) 400 000 500 000 600 000 329 692 676 303 19 month increment (Jun 2019 - Dec 2020)
5. when # lines of code grow the compilation time follows
6. Clean Build (s) 190 130 88 177 220 250 307 12 month increment (Jun 2019 - May 2020) 238
7. Clean Build (s) 190 130 88 177 220 250 307 19 month increment (Jun 2019 - Dec 2020) 238
8. Codebase lifecycle
9. Clean Build (s) 190 130 177 220 250 307 19 month increment (Jun 2019 - Dec 2020) 238 How did we compile 78 modules in 88 s? 88
10. How to achieve fast compile times? build & cache binaries of frameworks tuist.io bazel.build buck.build
11. BUILD project files bazel build //Modules/Cart single language codebase unified structure of sub-projects
12. one more thing … goal: provide seamless developer experience BUILD project files bazel build //Modules/Cart multi language codebase (Swift+ObjC) undefined structure of sub-projects
13. Unify projects’ structure
14. Unify projects’ structure and settings Cart-Debug.xcconfig
16. yonaskolb/XcodeGen
17. Describe project’s structure in an .yml manifest xcodegen -spec project.yml
18. How to enforce same structure for new modules? modulegen tool: unzips a template replaces <ProjectName> integrates into workspace links as a dependency
19. apple/swift-argument-parser
20. github.com/paciej00/replace-in-file-cli
21. ./replace project.yml <ProjectName> Cart
22. ./replace project.yml <ProjectName> Cart
23. ./modulegen generate NewFancyFeatureModule ./xcodegen generate --spec Modules/Cart/project.yml ./fixme.sh --force
24. ./steve generate NewFancyFeatureModule ./steve fixme -g ./steve fixme -f
25. OVERVIEW: Steve provides you a convenient way to solve daily development problems and complete repetitive tasks. USAGE: steve <subcommand> OPTIONS: --version Show the version. -h, --help Show help information. SUBCOMMANDS: fixme Fixes most common issues that developers encounter generate Allows to generate multiple items related to Allegro app bazel Use this command to invoke bazel with parameters focus Allows you to focus on selected modules from Allegro workspace test Run tests for given project and scheme
26. unified structure of sub-projects single language codebase BUILD file bazel build //Modules/Cart Bazel integration
27. Bazel BUILD file
28. bazel query 'deps(//Modules/Cart)' --noimplicit_deps --output package
30. bazel query 'rdeps(//Modules/..., //Modules/Networking)’ --output graph
32. Bazel BUILD file
33. Bazel rule allegro_xcodebuild_module.bzl
34. allegro_xcodebuild_module.bzl Bazel rule
35. Bazel rule allegro_xcodebuild_module.bzl
36. Bazel rule allegro_xcodebuild_module.bzl
37. Bazel rule allegro_xcodebuild_module.bzl
38. Bazel rule allegro_xcodebuild_module.bzl
40. ./steve focus cart
41. ./steve focus cart
42. Continuous Integration A CI plan with 2 jobs fills a remote cache per each PR and the main branch: - job building frameworks for iOS simulator - job building frameworks for iOS device
43. Bazel can cache any output by executing an action on inputs
44. bazel test //Modules/Cart:Cart_Tests
45. bazel test //Modules/Cart:Cart_Tests
46. Bazel gotchas
47. 190 130 88 177 220 250 307 238 Bazel gotchas
48. Bazel gotchas just opening Storyboard/XIBs can invalidate cache exclude .DS_STORE otherwise it can invalidate cache single language codebase not needed no progress bar while running Bazel by a run script in Xcode
49. from 32 to 78 in 1 year ./steve jobs more small modules = faster builds 307s 88s 57s ./xcodegen generate -s project.yml
50. allegro.tech Speeding up iOS builds with Bazel Speeding up warm builds in Xcode Maciej Piotrowski twitter.com/paciej00
51. https://allegro.tech/2020/12/speeding-up-ios-builds-with-bazel.html https://allegro.tech/2020/12/speeding-up-warm-builds.html https://bazel.build https://buck.build https://tuist.io https://docs.bazel.build/versions/3.7.0/tutorial/ios-app.html https://github.com/yonaskolb/XcodeGen https://github.com/bazelbuild/rules_apple https://github.com/apple/swift-argument-parser https://github.com/paciej00/replace-in-file-cli