How Sprockets works

Almost all applications have assets like CSS, JavaScript and others. That means the asset pipeline is an integral part of the Ruby on Rails framework. In this talk we'll show you how the asset pipeline works, and how you can take full advantage of the asset pipeline's features. Ever wondered how to convert an SVG to PNG automatically? Wanted to know what exactly happens to your CoffeeScript files? We'll explore that, and more.

Transcript

1. How Sprockets works
2. Rafael França rafaelfranca rafaelfranca
3. Core Member
5. Maintainer
6. Ruby on Rails Bot rails-bot
7. Spring Turbolinks jquery-ujs Action View Active Record Active Job Action Controller Action Mailer web-console Sprockets rails-html-sanitizer
8. Agenda • Why do we need an assets pipeline? • The gems responsible • How it works in a Rails application • How to extend it
9. Why do we need an assets pipeline?
10. Where should I put my assets?
11. public/*
12. Code organization X Performance
13. Small self-contained files
14. Fewer assets request
15. Legible code
16. Fewer bytes being transferred
17. New technologies • CoffeeScript • Sass • ES6
18. Rails assets pipeline
19. How it works in Rails
20. Assets live in app/assets • app/assets/stylesheet • app/assets/javascript • app/assets/images • There are also: lib/assets/ and vendor/assets/
21. Assets are compiled on-the-fly in development and need to be precompiled in production
22. Assets names are generated with digest to cache busting • public/assets/application- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322 950ec65b6.js
23. The gems responsible
24. Gems • sprockets • sprockets-rails • sass-rails • execjs • coffee-rails
25. sprockets
26. Compiles and serves assets
27. Processors pipeline
28. Sprockets' key components • processors • transformers • compressors • directives • environment • manifest • pipelines
29. Processor
30. Most important component in Sprockets
31. Any callable. Accepts an input Hash and returns a Hash of metadata
32. -> (input) { data = input[:data].gsub(";", "") { data: data } }
33. Input hash • :data - String asset contents • :environment - Current Sprockets::Environment instance. • :cache - A Sprockets::Cache instance. • :uri - String Asset URI. • :source_path - String full path to original file. • :load_path - String current load path for filename. • :name - String logical path for filename. • :content_type - String content type of the output asset. • :metadata - Hash of processor metadata.
34. Return hash • :data - Replaces the assets input[:data] to the next processor in the chain. • :required - A Set of String Asset URIs that the Bundle processor should concatenate together. • :stubbed - A Set of String Asset URIs that will be omitted from the :required set. • :links - A Set of String Asset URIs that should be compiled along with this asset. • :dependencies - A Set of String Cache URIs that should be monitored for caching. • :map - An Array of source maps for the asset. • :charset - The mime charset for an asset.
35. Builtin processors • BabelProcessor • CoffeScriptProcessor • SassProcessor
36. Bundler processor
37. A processor that run concatenated assets rather than individual files
38. register_bundle_processor 'application/javascript', Bundle register_bundle_processor 'text/css', Bundle
39. Bundle processor takes a single file asset and prepends all the `:required` URIs to the contents.
40. Transformer
41. A processor that convert a file from one format to another
42. register_transformer ‘text/coffeescript', ‘application/javascript', CoffeeScriptProcessor
43. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js, map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
44. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js, map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
45. module CoffeeScriptProcessor # ... def self.call(input) data = input[:data] js, map = input[:cache].fetch([self.cache_key, data]) do result = CoffeeScript.compile(data, sourceMap: true, sourceFiles: [input[:source_path]]) [result['js'], decode_source_maps(result['v3SourceMap'])] end map = SourceMapUtils.combine_source_maps(input[:metadata][:map], map) { data: js, map: map } end end
46. Compressor
47. Special kind of bundle processor
48. register_compressor ‘application/javascript’, :uglify, UglifierCompressor
49. env.js_compressor = :uglify
50. Directives
51. Special comments that declares bundlers and their dependencies
52. // app/assets/javascripts/application.js //= require jquery //= require jquery-ui //= require users //= require_tree .
53. Rails.application.config.assets.precompile << %w(application.js application.css)
54. LOOSE_APP_ASSETS = lambda do |logical_path, filename| filename.start_with?(::Rails.root.join("app/assets").to_s) && !['.js', '.css', ''].include?(File.extname(logical_path)) end config.assets.precompile = [LOOSE_APP_ASSETS, /(?:\/|\\|\A)application\.(css|js)$/]
55. // app/assets/config/manifest.js //= link_tree ../images //= link_directory ../javascripts .js //= link_directory ../stylesheets .css //= link my_engine // my_engine/app/assets/config/my_engine.js //= link_tree ../images/bootstrap
56. Directives • require • require_directory • require_tree • require_self • link • link_directory • link_tree • depend_on • depend_on_asset • stub
57. Environment
58. It has methods to retrieve and serve assets, manipulating load path, and registering processors.
59. environment = Sprockets::Environment.new environment.find_asset('application.js')
60. environment = Sprockets::Environment.new environment.register_transformer 'image/svg+xml', 'image/png', SVGTransformer.new
61. What is the manifest
62. Logs the contents of assets compiled to a single directory. It records basic attributes about the asset for fast lookup without having to compile. A pointer from each logical path indicates which fingerprinted asset is the current one.
63. Points the asset path to the fingerprinted version
64. javascript_include_tag 'application' # <script # src="/assets/application.debug- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322950ec65b6.js" # ></script>
65. { "application.js" => "application-2e8e9a7c6b0aafa0c9bdeec90ea30213.js", "jquery.js" => "jquery-ae0908555a245f8266f77df5a8edca2e.js" }
66. { "application-2e8e9a7c6b0aafa0c9bdeec90ea30213.js" => { 'logical_path' => "application.js", 'mtime' => "2011-12-13T21:47:08-06:00", 'digest' => "2e8e9a7c6b0aafa0c9bdeec90ea30213" } }
67. More • mime types • dependency resolver • tranformer suffix • bundle metadata reducer
68. sprockets-rails
69. Integrates Sprockets to the Rails application
70. Defines helpers
71. • javascript_include_tag • stylesheet_link_tag
72. Configures the Sprockets environment
73. Checks the precompile list
75. sass-rails
76. Integrates the Sass processor with the Rails application
77. Defines generators
78. Creates a importer that knows how to handle globs and ERB
79. @import "foo/*"; // bar.scss.erb @import "bar";
80. Configures the Sass processor
81. execjs
82. It allows you to run JavaScript code from Ruby
83. It uses the JavaScript environment available in the machine
84. • therubyracer - Google V8 embedded within Ruby • therubyrhino - Mozilla Rhino embedded within JRuby • Duktape.rb - Duktape JavaScript interpreter • Node.js • Apple JavaScriptCore - Included with Mac OS X • Microsoft Windows Script Host (JScript) • Google V8
85. require "execjs" require "open-uri" source = open("http://coffeescript.org/extras/coffee-script.js").read context = ExecJS.compile(source) context.call("CoffeeScript.compile", "square = (x) -> x * x", bare: true) # => "var square;\nsquare = function(x) {\n return x * x;\n};"
86. It is used by the coffee-script gem to compile CoffeeScript code
87. coffee-rails
88. Configures generators
89. Defines a template handler
90. How the assets are generated in development.
91. <%= javascript_include_tag 'application' %>
92. javascript_include_tag 'application' # <script # src="/assets/application.debug- ddbd4593b22ac054471df143715c8ce65ef84938965c7db19d8a322950ec65b6.js" # ></script>
93. GET /assets/application.debug- *.js
94. sprockets-rails use the Sprockets’ pipeline `debug`
95. register_pipeline :debug do [SourceMapCommentProcessor] end
96. The SourceMapCommentProccessor will generate the asset bundle and add the source map comment
97. //# sourceMappingURL=application.js- bf4c2d805fe200d419d3ea539cd105732f1e1b2848689fd449bce22a6be39c88.map
98. To build the asset bundle the default pipeline is used
99. register_pipeline :default do |env, type, file_type| env.default_processors_for(type, file_type) end
100. def default_processors_for(type, file_type) bundled_processors = config[:bundle_processors][type] if bundled_processors.any? bundled_processors else self_processors_for(type, file_type) end end
101. def default_processors_for(type, file_type) bundled_processors = config[:bundle_processors][type] if bundled_processors.any? bundled_processors else self_processors_for(type, file_type) end end
102. The bundle pipeline will compile all required files and merge them
103. To compile the required files the self pipeline is used
104. register_pipeline :self do |env, type, file_type| env.self_processors_for(type, file_type) end
105. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
106. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
107. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
108. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
109. def self_processors_for(type, file_type) processors = [] processors.concat config[:postprocessors][type] if type != file_type && processor = config[:transformers][file_type][type] processors << processor end processors.concat config[:preprocessors][file_type] if processors.any? || mime_type_charset_detecter(type) processors << FileReader end processors end
111. The bundler processor merge all of them
112. And the result is sent to the browser
113. What are the key difference between development and production
114. In production all of this happens in the precompile task and only a static asset is returned
115. Extending it
116. Creating new directives
117. class NpmDirectiveProcessor < Sprockets::DirectiveProcessor def process_npm_directive(path) dirs = node_modules_paths(@filename) uri, deps = @environment.resolve!( path, accept: @content_type, pipeline: :self, load_paths: dirs ) @dependencies.merge(deps) @required << uri end # ... end
118. register_preprocessor 'application/javascript', NpmDirectiveProcessor.new(comments: ["//", ["/*", "*/"]])
119. // app/assets/javascripts/my_component.js //= npm lodash
120. How to create a new processor to transform SVGs to PNGs
121. environment.register_transformer 'image/svg+xml', 'image/png', SVGTransformer.new
122. // app/assets/config/manifest.js // Given you have foo.svg //= link foo.png // or //= link_tree ../images .png
123. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) { self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
124. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) { self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
125. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) { self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
126. require 'rmagick' class SVGTransformer def self.call(input) image_list = Magick::Image.from_blob(input[:data]) { self.format = 'SVG' } image = image_list.first image.format = 'PNG' { data: image.to_blob } end end
127. Sprockets is used in many Rails applications
128. Many users don’t know it exists
129. Many users don’t know how it works
130. Understand your tools
131. Document your understanding
132. Share with the community
133. Saving Sprockets Richard Schneeman Thursday, May 5 Room 3501 EF 2:40PM
134. We’re hiring! 70+ open positions in a wide range of disciplines Ottawa, Montreal, Toronto, Waterloo, Remote https://shopify.com/careers
135. Testing Rails at Scale Emil Stolarsky May 4, 4:30PM Room 3501 AB Rails 5 Features You Haven't Heard About Sean Griffin May 5, 11:40AM Room 3501 DC
136. Rafael França rafaelfranca rafaelfranca Thank you