Advanced JavaScript build pipelines using Gulp.js

It has been some time since JavaScript build tools like Grunt or Gulp were just the "next big thing" for web developers. Working without them is nearly unimaginable nowadays and it seems that there's almost no problem in our day to day workflow which cannot be solved by simply using just another plugin.

But are build tools really the answer to everything?

In this talk, we will take a look at the ten most common problems which seem to be unsolved by using just the build tool basics. We will create advanced building pipelines for our custom processes and find out reusable patterns which can be applied to similar issues.

Transcript

1. Advanced JavaScript Build Pipelines with Gulp.js
2. Let's talk a short bit about the JS build tool revolution
3. HTML CSS JavaScript
4. Sass CoffeeScript LESS P o s t C S S H A M L J a d e U g l i f y E S 6 R e a c t J S B r o w s e r i f y A n g u l a r J S E m b e r C S S M i n J S L i n t ESHint ImageOptim Mocha Jasmine TypeScript
9. 1530 lines of code original Ant tasks used: concat — copy — delete — mkdir
11. Maybe Java isn't the right tool? Java is to JavaScript what
 Alf is to Gandalf
12. Grunt started a boom
13. Gruntfiles get long
14. Grunt tasks get slow: lots of reads and writes
15. One unmanageable horde of (sometimes) low quality plug-ins
16. And then came Gulp
17. React is in love with Webpack
18. Ember.js digs Broccoli.js
19. Ember.js digs Broccoli.js
21. So… every community seems to have its preference Stick with your communities preference!
22. For all the rest? Gulp might be a good, very good option
23. Disclaimer
24. I (occasionally) contribute to Gulp
25. I'm writing a book on Gulp http://bit.ly/gulp-tool-book 39% off with 39baumgar
 coupon code!
26. gulp.src(…) gulp.dest(…) Reads files Writes files
27. gulp.src(…) .pipe(uglify()) gulp.dest(…)
28. gulp.src(…) .pipe(uglify()) gulp.dest(…) .pipe(concat())
29. task manager streaming file system file watcher
30. gulp.task('scripts', function() { return gulp.src(‘src/**/*.js') .pipe(uglify()) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist')); }); gulp.task('default', function(done) { gulp.watch(‘src/**/*.js', gulp.parallel(‘scripts')); done(); });
31. undertaker.task('scripts', function() { return vinyl.src(‘src/**/*.js') .pipe(uglify()) .pipe(concat('main.min.js')) .pipe(vinyl.dest('dist')); }); undertaker.task('default', function(done) { chokidar.watch(‘src/**/*.js', undertaker.parallel(‘scripts')); done(); });
32. so similar … yet so different? Gulp Streams Browserify 
 Streams
33. var source = require(‘vinyl-source-stream’); var b = browserify({ entries: ['_js/main.js'] }); var bundle = function() { return b.bundle() .pipe(source(‘main.js’)) .pipe(gulp.dest('js')); } gulp.task(‘bundle’, bundle);
34. This also goes the other way around…
35. var app = express(); var router = express.Router(); router.get('/*', function(req, res) { var stream = request('http://host.to.forward.to' + req.originalUrl); stream.pipe(res); stream .pipe(source('.' + req.originalUrl)) .pipe(gulp.dest('./cachedir')); }); app.use(express.static('_site')); app.use(express.static('cachedir')); app.use(router);
36. var app = express(); var router = express.Router(); router.get('/*', function(req, res) { var stream = request('http://host.to.forward.to' + req.originalUrl); stream.pipe(res); stream .pipe(source('.' + req.originalUrl)) .pipe(gulp.dest('./cachedir')); }); app.use(express.static('_site')); app.use(express.static('cachedir')); app.use(router);
37. So start thinking streams …
38. Multiple input formats return gulp.src(‘src/**/*.coffee) .pipe(coffee()) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’))
39. Multiple input formats return gulp.src(‘src/**/*.js) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’))
40. Multiple input formats return gulp.src(‘src/**/*.js) .pipe(uglify()) .pipe(concat(‘main.js’)) .pipe(gulp.dest(‘build’)) the same!
41. Multiple input formats And actually, you just want one bundle in the end
42. What if we could reuse parts of the stream?
43. return gulp.src(‘app1/src/**/*.coffee’) .pipe(coffee()) .pipe(uglify() .pipe(concat(‘main.js’)) .pipe(gulp.dest('build'));
44. var merge = require(‘merge2’); return merge(gulp.src(‘app1/src/**/*.coffee’) .pipe(coffee()), gulp.src(‘app1/src/**/*.js’)) .pipe(uglify() .pipe(concat(‘main.js’)) .pipe(gulp.dest('build'));
45. var merge = require(‘merge2’); return gulp.src(‘app1/src/**/*.coffee’) .pipe(markdown()), .pipe(gulp.src(‘app1/src/**/*.js’), {passthrough: true}) .pipe(rename(blogFn)) .pipe(wrap(layoutStr)) .pipe(swig()) .pipe(gulp.dest('build'));
46. Multiple bundles And now we need this over and over again for all our applications…
47. var elems = [ { dir: ‘app1’, bundleName: ‘app1.min.js’ }, { dir: ‘app2’, bundleName: ‘app2.min.js’ } ];
48. var streams = elems.map(function(el) { return gulp.src(el.dir + ‘src/**/*.coffee’) .pipe(coffee()), .pipe(gulp.src(el.dir + ‘src/**/*.js’), {passthrough: true}) .pipe(uglify()) .pipe(concat(el.bundleName)) }); return merge(streams) .pipe(gulp.dest('build'));
49. Incremental builds
50. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())
51. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())
52. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat()) Too much is going on! Each change: Uglify all the files?
53. Some tasks take long gulp.src(‘scripts/*.js’) .pipe(uglify()) .pipe(gulp.dest()) .pipe(concat())
55. filter files that have changed
56. filter files that have changed do performance heavy operations
57. filter files that have changed do performance heavy operations remember the old files
58. filter files that have changed do performance heavy operations remember the old files and continue with the other ops
59. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); });
60. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); }); we use the cache to check if files have changed
61. gulp.task('scripts', function () { return gulp.src('app/scripts/**/*.js') .pipe(cached('ugly')) .pipe(uglify()) .pipe(remember('ugly')) .pipe(concat('main.min.js')) .pipe(gulp.dest('dist/scripts')); }); once we are done, we remember all the other files we stored in the cache
62. gulp.task('lint', function () { return gulp.src(‘src/**/*.js’) .pipe(jshint()) .pipe(jshint.reporter(‘default’)) .pipe(jshint.reporter(‘fail’)); });
63. gulp.task('lint', function () { return gulp.src(‘src/**/*.js’, { since: gulp.lastRun(‘lint’) }) .pipe(jshint()) .pipe(jshint.reporter(‘default’)) .pipe(jshint.reporter(‘fail’)); });
64. gulp.task('images', function () { return gulp.src(‘src/images/**/*.*’) .pipe(imagemin()) .pipe(gulp.dest(‘dist’)); });
65. gulp.task('images', function () { return gulp.src(‘src/images/**/*.*’) .pipe(newer(‘dist’)) .pipe(imagemin()) .pipe(gulp.dest(‘dist’)); });
66. When architecting Gulp build pipelines…
67. Separate the process from the content
68. Think about how the basic pipeline would look like
69. Think about how your data looks like
70. And then posh your pipes up with stream tools that change your data accordingly
72. @ddprrt